Rust BGZF格式处理库noodles-bgzf的使用指南

BGZF(Blocked GNU Zip Format)是一种常用于生物信息学领域的压缩格式，特别适合处理基因组数据。noodles-bgzf是Rust生态中处理BGZF格式的高效库。

安装

在Cargo.toml中添加依赖：

[dependencies]
noodles-bgzf = "0.8"

基本使用

读取BGZF文件

use noodles_bgzf as bgzf;
use std::fs::File;

fn read_bgzf_file(path: &str) -> std::io::Result<()> {
    let mut reader = File::open(path)
        .map(bgzf::Reader::new)?;

    let mut buf = Vec::new();
    reader.read_to_end(&mut buf)?;

    println!("Read {} bytes", buf.len());
    Ok(())
}

写入BGZF文件

use noodles_bgzf as bgzf;
use std::fs::File;

fn write_bgzf_file(path: &str, data: &[u8]) -> std::io::Result<()> {
    let mut writer = File::create(path)
        .map(bgzf::Writer::new)?;

    writer.write_all(data)?;
    writer.finish()?;  // 确保所有数据被刷新并写入文件
    
    Ok(())
}

高级功能

随机访问

BGZF支持随机访问，这对于大型基因组文件特别有用：

use noodles_bgzf as bgzf;

fn random_access(path: &str) -> std::io::Result<()> {
    let mut reader = bgzf::indexed_reader::Builder::default()
        .build_from_path(path)?;

    // 获取虚拟偏移量(通常来自索引文件)
    let virtual_offset = bgzf::VirtualPosition::try_from(123456)?;
    
    reader.seek(virtual_offset)?;
    
    let mut buf = [0; 1024];
    let bytes_read = reader.read(&mut buf)?;
    
    println!("Read {} bytes from offset {}", bytes_read, virtual_offset);
    Ok(())
}

多线程处理

use noodles_bgzf as bgzf;
use std::{fs::File, thread};

fn parallel_processing(path: &str) -> std::io::Result<()> {
    let file = File::open(path)?;
    let reader = bgzf::Reader::new(file);
    
    // 创建多个解码器来处理不同块
    let mut decoders = reader.multithreaded_decoders(4)?;
    
    let handles: Vec<_> = decoders
        .into_iter()
        .map(|mut decoder| {
            thread::spawn(move || {
                let mut buf = Vec::new();
                decoder.read_to_end(&mut buf).unwrap();
                buf
            })
        })
        .collect();
    
    for handle in handles {
        let data = handle.join().unwrap();
        println!("Decoded {} bytes in thread", data.len());
    }
    
    Ok(())
}

性能提示

1. 对于顺序读取，使用bgzf::Reader即可
2. 对于随机访问，使用bgzf::indexed_reader
3. 大文件处理考虑使用多线程解码器
4. 写入时批量操作比多次小写入更高效

示例：处理VCF文件

use noodles_bgzf as bgzf;
use noodles_vcf as vcf;

fn read_vcf(path: &str) -> std::io::Result<()> {
    let mut reader = File::open(path)
        .map(bgzf::Reader::new)
        .map(vcf::Reader::new)?;

    let header = reader.read_header()?;
    
    for result in reader.records(&header) {
        let record = result?;
        println!("{:?}", record);
    }
    
    Ok(())
}

完整示例代码

下面是一个完整的BGZF文件处理示例，演示了如何读取、写入和随机访问BGZF文件：

use noodles_bgzf as bgzf;
use std::{
    fs::File,
    io::{Read, Write},
    thread,
};

fn main() -> std::io::Result<()> {
    // 1. 写入BGZF文件
    let data = b"Hello, BGZF! This is a test string to demonstrate BGZF compression.";
    write_bgzf_file("example.bgz", data)?;

    // 2. 读取BGZF文件
    read_bgzf_file("example.bgz")?;

    // 3. 随机访问示例
    random_access("example.bgz")?;

    // 4. 多线程处理示例
    parallel_processing("example.bgz")?;

    Ok(())
}

fn write_bgzf_file(path: &str, data: &[u8]) -> std::io::Result<()> {
    println!("Writing BGZF file...");
    let mut writer = File::create(path)
        .map(bgzf::Writer::new)?;

    writer.write_all(data)?;
    writer.finish()?;
    println!("Successfully wrote to {}", path);
    Ok(())
}

fn read_bgzf_file(path: &str) -> std::io::Result<()> {
    println!("\nReading BGZF file...");
    let mut reader = File::open(path)
        .map(bgzf::Reader::new)?;

    let mut buf = String::new();
    reader.read_to_string(&mut buf)?;
    println!("File content: {}", buf);
    Ok(())
}

fn random_access(path: &str) -> std::io::Result<()> {
    println!("\nTesting random access...");
    let mut reader = bgzf::indexed_reader::Builder::default()
        .build_from_path(path)?;

    // 获取文件开头位置
    let virtual_offset = bgzf::VirtualPosition::try_from(0)?;
    
    reader.seek(virtual_offset)?;
    
    let mut buf = [0; 10];  // 只读取前10个字节
    let bytes_read = reader.read(&mut buf)?;
    
    println!("Read {} bytes from offset {}: {:?}", 
             bytes_read, 
             virtual_offset,
             std::str::from_utf8(&buf).unwrap_or("invalid utf-8"));
    Ok(())
}

fn parallel_processing(path: &str) -> std::io::Result<()> {
    println!("\nTesting parallel processing...");
    let file = File::open(path)?;
    let reader = bgzf::Reader::new(file);
    
    // 创建2个解码器
    let mut decoders = reader.multithreaded_decoders(2)?;
    
    let handles: Vec<_> = decoders
        .into_iter()
        .enumerate()
        .map(|(i, mut decoder)| {
            thread::spawn(move || {
                let mut buf = Vec::new();
                decoder.read_to_end(&mut buf).unwrap();
                println!("Thread {} decoded {} bytes", i, buf.len());
                buf
            })
        })
        .collect();
    
    for handle in handles {
        let _ = handle.join().unwrap();
    }
    
    Ok(())
}

这个示例展示了：

1. 如何创建并写入BGZF文件
2. 如何读取BGZF文件内容
3. 如何使用随机访问功能
4. 如何使用多线程并行处理BGZF文件

要运行这个示例，只需将代码保存为main.rs，然后在项目目录下运行cargo run。