lingua-croatian-language-model：克罗地亚语NLP处理库

lingua-croatian-language-model 是 Rust 语言中一个专门用于克罗地亚语自然语言处理的库，提供语言识别和文本处理功能。它是 lingua 语言检测库的克罗地亚语扩展模型。

主要功能

1. 克罗地亚语文本检测
2. 克罗地亚语文本处理
3. 与其他语言的高精度区分

安装方法

在 Cargo.toml 中添加依赖：

[dependencies]
lingua = "1.3"
lingua-croatian-language-model = "0.1"

基本使用方法

1. 语言检测

use lingua::{Language, LanguageDetector, LanguageDetectorBuilder};
use lingua_croatian_language_model::CROATIAN;

fn main() {
    let languages = vec![Language::English, CROATIAN];
    let detector: LanguageDetector = LanguageDetectorBuilder::from_languages(&languages).build();
    
    let text = "Ovo je primjer teksta na hrvatskom jeziku.";
    
    match detector.detect_language_of(text) {
        Some(language) => println!("检测到的语言: {:?}", language),
        None => println!("无法确定语言"),
    }
}

2. 多语言混合检测

use lingua::{LanguageDetectorBuilder, Language};
use lingua_croatian_language_model::CROATIAN;

fn main() {
    let detector = LanguageDetectorBuilder::from_all_languages()
        .with_preloaded_language_models()
        .build();
    
    let croatian_text = "Dobar dan, kako ste danas?";
    let english_text = "This is an example text in English.";
    let mixed_text = "Ovo je mješoviti tekst. This is a mixed text.";
    
    println!("克罗地亚语检测: {:?}", detector.detect_language_of(croatian_text));
    println!("英语检测: {:?}", detector.detect_language_of(english_text));
    println!("混合文本检测: {:?}", detector.detect_language_of(mixed_text));
}

3. 置信度检测

use lingua::{LanguageDetectorBuilder, Language};
use lingua_croatian_language_model::CROATIAN;

fn main() {
    let languages = vec![Language::English, CROATIAN];
    let detector = LanguageDetectorBuilder::from_languages(&languages)
        .with_preloaded_language_models()
        .build();
    
    let text = "Zagreb je glavni grad Hrvatske.";
    
    let confidence_values = detector.compute_language_confidence_values(text);
    
    for (language, confidence) in confidence_values {
        println!("{:?}: {:.4}", language, confidence);
    }
}

高级功能

批量检测

use lingua::{LanguageDetectorBuilder, Language};
use lingua_croatian_language_model::CROATIAN;

fn main() {
    let detector = LanguageDetectorBuilder::from_all_languages()
        .with_preloaded_language_models()
        .build();
    
    let texts = vec![
        "Ovo je tekst na hrvatskom.",
        "This is English text.",
        "Ovo je mješoviti text. With some English.",
    ];
    
    let results = detector.detect_languages_in(texts);
    
    for (i, language) in results.into_iter().enumerate() {
        println!("文本 {}: {:?}", i+1, language);
    }
}

性能优化

对于大量文本处理，可以预先加载语言模型：

use lingua::{LanguageDetectorBuilder, Language};
use lingua_croatian_language_model::CROATIAN;

fn main() {
    let detector = LanguageDetectorBuilder::from_languages(&[CROATIAN, Language::English])
        .with_preloaded_language_models()
        .build();
    
    // 后续检测会更快
}

注意事项

1. 克罗地亚语与塞尔维亚语、波斯尼亚语等相似语言可能有混淆
2. 短文本检测准确度可能较低
3. 专业术语或方言可能影响检测结果

这个库为Rust开发者提供了处理克罗地亚语文本的便捷工具，特别适合需要多语言支持的应用场景。

完整示例代码

//! 克罗地亚语NLP处理完整示例
use lingua::{Language, LanguageDetector, LanguageDetectorBuilder};
use lingua_croatian_language_model::CROATIAN;

fn main() {
    // 1. 初始化语言检测器
    let languages = vec![
        Language::English,
        Language::Spanish,
        CROATIAN,
        Language::French,
        Language::German
    ];
    
    let detector: LanguageDetector = LanguageDetectorBuilder::from_languages(&languages)
        .with_preloaded_language_models()
        .build();
    
    // 2. 检测单个文本的语言
    let croatian_text = "Danas je lijep dan u Zagrebu.";
    if let Some(lang) = detector.detect_language_of(croatian_text) {
        println!("检测到语言: {:?}", lang);
    }
    
    // 3. 获取置信度分数
    let mixed_text = "Ovo je tekst na hrvatskom. This is English text.";
    let confidences = detector.compute_language_confidence_values(mixed_text);
    println!("\n语言置信度:");
    for (lang, score) in confidences {
        println!("{:?}: {:.2}%", lang, score * 100.0);
    }
    
    // 4. 批量检测多语言文本
    let texts = vec![
        "Bok! Kako si danas?",
        "Good morning! How are you?",
        "Dobro jutro. Have a nice day!",
        "Ovo je kombinacija hrvatskog i engleskog."
    ];
    
    println!("\n批量检测结果:");
    for (i, result) in detector.detect_languages_in(texts).into_iter().enumerate() {
        println!("文本 {}: {:?}", i + 1, result);
    }
    
    // 5. 性能优化示例
    let optimized_detector = LanguageDetectorBuilder::from_languages(&[CROATIAN, Language::English])
        .with_preloaded_language_models()
        .build();
    
    let long_text = std::iter::repeat("Ovo je dugi hrvatski tekst. ")
        .take(100)
        .collect::<String>();
    
    println!("\n优化后检测结果:");
    println!("{:?}", optimized_detector.detect_language_of(&long_text));
}

这个完整示例演示了:

1. 初始化支持多种语言(包括克罗地亚语)的检测器
2. 单个文本语言检测
3. 获取检测结果的置信度分数
4. 批量检测多个文本
5. 性能优化后的检测器使用

输出结果会显示每个文本的检测结果以及对应的置信度分数。