Flutter音频处理插件flutter_sound_processing的使用

Flutter音频处理插件flutter_sound_processing的使用

Flutter库flutter_sound_processing提供了音频处理功能，包括从音频信号计算特征矩阵。它使Flutter开发者能够从音频数据中提取有意义的信息，如梅尔频率倒谱系数（MFCC）和频谱特征。

使用方法

要使用此插件，将flutter_sound_processing添加到你的pubspec.yaml文件中，并运行以下命令：

flutter pub add flutter_sound_processing

API 参考

getFeatureMatrix 函数基于提供的音频信号和参数计算特征矩阵。

• signals: 代表音频信号的双精度浮点数列表。
• sampleRate: 音频信号的采样率。
• hopLength: 分析使用的跳长（以样本为单位）。
• nMels: 在分析中使用的Mel滤波器的数量。
• fftSize: 用于频谱分析的FFT大小。
• mfcc: 要计算的MFCC系数数量。
• 返回一个Future。

示例代码

以下是一个完整的示例，展示了如何使用flutter_sound_processing插件来处理音频信号并计算特征矩阵。

import 'dart:typed_data';

import 'package:flutter/material.dart';
import 'dart:async';

import 'package:flutter_sound/flutter_sound.dart';
import 'package:flutter_sound_processing/flutter_sound_processing.dart';
import 'package:permission_handler/permission_handler.dart';

const int bufferSize = 7839;
const int sampleRate = 16000;
const int hopLength = 350;
const int nMels = 40;
const int fftSize = 512;
const int mfcc = 40;

void main() {
  runApp(const MyApp());
}

class MyApp extends StatefulWidget {
  const MyApp({super.key});

  [@override](/user/override)
  State<MyApp> createState() => _MyAppState();
}

class _MyAppState extends State<MyApp> {
  final _flutterSoundProcessingPlugin = FlutterSoundProcessing();
  final _mRecorder = FlutterSoundRecorder();
  final signals = List<double>.filled(
    bufferSize,
    0,
  );
  late StreamSubscription _mRecordingDataSubscription;
  late StreamController<Food> _recordingDataController;
  int indexSignal = 0;
  bool running = false;

  [@override](/user/override)
  void initState() {
    super.initState();
  }

  [@override](/user/override)
  Future<void> dispose() async {
    await closeRecorder();

    super.dispose();
  }

  Future<void> closeRecorder() async {
    await _mRecordingDataSubscription.cancel();
    await _recordingDataController.close();
    await _mRecorder.closeRecorder();
  }

  Future<void> start() async {
    final status = await Permission.microphone.request();

    if (!status.isGranted) {
      return;
    }

    setState(() {
      running = true;
    });

    _recordingDataController = StreamController<Food>();
    await _mRecorder.openRecorder();

    _mRecordingDataSubscription = _recordingDataController.stream.listen(
      (dynamic buffer) async {
        if (buffer is FoodData && buffer.data != null) {
          final samples = Uint8List.fromList(buffer.data!);
          final byteData = samples.buffer.asByteData();

          for (var offset = 0; offset < samples.length; offset += 2) {
            signals[indexSignal] =
                byteData.getInt16(offset, Endian.little).toDouble();
            indexSignal++;

            if (indexSignal == bufferSize) {
              indexSignal = 0;

              final featureMatrix =
                  await _flutterSoundProcessingPlugin.getFeatureMatrix(
                signals: signals,
                fftSize: fftSize,
                hopLength: hopLength,
                nMels: nMels,
                mfcc: mfcc,
                sampleRate: sampleRate,
              );

              print(featureMatrix?.toList());
            }
          }
        }
      },
    );

    await _mRecorder.startRecorder(
      toStream: _recordingDataController.sink,
      codec: Codec.pcm16,
      numChannels: 1,
      sampleRate: sampleRate,
    );
  }

  void pause() {
    _mRecordingDataSubscription.pause();

    setState(() {
      running = false;
    });
  }

  [@override](/user/override)
  Widget build(BuildContext context) {
    return MaterialApp(
      home: Scaffold(
        appBar: AppBar(
          title: const Text('插件示例应用'),
        ),
        body: Center(
          child: ElevatedButton(
            onPressed: running ? pause : start,
            child: Text(running ? '暂停' : '开始'),
          ),
        ),
      ),
    );
  }
}