Flutter机器学习插件pytorch_mobile的使用

pytorch_mobile 是一个Flutter插件，它允许开发者在移动应用中加载和运行PyTorch模型。该插件支持Android和iOS平台，并提供了简单易用的API来执行模型推理。

使用方法

安装

要使用此插件，您需要将 pytorch_mobile 作为依赖项添加到您的 pubspec.yaml 文件中。同时创建一个 assets 文件夹用于存放PyTorch模型文件和标签文件（如果需要），并相应地修改 pubspec.yaml 文件：

assets:
 - assets/models/model.pt
 - assets/labels.csv

安装依赖：

flutter pub get

导入库

在Dart代码中导入 pytorch_mobile 库：

import 'package:pytorch_mobile/pytorch_mobile.dart';

加载模型

可以加载自定义模型或图像分类模型：

• 自定义模型

Model customModel = await PyTorchMobile.loadModel('assets/models/custom_model.pt');

• 图像模型

Model imageModel = await PyTorchMobile.loadModel('assets/models/resnet18.pt');

获取预测结果

• 获取自定义预测

List prediction = await customModel.getPrediction([1, 2, 3, 4], [1, 2, 2], DType.float32);

• 获取图像预测

String prediction = await _imageModel.getImagePrediction(image, 224, 224, "assets/labels/labels.csv");

• 带自定义均值和标准差的图像预测

final mean = [0.5, 0.5, 0.5];
final std = [0.5, 0.5, 0.5];
String prediction = await _imageModel.getImagePrediction(image, 224, 224, "assets/labels/labels.csv", mean: mean, std: std);

示例Demo

以下是一个完整的示例应用程序，展示了如何在Flutter中使用 pytorch_mobile 插件进行图像分类和自定义输入预测。

import 'dart:io';

import 'package:flutter/material.dart';
import 'package:flutter/services.dart';
import 'package:image_picker/image_picker.dart';

import 'package:pytorch_mobile/pytorch_mobile.dart';
import 'package:pytorch_mobile/model.dart';
import 'package:pytorch_mobile/enums/dtype.dart';

void main() => runApp(MyApp());

class MyApp extends StatefulWidget {
  @override
  _MyAppState createState() => _MyAppState();
}

class _MyAppState extends State<MyApp> {
  Model? _imageModel, _customModel;

  String? _imagePrediction;
  List? _prediction;
  File? _image;
  ImagePicker _picker = ImagePicker();

  @override
  void initState() {
    super.initState();
    loadModel();
  }

  // 加载模型
  Future loadModel() async {
    String pathImageModel = "assets/models/resnet.pt";
    String pathCustomModel = "assets/models/custom_model.pt";
    try {
      _imageModel = await PyTorchMobile.loadModel(pathImageModel);
      _customModel = await PyTorchMobile.loadModel(pathCustomModel);
    } on PlatformException {
      print("only supported for android and ios so far");
    }
  }

  // 运行图像模型
  Future runImageModel() async {
    // 拍照或从相册选择图片
    final PickedFile? image = await _picker.getImage(
        source: (Platform.isIOS ? ImageSource.gallery : ImageSource.camera),
        maxHeight: 224,
        maxWidth: 224);
    // 获取预测结果
    _imagePrediction = await _imageModel!.getImagePrediction(
        File(image!.path), 224, 224, "assets/labels/labels.csv");

    setState(() {
      _image = File(image.path);
    });
  }

  // 运行自定义模型
  Future runCustomModel() async {
    _prediction = await _customModel!
        .getPrediction([1, 2, 3, 4], [1, 2, 2], DType.float32);

    setState(() {});
  }

  @override
  Widget build(BuildContext context) {
    return MaterialApp(
      home: Scaffold(
        appBar: AppBar(
          title: const Text('Pytorch Mobile Example'),
        ),
        body: Column(
          mainAxisAlignment: MainAxisAlignment.center,
          children: <Widget>[
            _image == null ? Text('No image selected.') : Image.file(_image!),
            Center(
              child: Visibility(
                visible: _imagePrediction != null,
                child: Text("$_imagePrediction"),
              ),
            ),
            Center(
              child: TextButton(
                onPressed: runImageModel,
                child: Icon(
                  Icons.add_a_photo,
                  color: Colors.grey,
                ),
              ),
            ),
            TextButton(
              onPressed: runCustomModel,
              style: TextButton.styleFrom(
                backgroundColor: Colors.blue,
              ),
              child: Text(
                "Run custom model",
                style: TextStyle(
                  color: Colors.white,
                ),
              ),
            ),
            Center(
              child: Visibility(
                visible: _prediction != null,
                child: Text(_prediction != null ? "${_prediction![0]}" : ""),
              ),
            )
          ],
        ),
      ),
    );
  }
}

如果您有任何问题或建议，请联系：fynnmaarten.business@gmail.com

希望这个指南对您有所帮助！