Flutter手写数字识别插件digital_ink_recognition

Flutter手写数字识别插件digital_ink_recognition_mlkit的使用

概述

这是一个用于在Flutter中使用Google的ML Kit Digital Ink Recognition来识别手写文本（支持数百种语言）和分类草图的插件。此插件并未得到Google官方的支持或维护，而是由一群热衷于机器学习的开发者开发，旨在将Google的原生API暴露给Flutter。

平台通道

由于ML Kit仅支持iOS和Android平台，因此该插件使用了Flutter的平台通道机制。所有的机器学习处理都是通过原生代码进行的，而不是在Flutter/Dart环境中进行。这意味着所有的方法调用都会通过MethodChannel（Android）或FlutterMethodChannel（iOS）传递到原生平台，并在那里执行。插件的作用就像一个桥梁，连接你的应用和Google的原生ML Kit API。因此，在调试错误时，你需要理解这一概念。

要求

iOS

最低iOS部署目标：12.0
Xcode 13.2.1 或更新版本
Swift 5
ML Kit不支持32位架构（i386 和 armv7），仅支持64位架构（x86_64 和 arm64）。请确保你的设备具有所需的设备能力。更多详情可以查看这里。

由于ML Kit不支持32位架构（i386 和 armv7），你必须在Xcode中排除armv7架构，以便运行flutter build ios或flutter build ipa。更多详情可以查看这里。

请转到Project > Runner > Building Settings > Excluded Architectures > Any SDK > armv7。

你的Podfile应该看起来像这样：

platform :ios, '12.0'  # or newer version

...

# 添加以下行：
$iOSVersion = '12.0'  # or newer version

post_install do |installer|
  # 添加这些行：
  installer.pods_project.build_configurations.each do |config|
    config.build_settings["EXCLUDED_ARCHS[sdk=*]"] = "armv7"
    config.build_settings['IPHONEOS_DEPLOYMENT_TARGET'] = $iOSVersion
  end
  
  installer.pods_project.targets.each do |target|
    flutter_additional_ios_build_settings(target)
    
    # 添加这些行：
    target.build_configurations.each do |config|
      if Gem::Version.new($iOSVersion) > Gem::Version.new(config.build_settings['IPHONEOS_DEPLOYMENT_TARGET'])
        config.build_settings['IPHONEOS_DEPLOYMENT_TARGET'] = $iOSVersion
      end
    end
    
  end
end

请注意，最低的IPHONEOS_DEPLOYMENT_TARGET是12.0，你可以设置为更新的版本但不能低于这个版本。

Android

minSdkVersion: 21
targetSdkVersion: 33
compileSdkVersion: 33

使用

导入

首先，你需要导入插件：

import 'package:digital_ink_recognition_mlkit/digital_ink_recognition_mlkit.dart';

创建实例

创建一个DigitalInkRecognizer实例：

String languageCode; // BCP-47 Code from https://developers.google.com/ml-kit/vision/digital-ink-recognition/base-models?hl=zh
final digitalInkRecognizer = DigitalInkRecognizer(languageCode: languageCode);

下载模型

下载模型以供后续使用：

final bool response = await digitalInkRecognizer.downLoadModel(model);

如果模型成功下载或者已经下载，则返回true。如果下载失败则会抛出异常。

删除模型

删除已下载的模型：

final bool response = await digitalInkRecognizer.deleteModel(model);

如果模型删除成功或不存在，则返回true。

处理墨迹

处理墨迹并识别文本：

final p1 = StrokePoint(x: x1, y: y1, t: DateTime.now().millisecondsSinceEpoch); // 确保`t`是一个long
final p2 = StrokePoint(x: x1, y: y1, t: DateTime.now().millisecondsSinceEpoch); // 确保`t`是一个long

Stroke stroke1 = Stroke(); // 它包含所有的StrokePoint
stroke1.points = [p1, p2, ...]

Ink ink = Ink(); // 它包含所有的Stroke
ink.strokes = [stroke1, stroke2, ...];

final List<RecognitionCandidate> candidates = await digitalInkRecognizer.recognize(ink);

for (final candidate in candidates) {
  final text = candidate.text;
  final score = candidate.score;
}

确保在处理任何Ink之前先下载语言模型。

为了提高文本识别的准确性，你可以设置书写区域和预上下文。更多信息可以查看这里。

String preContext;
double width;
double height;
final context = DigitalInkRecognitionContext(
  preContext: preContext,
  writingArea: WritingArea(width: width, height: height),
);

final List<RecognitionCandidate> candidates = await digitalInkRecognizer.recognize(ink, context: context);

释放资源

在不再需要时释放资源：

digitalInkRecognizer.close();

完整示例代码

以下是完整的示例代码，展示了如何使用digital_ink_recognition_mlkit插件进行手写数字识别：

// ignore_for_file: avoid_print

import 'package:flutter/material.dart' hide Ink;
import 'dart:async';

import 'package:digital_ink_recognition_mlkit/digital_ink_recognition_mlkit.dart';

void main() {
  runApp(const MyApp());
}

class MyApp extends StatefulWidget {
  const MyApp({super.key});

  [@override](/user/override)
  State<MyApp> createState() => _MyAppState();
}

class _MyAppState extends State<MyApp> with TickerProviderStateMixin {
  final _digitalInkRecognitionMlkitPlugin =
      DigitalInkRecognizer(languageCode: 'ja');

  late AnimationController animationController;
  late Animation<double> animation;

  [@override](/user/override)
  void initState() {
    super.initState();
    animationController =
        AnimationController(vsync: this, duration: const Duration(seconds: 2));
    animation = Tween<double>(begin: 0, end: 1).animate(animationController);
    animationController.repeat();
  }

  // 平台消息是异步的，所以我们初始化时使用async方法。
  Future<void> download() async {
    final result = await _digitalInkRecognitionMlkitPlugin.downLoadModel();
    print(result.toString());
  }

  Future<void> deleteModel() async {
    final result = await _digitalInkRecognitionMlkitPlugin.deleteModel();
    print(result);
  }

  [@override](/user/override)
  Widget build(BuildContext context) {
    return MaterialApp(
      home: DigitalInkView(),
    );
  }
}

class DigitalInkView extends StatefulWidget {
  [@override](/user/override)
  State<DigitalInkView> createState() => _DigitalInkViewState();
}

class _DigitalInkViewState extends State<DigitalInkView> {
  var _language = 'en';
  // 代码来自 https://developers.google.com/ml-kit/vision/digital-ink-recognition/base-models?hl=zh
  final _languages = [
    'en',
    'es',
    'fr',
    'hi',
    'it',
    'ja',
    'pt',
    'ru',
    'zh-Hani',
  ];
  late DigitalInkRecognizer _digitalInkRecognizer =
      DigitalInkRecognizer(languageCode: _language);
  final Ink _ink = Ink();
  List<StrokePoint> _points = [];
  String _recognizedText = '';

  [@override](/user/override)
  void dispose() {
    _digitalInkRecognizer.close();
    super.dispose();
  }

  [@override](/user/override)
  Widget build(BuildContext context) {
    return Scaffold(
      appBar: AppBar(title: Text('Digital Ink Recognition')),
      body: SafeArea(
        child: Column(
          children: [
            SizedBox(height: 8),
            Padding(
              padding: EdgeInsets.symmetric(horizontal: 16),
              child: Row(
                mainAxisAlignment: MainAxisAlignment.spaceBetween,
                children: [
                  _buildDropdown(),
                  ElevatedButton(
                    onPressed: _downloadModel,
                    child: Icon(Icons.download),
                  ),
                  ElevatedButton(
                    onPressed: _deleteModel,
                    child: Icon(Icons.delete),
                  ),
                ],
              ),
            ),
            SizedBox(height: 8),
            Padding(
              padding: EdgeInsets.symmetric(horizontal: 16),
              child: Row(
                mainAxisAlignment: MainAxisAlignment.spaceBetween,
                children: [
                  ElevatedButton(
                    onPressed: _recogniseText,
                    child: Text('Read Text'),
                  ),
                  ElevatedButton(
                    onPressed: _clearPad,
                    child: Text('Clear Pad'),
                  ),
                ],
              ),
            ),
            Expanded(
              child: GestureDetector(
                onPanStart: (DragStartDetails details) {
                  _ink.strokes.add(Stroke());
                },
                onPanUpdate: (DragUpdateDetails details) {
                  setState(() {
                    final RenderObject? object = context.findRenderObject();
                    final localPosition = (object as RenderBox?)
                        ?.globalToLocal(details.localPosition);
                    if (localPosition != null) {
                      _points = List.from(_points)
                        ..add(StrokePoint(
                          x: localPosition.dx,
                          y: localPosition.dy,
                          t: DateTime.now().millisecondsSinceEpoch,
                        ));
                    }
                    if (_ink.strokes.isNotEmpty) {
                      _ink.strokes.last.points = _points.toList();
                    }
                  });
                },
                onPanEnd: (DragEndDetails details) {
                  _points.clear();
                  setState(() {});
                },
                child: CustomPaint(
                  painter: Signature(ink: _ink),
                  size: Size.infinite,
                ),
              ),
            ),
            if (_recognizedText.isNotEmpty)
              Text(
                'Candidates: $_recognizedText',
                style: TextStyle(fontSize: 23),
              ),
          ],
        ),
      ),
    );
  }

  Widget _buildDropdown() => DropdownButton<String>(
        value: _language,
        icon: const Icon(Icons.arrow_downward),
        elevation: 16,
        style: const TextStyle(color: Colors.blue),
        underline: Container(
          height: 2,
          color: Colors.blue,
        ),
        onChanged: (String? lang) {
          if (lang != null) {
            setState(() {
              _language = lang;
              _digitalInkRecognizer.close();
              _digitalInkRecognizer =
                  DigitalInkRecognizer(languageCode: _language);
            });
          }
        },
        items: _languages.map<DropdownMenuItem<String>>((lang) {
          return DropdownMenuItem<String>(
            value: lang,
            child: Text(lang),
          );
        }).toList(),
      );

  void _clearPad() {
    setState(() {
      _ink.strokes.clear();
      _points.clear();
      _recognizedText = '';
    });
  }

  Future<void> _deleteModel() async {
    _digitalInkRecognizer
        .deleteModel()
        .then((value) => value ? 'success' : 'failed');
  }

  Future<void> _downloadModel() async {
    _digitalInkRecognizer
        .downLoadModel()
        .then((value) => value ? 'success' : 'failed');
  }

  Future<void> _recogniseText() async {
    showDialog(
        context: context,
        builder: (context) => AlertDialog(
              title: Text('Recognizing'),
            ),
        barrierDismissible: true);
    try {
      final candidates = await _digitalInkRecognizer.recognize(_ink);
      _recognizedText = '';
      for (final candidate in candidates) {
        _recognizedText += '\n${candidate.text}';
      }
      setState(() {});
    } catch (e) {
      ScaffoldMessenger.of(context).showSnackBar(SnackBar(
        content: Text(e.toString()),
      ));
    }
    Navigator.pop(context);
  }
}

class Signature extends CustomPainter {
  Ink ink;

  Signature({required this.ink});

  [@override](/user/override)
  void paint(Canvas canvas, Size size) {
    final Paint paint = Paint()
      ..color = Colors.blue
      ..strokeCap = StrokeCap.round
      ..strokeWidth = 4.0;

    for (final stroke in ink.strokes) {
      for (int i = 0; i < stroke.points.length - 1; i++) {
        final p1 = stroke.points[i];
        final p2 = stroke.points[i + 1];
        canvas.drawLine(Offset(p1.x.toDouble(), p1.y.toDouble()),
            Offset(p2.x.toDouble(), p2.y.toDouble()), paint);
      }
    }
  }

  [@override](/user/override)
  bool shouldRepaint(Signature oldDelegate) => true;
}

更多关于Flutter手写数字识别插件digital_ink_recognition_mlkit的使用的实战教程也可以访问 https://www.itying.com/category-92-b0.html

ionicwang 1楼

更多关于Flutter手写数字识别插件digital_ink_recognition_mlkit的使用的实战系列教程也可以访问 https://www.itying.com/category-92-b0.html

当然，以下是如何在Flutter应用中使用digital_ink_recognition_mlkit插件来进行手写数字识别的示例代码。这个插件依赖于Google的ML Kit，用于识别用户手写的数字。

首先，确保你的Flutter项目已经配置好，并且已经添加了digital_ink_recognition_mlkit依赖。你可以在pubspec.yaml文件中添加以下依赖：

dependencies:
  flutter:
    sdk: flutter
  digital_ink_recognition_mlkit: ^最新版本号 # 请替换为最新版本号

然后运行flutter pub get来安装依赖。

接下来，我们编写Flutter代码来实现手写数字识别。以下是一个简单的示例，展示了如何使用digital_ink_recognition_mlkit插件：

import 'package:flutter/material.dart';
import 'package:digital_ink_recognition_mlkit/digital_ink_recognition_mlkit.dart';

void main() {
  runApp(MyApp());
}

class MyApp extends StatelessWidget {
  @override
  Widget build(BuildContext context) {
    return MaterialApp(
      title: 'Handwritten Digit Recognition',
      theme: ThemeData(
        primarySwatch: Colors.blue,
      ),
      home: HandwrittenDigitRecognitionPage(),
    );
  }
}

class HandwrittenDigitRecognitionPage extends StatefulWidget {
  @override
  _HandwrittenDigitRecognitionPageState createState() => _HandwrittenDigitRecognitionPageState();
}

class _HandwrittenDigitRecognitionPageState extends State<HandwrittenDigitRecognitionPage> {
  final List<Offset> _points = [];
  RecognitionResult? _recognitionResult;

  void _addPoint(Offset point) {
    setState(() {
      _points.add(point);
    });
  }

  Future<void> _recognizeDigit() async {
    if (_points.isEmpty) return;

    try {
      final List<List<Offset>> ink = [_points];
      final result = await DigitalInkRecognizer.recognizeDigits(ink);

      setState(() {
        _recognitionResult = result;
      });
    } catch (e) {
      print('Recognition failed: $e');
    }
  }

  @override
  Widget build(BuildContext context) {
    return Scaffold(
      appBar: AppBar(
        title: Text('Handwritten Digit Recognition'),
      ),
      body: Column(
        children: [
          Expanded(
            child: GestureDetector(
              onPanUpdate: (details) => _addPoint(details.localPosition),
              onPanEnd: (details) async {
                _addPoint(details.localPosition); // Add the last point when pan ends
                await _recognizeDigit();
              },
              child: CustomPaint(
                size: Size.infinite,
                painter: DigitPainter(_points),
              ),
            ),
          ),
          if (_recognitionResult != null)
            Padding(
              padding: const EdgeInsets.all(16.0),
              child: Text(
                'Recognized Digit: ${_recognitionResult!.label ?? 'Unknown'}'
                    '\nConfidence: ${_recognitionResult!.confidence.toStringAsFixed(2)}',
                style: TextStyle(fontSize: 24),
              ),
            ),
        ],
      ),
      floatingActionButton: FloatingActionButton(
        onPressed: () {
          setState(() {
            _points.clear();
            _recognitionResult = null;
          });
        },
        tooltip: 'Clear',
        child: Icon(Icons.clear),
      ),
    );
  }
}

class DigitPainter extends CustomPainter {
  final List<Offset> points;

  DigitPainter(this.points);

  @override
  void paint(Canvas canvas, Size size) {
    final Paint paint = Paint()
      ..color = Colors.black
      ..strokeWidth = 4.0
      ..style = PaintingStyle.stroke;

    if (points.isNotEmpty) {
      for (int i = 0; i < points.length - 1; i++) {
        canvas.drawLine(points[i], points[i + 1], paint);
      }
    }
  }

  @override
  bool shouldRepaint(covariant CustomPainter oldDelegate) {
    return oldDelegate != this;
  }
}

这个示例应用包含一个画布，用户可以在上面绘制数字。当用户完成绘制后（即手势结束），应用会使用digital_ink_recognition_mlkit插件来识别绘制的数字，并显示识别结果。

GestureDetector用于监听用户的绘制手势。
CustomPaint和DigitPainter用于在画布上绘制用户绘制的点。
_recognizeDigit方法调用DigitalInkRecognizer.recognizeDigits来识别绘制的数字。
识别结果会显示在页面上。