Flutter Gemma

Gemma is a family of lightweight, state-of-the-art open models built from the same research and technology used to create the Gemini models.

gemma_github_cover

Bring the power of Google’s lightweight Gemma language models directly to your Flutter applications. With Flutter Gemma, you can seamlessly incorporate advanced AI capabilities into your iOS and Android apps, all without relying on external servers.

gemma_github_gif

Features

Local Execution: Run Gemma models directly on user devices for enhanced privacy and offline functionality.
Platform Support: Compatible with both iOS and Android platforms.
Ease of Use: Simple interface for integrating Gemma models into your Flutter projects.

Installation

Add flutter_gemma to your pubspec.yaml:

dependencies:
  flutter_gemma: latest_version

Run flutter pub get to install.

Setup

Download Model

Obtain a pre-trained Gemma model (recommended: 2b or 2b-it) from Kaggle.

Optionally, fine-tune a model for your specific use case.

Platform Specific Setup

iOS

Enable file sharing in info.plist:

<key>UIFileSharingEnabled</key>
<true/>

Change the linking type of pods to static, replace use_frameworks! in Podfile with use_frameworks! :linkage => :static.

Android

If you want to use a GPU to work with the model, you need to add OpenGL support in the AndroidManifest.xml. If you plan to use only the CPU, you can skip this step.

Add to AndroidManifest.xml above tag </application>:

<uses-native-library
    android:name="libOpenCL.so"
    android:required="false"/>
<uses-native-library android:name="libOpenCL-car.so" android:required="false"/>
<uses-native-library android:name="libOpenCL-pixel.so" android:required="false"/>

Web

Web currently works only with GPU backend models; CPU backend models are not supported by Mediapipe yet.

Add dependencies to index.html file in the web folder:

<script type="module">
import { FilesetResolver, LlmInference } from 'https://cdn.jsdelivr.net/npm/@mediapipe/tasks-genai';
window.FilesetResolver = FilesetResolver;
window.LlmInference = LlmInference;
</script>

Prepare Model

Place the model in the assets or upload it to a network drive, such as Firebase.

ATTENTION!! You do not need to load the model every time the application starts; it is stored in the system files and only needs to be done once. Please carefully review the example application. You should use loadAssetModel and loadNetworkModel methods only when you need to upload the model to the device.

Usage

Loading Models from Assets (available only in debug mode)

Don’t forget to add your model to pubspec.yaml.

Loading from assets:

await FlutterGemmaPlugin.instance.loadAssetModel(fullPath: 'model.bin');

Loading from assets with progress status:

FlutterGemmaPlugin.instance.loadAssetModelWithProgress(fullPath: 'model.bin').listen(
  (progress) {
    print('Loading progress: $progress%');
  },
  onDone: () {
    print('Model loading complete.');
  },
  onError: (error) {
    print('Error loading model: $error');
  },
);

Loading Models from Network

For web usage, you will also need to enable CORS (Cross-Origin Resource Sharing) for your network resource. To enable CORS in Firebase, you can follow the guide in the Firebase documentation: Setting up CORS.

Loading from the network:

await FlutterGemmaPlugin.instance.loadNetworkModel(url: 'https://example.com/model.bin');

Loading from the network with progress status:

FlutterGemmaPlugin.instance.loadNetworkModelWithProgress(url: 'https://example.com/model.bin').listen(
  (progress) {
    print('Loading progress: $progress%');
  },
  onDone: () {
    print('Model loading complete.');
  },
  onError: (error) {
    print('Error loading model: $error');
  },
);

Initialize

void main() async {
  WidgetsFlutterBinding.ensureInitialized();
  await FlutterGemmaPlugin.instance.init(
    maxTokens: 512,  // maxTokens is optional, by default the value is 1024
    temperature: 1.0,  // temperature is optional, by default the value is 1.0
    topK: 1,  // topK is optional, by default the value is 1
    randomSeed: 1,  // randomSeed is optional, by default the value is 1
  );

  runApp(const MyApp());
}

Generate Response

final flutterGemma = FlutterGemmaPlugin.instance;
String response = await flutterGemma.getResponse(prompt: 'Tell me something interesting');
print(response);

Generate Response as a Stream

final flutterGemma = FlutterGemmaPlugin.instance;
flutterGemma.getAsyncResponse(prompt: 'Tell me something interesting').listen((String? token) => print(token));

Generate Chat Response

This method works properly only for instruction-tuned models.

final flutterGemma = FlutterGemmaPlugin.instance;
final messages = <Message>[];
messages.add(Message(text: 'Who are you?', isUser: true));
String response = await flutterGemma.getChatResponse(messages: messages);
print(response);
messages.add(Message(text: response));
messages.add(Message(text: 'Really?', isUser: true));
String response = await flutterGemma.getChatResponse(messages: messages);
print(response);

Generate Chat Response as a Stream

This method works properly only for instruction-tuned models.

final flutterGemma = FlutterGemmaPlugin.instance;
final messages = <Message>[];
messages.add(Message(text: 'Who are you?', isUser: true));
flutterGemma.getAsyncChatResponse(messages: messages).listen((String? token) => print(token));

The full and complete example can be found in the example folder.

Important Considerations

Larger models (like 7b and 7b-it) may be too resource-intensive for on-device use.

Coming Soon

LoRA (Low Rank Adaptation) support

Example Code

Here is a complete example of a Flutter app that uses flutter_gemma to generate responses and handle chat interactions.

import 'package:flutter/material.dart';
import 'package:flutter_gemma_example/chat_screen.dart';

void main() async {
  WidgetsFlutterBinding.ensureInitialized();
  runApp(const ChatApp());
}

class ChatApp extends StatelessWidget {
  const ChatApp({super.key});

  @override
  Widget build(BuildContext context) {
    return MaterialApp(
      title: 'Flutter Gemma Example',
      darkTheme: ThemeData(
        brightness: Brightness.dark,
        textTheme: const TextTheme(
          bodyLarge: TextStyle(color: Colors.white),
          bodyMedium: TextStyle(color: Colors.white),
        ),
      ),
      themeMode: ThemeMode.dark,
      home: const SafeArea(child: ChatScreen()),
    );
  }
}

Chat Screen Example

import 'package:flutter/material.dart';
import 'package:flutter_gemma_example/flutter_gemma_plugin.dart';

class ChatScreen extends StatefulWidget {
  const ChatScreen({Key? key}) : super(key: key);

  @override
  _ChatScreenState createState() => _ChatScreenState();
}

class _ChatScreenState extends State<ChatScreen> {
  final TextEditingController _textController = TextEditingController();
  final List<Message> _messages = [];
  final FlutterGemmaPlugin _flutterGemma = FlutterGemmaPlugin.instance;

  @override
  void initState() {
    super.initState();
    _initializeGemma();
  }

  Future<void> _initializeGemma() async {
    await _flutterGemma.init(
      maxTokens: 512,
      temperature: 1.0,
      topK: 1,
      randomSeed: 1,
    );
    await _flutterGemma.loadAssetModel(fullPath: 'model.bin');
  }

  void _handleSubmitted(String text) {
    _textController.clear();
    Message message = Message(text: text, isUser: true);
    setState(() {
      _messages.insert(0, message);
    });
    _generateResponse(message);
  }

  void _generateResponse(Message message) async {
    String response = await _flutterGemma.getChatResponse(messages: [_message]);
    Message botMessage = Message(text: response, isUser: false);
    setState(() {
      _messages.insert(0, botMessage);
    });
  }

  @override
  Widget build(BuildContext context) {
    return Scaffold(
      appBar: AppBar(
        title: const Text('Flutter Gemma Chat'),
      ),
      body: Column(
        children: <Widget>[
          Flexible(
            child: ListView.builder(
              reverse: true,
              padding: const EdgeInsets.all(8.0),
              itemCount: _messages.length,
              itemBuilder: (context, index) {
                return _buildMessage(_messages[index], index == _messages.length - 1);
              },
            ),
          ),
          const Divider(height: 1.0),
          Container(
            decoration: BoxDecoration(color: Theme.of(context).cardColor),
            child: _buildTextComposer(),
          ),
        ],
      ),
    );
  }

  Widget _buildMessage(Message message, bool isLast) {
    return Padding(
      key: Key('message_${message.text}'),
      padding: EdgeInsets.only(right: 16.0, left: 16.0, bottom: isLast ? 20.0 : 0.0),
      child: Row(
        crossAxisAlignment: CrossAxisAlignment.start,
        children: <Widget>[
          if (!message.isUser)
            Container(
              margin: const EdgeInsets.only(right: 16.0),
              child: CircleAvatar(child: Text('G')),
            ),
          Expanded(
            child: Column(
              crossAxisAlignment: message.isUser ? CrossAxisAlignment.end : CrossAxisAlignment.start,
              children: <Widget>[
                Text(
                  message.text,
                  style: TextStyle(fontSize: 16.0, color: message.isUser ? Colors.blue : Colors.black),
                ),
              ],
            ),
          ),
          if (message.isUser)
            Container(
              margin: const EdgeInsets.only(left: 16.0),
              child: CircleAvatar(child: Text('U')),
            ),
        ],
      ),
    );
  }

  Widget _buildTextComposer() {
    return IconTheme(
      data: IconThemeData(color: Theme.of(context).colorScheme.secondary),
      child: Container(
        margin: const EdgeInsets.symmetric(horizontal: 8.0),
        child: Row(
          children: <Widget>[
            Flexible(
              child: TextField(
                controller: _textController,
                onSubmitted: _handleSubmitted,
                decoration: const InputDecoration.collapsed(hintText: 'Send a message'),
              ),
            ),
            IconButton(
              icon: const Icon(Icons.send),
              onPressed: () => _handleSubmitted(_textController.text),
            ),
          ],
        ),
      ),
    );
  }
}

class Message {
  final String text;
  final bool isUser;

  Message({required this.text, required this.isUser});
}

This example demonstrates how to set up a simple chat interface using flutter_gemma to generate responses based on user input. The ChatScreen widget handles user input, displays messages, and generates responses using the Gemma model.

更多关于Flutter未知功能插件flutter_gemma的使用（注：由于介绍为undefined，故功能为假设性描述，便于SEO）的实战系列教程也可以访问 https://www.itying.com/category-92-b0.html

sinazl 1楼•1 天前

当然，以下是一个假设性的Flutter插件flutter_gemma的使用示例代码。由于flutter_gemma是一个假设性的插件，并且具体功能未定义，因此我会创建一个示例性的插件接口和使用案例。请注意，以下代码完全基于假设，并且在实际中你需要根据真实插件的实现进行调整。

假设的`flutter_gemma`插件功能

假设flutter_gemma插件提供以下功能：

显示一个自定义的对话框。
执行一些后台任务（例如网络请求）。

插件的假设性接口

pubspec.yaml

dependencies:
  flutter:
    sdk: flutter
  flutter_gemma:
    git:
      url: https://github.com/example/flutter_gemma.git  # 假设的仓库地址

flutter_gemma.dart（假设的插件实现，通常你会在插件的Dart部分找到这样的代码）

import 'package:flutter/services.dart';

class FlutterGemma {
  static const MethodChannel _channel = MethodChannel('flutter_gemma');

  // 显示自定义对话框
  static Future<void> showCustomDialog({required String message}) async {
    try {
      await _channel.invokeMethod('showCustomDialog', {'message': message});
    } on PlatformException catch (e) {
      print("Failed to show custom dialog: '${e.message}'.");
    }
  }

  // 执行后台任务（例如网络请求）
  static Future<String?> performBackgroundTask({required String url}) async {
    try {
      final Map<String, dynamic> result = await _channel.invokeMethod('performBackgroundTask', {'url': url});
      return result['response'] as String?;
    } on PlatformException catch (e) {
      print("Failed to perform background task: '${e.message}'.");
      return null;
    }
  }
}

在Flutter应用中使用`flutter_gemma`

main.dart

import 'package:flutter/material.dart';
import 'package:flutter_gemma/flutter_gemma.dart';

void main() {
  runApp(MyApp());
}

class MyApp extends StatelessWidget {
  @override
  Widget build(BuildContext context) {
    return MaterialApp(
      title: 'Flutter Gemma Demo',
      theme: ThemeData(
        primarySwatch: Colors.blue,
      ),
      home: MyHomePage(),
    );
  }
}

class MyHomePage extends StatefulWidget {
  @override
  _MyHomePageState createState() => _MyHomePageState();
}

class _MyHomePageState extends State<MyHomePage> {
  String? taskResult;

  void _showDialog() {
    FlutterGemma.showCustomDialog(message: "Hello, this is a custom dialog from flutter_gemma!");
  }

  void _performBackgroundTask() async {
    setState(() {
      taskResult = "Loading...";
    });
    final String? result = await FlutterGemma.performBackgroundTask(url: "https://api.example.com/data");
    setState(() {
      taskResult = result;
    });
  }

  @override
  Widget build(BuildContext context) {
    return Scaffold(
      appBar: AppBar(
        title: Text('Flutter Gemma Demo'),
      ),
      body: Center(
        child: Column(
          mainAxisAlignment: MainAxisAlignment.center,
          children: <Widget>[
            TextButton(
              onPressed: _showDialog,
              child: Text('Show Custom Dialog'),
            ),
            SizedBox(height: 20),
            TextButton(
              onPressed: _performBackgroundTask,
              child: Text('Perform Background Task'),
            ),
            SizedBox(height: 20),
            if (taskResult != null)
              Text(
                "Task Result: $taskResult",
                style: TextStyle(fontSize: 18),
              ),
          ],
        ),
      ),
    );
  }
}

注意事项

平台通道：在实际插件开发中，你会使用MethodChannel（如上面的示例所示）与原生平台代码进行通信。
权限：如果你的插件需要特定的权限（例如网络访问），请确保在原生平台代码中请求这些权限。
错误处理：上面的示例代码包含了基本的错误处理，但在实际开发中你可能需要更详细的错误处理和用户反馈。
插件实现：由于flutter_gemma是假设性的，所以你需要根据实际需求在原生Android和iOS代码中实现相应的功能。

希望这个示例对你有所帮助！

Flutter未知功能插件flutter_gemma的使用（注：由于介绍为undefined，故功能为假设性描述，便于SEO）