The Guide to Perform Text Recognition Using Firebase ML Kit in Flutter

It’s a common misconception among app developers that incorporating machine learning (ML) into any app is difficult. It’s a lot easier to incorporate machine learning into your apps now, thanks to recent advances in ML technologies, even if you don’t have any prior experience in the subject.

With platforms and development tools like Fritz AI and Firebase’s ML Kit, incorporating machine learning into your mobile app development toolset is becoming simpler. By packaging pre-trained machine learning models for usage, these tools hide technological complexity. You may also train and import your own models with them. So, in this blog, we will be exploring how to perform text recognition in Flutter using Firebase’s ML Kit.

Introduction to Firebase ML Kit

Google’s ML Kit SDK is a relatively new product that was introduced in 2018. ML Kit is a software development kit that makes it easier for developers to include machine learning models into mobile apps. In addition, even a junior developer may complete this task. The ML Kit mobile SDK is a powerful yet simple-to-use package that brings Google’s machine learning capabilities to Android and iOS apps. You can obtain the functionality you need with only a few lines of code, whether you’re new to machine learning or a seasoned veteran. You don’t need much expertise with neural networks or model optimization to get started.

ML Kit makes it easy to apply machine learner methods on your applications by integrating Google’s ML technology, including the Google Cloud Vision API, TensorFlow Lite, and Android Neural Networks API in a single SDK. Just a few lines of code allow you to obtain the power of cloud-based processing, mobile on-device optimized models or the flexibility of customized TensorFlow Lite models in real-time.

Benefits of using Firebase ML Kit

Here are some benefits of using Firebase ML Kit:

• It is Production-Ready for common use cases

ML Kit provides a collection of ready-to-use APIs for typical mobile use cases, including text recognition, face detection, landmark identification, barcode scanning, picture labelling, and text language identification. Simply input data into the ML Kit library, and it will provide you with the information you want.

• It features On-device as well as Cloud-based capabilities

The APIs in the ML Kit operates on-device or in the cloud. Its on-device APIs can process your data rapidly and function even if you don’t have access to the internet. And cloud-based APIs, on the other hand, take advantage of Google Cloud’s machine learning technologies to improve accuracy even more.

Read our other post on What’s New in Flutter 2.2.0 to make it more adaptive?

• It allows for the deployment of custom models

If the APIs provided by ML Kit do not meet your needs, you may always introduce your own TensorFlow Lite models. Simply upload your model to Firebase, and it’ll host and provide it to your app for you. ML Kit works with your own model as an API layer, making it easier to execute and utilize.

How to Perform Text Recognition using ML Kit

To make use of all Firebase Machine Learning has to offer, you must first create a Firebase project in the Firebase console. So, create a Firebase Project, connect it with Flutter, and simultaneously add a dependency for Flutter ML Kit in Firebase. As we move past this, we will use the ML Kit addition in the Firebase to recognize text.

• Step:1 – Set up Firebase for the Project on Android/iOS

First, you need to create a new Firebase Project on either the Android or iOS Platform.

Click on Add Project to create the new project and enter the project name. Now, Firebase Console will create a new project in the Dashboard.

• Step:2 –Provide App Description

We will be building a Flutter app containing two different screens, i.e., CameraScreen (to show camera view and click pictures) and a DetailScreen (to show recognized text details from the image).

• Step:3 – Give access to the device camera

To let your Flutter app use the device camera, you need to add a separate plugin to the app. This will allow for retrieval of all camera data available as well as access to it.

Add plugin to your pubspec.yaml file:

camera: ^0.5.7+4

Replace the demo counter app code available in main.dart file with the code given here:

import ‘package:flutter/material.dart’;

import ‘package:camera/camera.dart’;

// Global variable for storing the list of

// cameras available

List<CameraDescription> cameras = [];

Future<void> main() async {

try {

WidgetsFlutterBinding.ensureInitialized();

// Retrieve the device cameras

cameras = await availableCameras();

} on CameraException catch (e) {

print(e);

}

runApp(MyApp());

}

class MyApp extends StatelessWidget {

@override

Widget build(BuildContext context) {

return MaterialApp(

title: ‘ML Vision’,

theme: ThemeData(

primarySwatch: Colors.blue,

),

home: CameraScreen(),

);

}

}

Now, define CameraScreen that will let you see the button for clicking images and see the camera preview.

class CameraScreen extends StatefulWidget {

@override

_CameraScreenState createState() => _CameraScreenState();

}

class _CameraScreenState extends State<CameraScreen> {

@override

Widget build(BuildContext context) {

return Scaffold(

appBar: AppBar(

title: Text(‘ML Vision’),

),

body: Container(),

);

}

}

1. Create CameraControll object

// Inside _CameraScreenState class

CameraController _controller;

Check out here how to create Video-Streaming App with Flutter (Using MUX)

2. Now, initialize the _controller inside the initState() method

@override

void initState() {

super.initState();

_controller = CameraController(cameras[0], ResolutionPreset.medium);

_controller.initialize().then((_) {

if (!mounted) {

return;

}

setState(() {});

});

}

There are two parameters, which are required by CameraController()

• CameraDescription to pass which device camera are you trying to access, where 1 indicated the front camera and 0 represents the back camera.

• ResolutionPreset to pass the quality of the resolution of the images captured with the camera.

3. Dispose the _controller

@override

void dispose() {

_controller.dispose();

super.dispose();

}

4. Now, you will need to define the method _takePicture(), which allows you to take pictures and save them in the file system. it will return the path of the stored image file. So, now you will need two new plugins in the pubspec.yaml file.

path_provider: ^1.6.7

intl: ^0.16.1

• Path_provider is for retrieving paths from the file system

• intl helps in formatting the time and date

import ‘dart:io’;

import ‘package:intl/intl.dart’;

import ‘package:path_provider/path_provider.dart’;

// …

Future<String> _takePicture() async {

// Checking whether the controller is initialized

if (!_controller.value.isInitialized) {

print(“Controller is not initialized”);

return null;

}

// Formatting Date and Time

String dateTime = DateFormat.yMMMd()

.addPattern(‘-‘)

.add_Hms()

.format(DateTime.now())

.toString();

String formattedDateTime = dateTime.replaceAll(‘ ‘, ”);

print(“Formatted: $formattedDateTime”);

// Retrieving the path for saving an image

final Directory appDocDir = await getApplicationDocumentsDirectory();

final String visionDir = ‘${appDocDir.path}/Photos/Vision\ Images’;

await Directory(visionDir).create(recursive: true);

final String imagePath = ‘$visionDir/image_$formattedDateTime.jpg’;

// Checking whether the picture is being taken

// to prevent execution of the function again

// if previous execution has not ended

if (_controller.value.isTakingPicture) {

print(“Processing is in progress…”);

return null;

}

try {

// Captures the image and saves it to the

// provided path

await _controller.takePicture(imagePath);

} on CameraException catch (e) {

print(“Camera Exception: $e”);

return null;

}

return imagePath;

}

5. In this step, we will build UI of the CameraScreen, where UI comprises of a Stack with preview. Also, there is a button to click for capturing images. you will be navigated to DetailScreen.

@override

Widget build(BuildContext context) {

return Scaffold(

appBar: AppBar(

title: Text(‘ML Vision’),

),

body: _controller.value.isInitialized

? Stack(

children: <Widget>[

CameraPreview(_controller),

Padding(

padding: const EdgeInsets.all(20.0),

child: Container(

alignment: Alignment.bottomCenter,

child: RaisedButton.icon(

icon: Icon(Icons.camera),

label: Text(“Click”),

onPressed: () async {

await _takePicture().then((String path) {

if (path != null) {

Navigator.push(

context,

MaterialPageRoute(

builder: (context) => DetailScreen(path),

),

);

}

});

},

),

),

)

],

)

: Container(

color: Colors.black,

child: Center(

child: CircularProgressIndicator(),

),

),

);

}

Read all the points in details at: Text recognition with firebase ml kit

Conclusion

Using Firebase’s ML Kit, you can see how quick and straightforward it is to implement machine learning into your mobile apps. Before putting this app into production, double-check that you have all of the necessary configurations in place using Firebase’s checklist. If you decide to utilize the cloud-based API, make sure you have the appropriate API access privileges. This will prevent your application from failing in production.

At your Team in India, we have a team of flutter experts. If you want to hire Developers or have any questions on what all services we offer at Your team in India– Click here to contact us.

Originally posted on YourTeam in India. Click here to read more.