Live Text API in iOS 16 — Scanning Data With the Camera in SwiftUI
How to use DataScannerViewController in SwiftUI
WWDC22 brings brilliant Live Text data scanning tools which let users scan text and codes with the camera, similar to the Live Text interface in the Camera app for iOS and iPadOS developers.
In this article, I will focus on the new API called DataScannerViewController and share my experience with how to embed this API into your SwiftUI code. The following photo shows today’s demo.
Provide a reason for using the camera
Because this demo app can only be running on real-world devices, so to access users’ cameras you’d better provide a clear statement of why you need to access their cameras.
You provide the reason for using the camera in the Xcode project configuration. Add the NSCameraUsageDescription key to the target’s Information Property List in Xcode.
The following steps I provide come from Apple’s official document: Scanning data with the camera.
- In the project editor, select the target and click Info.
- Under Custom iOS Target Properties, click the Plus button in any row.
- From the pop-up menu in the Key column, choose Privacy — Camera Usage Description.
- In the Value column, enter the reason, such as “Your camera is used to scan text and codes.”
Creating the Main View
In your ContentView.swift file, add the following code:
VStack {
Text(scanResults)
.padding() Button {
// Enable Scan Document Action
} label: {
Text("Tap to Scan Documents")
.foregroundColor(.white)
.frame(width: 300, height: 50)
.background(Color.blue)
.cornerRadius(10)
}
}
Where scanResult is a String variable which represents the camera scan result that will use to illustrate what the camera sees during scanning.
@State private var scanResults: String = ""The button here is used to present the scanning view.
When someone taps the button, the device will be ready to scan data. However, not all devices support this function. Or even when the device supports scan data, when the user denies providing the camera usage permission, things may fail when tapping the button.
In such cases, I will provide an alert view that shows a message when the device is not capable of scanning data.
@State private var showDeviceNotCapacityAlert = falseThe code above provides a variable to choose whether shows the alert view or not. If showDeviceNotCapacityAlert is true, then shows the alert view.
Add the following code behind the VStack code:
.alert("Scanner Unavailable", isPresented: $showDeviceNotCapacityAlert, actions: {})Finally, when the device is ready for scanning data, we need to present a scanning view, like the above code, and add the following code to your ContentView.swift file:
@State private var showCameraScannerView = falsevar body: some View {
VStack {
...
}
.sheet(isPresented: $showCameraScannerView) {
// Present the scanning view
}
...
}
Now thing what we left is when we tap the button, if the device is not capable for scanning data, an alert view will show. We use isDeviceCapacity to check whether the device can use this function or not.
@State private var isDeviceCapacity = falseNow add the following code inside the Button action:
if isDeviceCapacity {
self.showCameraScannerView = true
} else {
self.showDeviceNotCapacityAlert = true
}Create a Camera Scanner View
Create a new swift file and named it CameraScanner.swift . Add the following code here:
struct CameraScanner: View {
@Environment(\.presentationMode) var presentationMode
var body: some View {
NavigationView {
Text("Scanning View")
.toolbar {
ToolbarItem(placement: .navigationBarLeading) {
Button {
self.presentationMode.wrappedValue.dismiss()
} label: {
Text("Cancel")
}
}
}
.interactiveDismissDisabled(true)
}
}
}Handle when the scanner becomes unavailable
When the app is opened by users, we need to check whether the scanner is available or not. Simply add the following code then this app is opened:
var body: some View {
VStack {
...
}
.onAppear {
isDeviceCapacity = (DataScannerViewController.isSupported && DataScannerViewController.isAvailable)
}
...
}Only if the above convenience property that checks both values returns true we can open the scanning view.
Create a data scanner view controller
To implement a view controller that can be used in a SwiftUI View, first of all we need to use UIViewControllerRepresentable to wrap a UIKit view controller so that it can be used inside SwiftUI. Create a new Swift file named CameraScannerViewController.swift and simply add the following code here:
import SwiftUI
import UIKit
import VisionKitstruct CameraScannerViewController: UIViewControllerRepresentable {
func makeUIViewController(context: Context) -> DataScannerViewController {
let viewController = DataScannerViewController(recognizedDataTypes: [.text()],qualityLevel: .fast,recognizesMultipleItems: false, isHighFrameRateTrackingEnabled: false, isHighlightingEnabled: true) return viewController
} func updateUIViewController(_ viewController: DataScannerViewController, context: Context) {}}
The above code returns a view controller which provides the interface for scanning items in the live video. In this article, I will only focus on scanning text data so the recognizedDataTypes here only contains .text() property.
Handle Delegate protocol
After creating the view controller and before we present it, set its delegate to an object in this app that handles the DataScannerViewControllerDelegate protocol callbacks.
In UIKit it’s easy to write the following code:
viewController.delegate = selfLuckily, it can be very convenient to handle the DataScannerViewControllerDelegate in SwiftUI.
SwiftUI’s coordinators are designed to act as delegates for UIKit view controllers. Remember, “delegates” are objects that respond to events that occur elsewhere. For example, UIKit lets us attach a delegate object to its text field view, and that delegate will be notified when the user types anything, when they press return, and so on. This meant that UIKit developers could modify the way their text field behaved without having to create a custom text field type of their own.
So add the following code inside the CameraScannerViewController:
func makeCoordinator() -> Coordinator {
Coordinator(self)
}class Coordinator: NSObject, DataScannerViewControllerDelegate {
var parent: CameraScannerViewController
init(_ parent: CameraScannerViewController) {
self.parent = parent
}
}
Now we can use the similar code inside makeUIViewController just on the top of the return code:
func makeUIViewController(context: Context) -> DataScannerViewController {
...
viewController.delegate = context.coordinator
return viewController
}Begin data scanning
It’s time to start the data scanning. Once the user allows access to the camera without restrictions, you can begin scanning for items that appear in the live video by invoking the startScanning() method. In this case, when the scanning view is presented we will need the view to perform the scanning action.
We need a value to tell the scanning view to scan, add the following code to the CameraScannerViewController :
@Binding var startScanning: BoolWhen the startScanning’s value is set to true, we need to update UIViewController and start scanning, add the following code inside updateUIViewController :
func updateUIViewController(_ viewController: DataScannerViewController, context: Context) {
if startScanning {
try? viewController.startScanning()
} else {
viewController.stopScanning()
}
}Respond when we tap an item
When we tap a recognized item in the live video, the view controller invokes the dataScanner(_:didTapOn:) delegate method and passes the recognized item. Implement this method to take some action depending on the item the user taps. Use the parameters of the RecognizedItem enum to get details about the item, such as the bounds.
In this case, to handle when we tap a text that the camera recognized, implement the dataScanner(_:didTapOn:) method to perform an action that shows the result in the screen. So add the following code inside Coordinator class:
class Coordinator: NSObject, DataScannerViewControllerDelegate {
...
func dataScanner(_ dataScanner: DataScannerViewController, didTapOn item: RecognizedItem) {
switch item {
case .text(let text):
parent.scanResult = text.transcript
default:
break
}
}
}And add a Binding property inside CameraScannerViewController :
@Binding var scanResult: StringScan Now
It’s time to update the CameraScanner.swift file. Simply add the following code:
@Binding var startScanning: Bool
@Binding var scanResult: StringAnd change the Text(“Scanning View”) to the following code:
var body: some View {
NavigationView {
CameraScannerViewController(startScanning: $startScanning, scanResult: $scanResult)
...
}
}Finally, add the following code to ContentView :
struct ContentView: View {
...
@State private var scanResults: String = "" var body: some View {
VStack {
...
}
.sheet(isPresented: $showCameraScannerView) {
CameraScanner(startScanning: $showCameraScannerView, scanResult: $scanResults)
}
...
}
}
scanResults is used to pass the value through views. Once the camera scan something, scanResults will be updated and then update the Text view.
Now run this project and enjoy yourself.
Next Step
WWDC not only brings Live Text in Camera, but also simplifies how developers can enable Live Text Interactions With Images. If you are interested in how to enable Live Text Interactions With Images, please check my new article: WWDC22: Enabling Live Text Interactions With Images in SwiftUI.
Source Code
You can find the source code on Github.
Supports Me
If you think this article is helpful, you can support me by downloading my first iOS App, WeTally, on the iOS App Store. WeTally is a paramount, exquisite and practical app, allowing you to grasp all financial information in an instant, with a soothing and pleasant use experience, and easily accomplish every accounting.. It is free and useful for many people. You can ask me for free one-month access to advanced features. Hope you like it.
