To implement image tracking, you will set up an ARSession that uses ARWorldTrackingConfiguration to detect images and track a user's movement through the environment. When one of the images you have prepared is discovered in the scene, an SCNPlane will be added above the picture with a short description of the picture itself.
Because ARKit uses the camera, your app must explicitly provide a reason for accessing the camera, so the user understands why your app needs permission to use their camera. Add the NSCameraUsageDescription key to the Info.plist file and add a short text about why the gallery needs access to the camera.
If you open ViewController.swift, you will find a property called artDescriptions. Make sure to update this dictionary with the names of the images you added to the resource group, and add a short description for each image.
Next, update viewDidLoad() so ViewController is set as the delegate for both ARSCNView and ARSession. Add the following lines of code to do this:
arKitScene.delegate = self arKitScene.session.delegate = self
The scene delegate and session delegate are very similar. The session delegate provides very fine-grained control of the content that is displayed in the scene, and you'll usually use this protocol extensively if you build your own rendering. Since the AR gallery is rendered using SceneKit, the only reason to adopt ARSessionDelegate is to respond to changes in the session's tracking state.
All of the interesting methods that you should adopt are part of ARSCNViewDelegate. This delegate is used to respond to specific events. For instance, when new features are discovered in the scene or when new content was added.
Currently, your AR gallery doesn't do much. You must configure the ARSession that is part of the scene to begin using ARKit. The best moment to set this all up is right before the view controller becomes visible. Therefore, you should do all of the remaining setup in viewWillAppear(_:). Add the following implementation for this method to ViewController:
override func viewWillAppear(_ animated: Bool) {
super.viewWillAppear(animated)
// 1
let imageSet = ARReferenceImage.referenceImages(inGroupNamed: "Art", bundle: Bundle.main)!
// 2
let configuration = ARWorldTrackingConfiguration()
configuration.planeDetection = [.vertical, .horizontal]
configuration.detectionImages = imageSet
// 3
arKitScene.session.run(configuration, options: [])
}
The first step in this method is to read the reference image from the app bundle. These are the images you added to Assets.xcassets. Next, ARWorldTrackingConfiguration is created, and it's configured to track both horizontal and vertical planes, as well as the reference images. Lastly, the configuration is passed to the session's run(_:options:) method. If you run your app now, you should already be prompted for camera usage, and you should see the error-handling working. Try covering the camera with your hand, that should make an error message appear.
Keeping an AR session alive if a view isn't visible anymore is quite wasteful, so it's a good idea to pause the session if the app is closed or if the view controller that contains the AR scene becomes invisible. Add the following method to ViewController to achieve this:
override func viewWillDisappear(_ animated: Bool) {
super.viewWillDisappear(animated)
arKitScene.session.pause()
}
In the current setup, the AR session detects your images, but it does nothing to visualize this. When one of the images you added is identified, ARSCNViewDelegate is notified of this. To be specific, the renderer(_:didAdd:for:) method is called on the scene delegate when a new SCNNode is added to the view. For instance, when the AR session discovers a flat surface, it adds a node for ARPlaneAnchor, or when it detects one if the image you're tracking, a node for ARImageAnchor is added. Since this method can be called with different reasons, it's essential that you add logic to differentiate between the various reasons that could cause a new SCNNode to be added to the scene.
Because the AR gallery will implement several other features that could trigger the addition of a new node, you should separate the different actions you want to take for each different type of anchor into specialized methods. Add the following method to ARSCNViewDelegate to add the information plane next to a detected image:
func placeImageInfo(withNode node: SCNNode, for anchor: ARImageAnchor) {
let referenceImage = anchor.referenceImage
// 1
let infoPlane = SCNPlane(width: 15, height: 10)
infoPlane.firstMaterial?.diffuse.contents = UIColor.white
infoPlane.firstMaterial?.transparency = 0.5
infoPlane.cornerRadius = 0.5
// 2
let infoNode = SCNNode(geometry: infoPlane)
infoNode.localTranslate(by: SCNVector3(0, 10, -referenceImage.physicalSize.height / 2 + 0.5))
infoNode.eulerAngles.x = -.pi / 4
// 3
let textGeometry = SCNText(string: artDescriptions[referenceImage.name ?? "flowers"], extrusionDepth: 0.2)
textGeometry.firstMaterial?.diffuse.contents = UIColor.red
textGeometry.font = UIFont.systemFont(ofSize: 1.3)
textGeometry.isWrapped = true
textGeometry.containerFrame = CGRect(x: -6.5, y: -4, width: 13, height: 8)
let textNode = SCNNode(geometry: textGeometry)
// 4
node.addChildNode(infoNode)
infoNode.addChildNode(textNode)
}
The preceding code should look somewhat familiar to you. First, an instance of SCNPlane is created. Then, this plane is added to SCNNode. This node is translated slightly to position it above the detected image. This translation uses SCNVector3 so it can be translated into three dimensions. The node is also rotated a little bit to create a nice-looking effect.
Next, add the following implementation for renderer(_:didAdd:for:):
func renderer(_ renderer: SCNSceneRenderer, didAdd node: SCNNode, for anchor: ARAnchor) {
if let imageAnchor = anchor as? ARImageAnchor {
placeImageInfo(withNode: node, for: imageAnchor)
}
}
This method checks whether the anchor that was discovered is an image anchor; if it is, placeImageInfo(withNode:for:) is called to display the information sign.
Go ahead and run your app now! When you find one of the images that you added to your resource group, an information box should appear on top of it as shown in the following screenshot:
Pretty awesome, right? Let's take it one step further and allow users to position some of the pictures from the collection view wherever they want in the scene.