Part III: Adding 3D Objects 📦
On iOS, there are two ways to work with 3D:
SceneKit, simply put, is a high-level 3D framework for creating 3D objects and working with 3D scenes. It includes a physics engine, and a particle generator to make it as easy as it can be to work with 3D objects and scenes.
Metal is a low-level API to the GPU-accelerated hardware on Apple devices. It is designed to be extremely efficient on Apple hardware. Although not the easiest to deal with if you’re looking to render simple 3D without GPU-acceleration.
Let’s deal with SceneKit to add 3D objects to our scene.
You should be familiar with setting up an AR Session by now. The fundamental concept behind adding 3D objects to an AR Session is adding Anchors to the scene and have the device track the anchors inside of the AR Session.
The AR Session object in your View Controller can be used to add anchors to your session. Like so:
An AR Anchor is the real-world position and orientation that can be used for placing objects in an AR scene. When plane detection is enabled, ARKit adds ARAnchor (more specifically ARPlaneAnchor) objects to the session.
SceneKit, meet ARKit
There are two ways to add 3D content to your session:
- SCNView’s child node
To understand how each of them works let’s make sure we’re through with the fundamental concept behind this first.
Any 3D content modeled with SceneKit can be used with ARKit. ARKit makes this easy with ARAnchors. The idea behind this is — AR Anchor objects that are added to the scene can hold or show 3D content in their position in real-world, which is tracked by your device. That’s it. That’s all there is to it.
Let’s use SceneKit to create a simple cube and see how each of the methods to add 3D content works.
Creating a Cube
If you’re new to SceneKit, here’s what you need to know to create 3D content:
- All 3D content are depicted by nodes:
- To create a node, you need to specify a 3D geometry after which the node is modeled
To create a cube, we use SceneKit’s
SCNBox geometry and model our node with it.
It’s as simple as that.
Adding 3D geometry to the AR Session
Now that we have our
cube we’ll look into how to add it to our scene.
Like I mentioned earlier, there are two ways to add 3D content to an AR session. But before we jump into how we do that, we need a position for the 3D content that you want to place. Let’s get to that.
Since we’re dealing with 3D space, there are two choices that we have to place 3D content:
- On Plane
- Off Plane
With SceneKit, a position of an object in the world is denoted with a
SCNVector3, this is nothing but a vector object with 3 axes:
To place an object on a plane that has been detected, the user generally taps on the screen within the extent of the plane that has been detected. The tap point we have to work with is a
How then do we get a
SCNVector3 with three elements, you ask?
CGPoint and an
ARPlaneAnchor to work with, we can hit-test the point against the plane anchor to get its position in 3D space. This is the general approach towards how objects are placed on a plane. If that sounds complicated, don’t worry, just follow along:
With ARKit, a 2D point can be hit-tested against a plane with ARSCNView’s
The Result type that will be passed is an enum on
ARHitTestResult, with the following values:
estimatedHorizontalPlane is a real-world planar surface detected by the search (without a corresponding anchor), whose orientation is perpendicular to gravity.
featurePoint is a point automatically identified by ARKit as part of a continuous surface, but without a corresponding anchor.
What we are interested in to add objects to a plane, are the plane anchors already added to the session by plane detection. To use this we use:
existingPlaneUsingExtent type when hit-testing.
UsingExtent makes the hit-test respect the plane’s limited size unlike simply
Now that we have the hit-test method, we get back a
ARHitTestResult object that contains the position information. Not exactly. But we can use the information in it to build a
SCNVector3, which denotes a position in 3D space like:
worldTransform property of the
ARHitTestResult contains information about the object in its 3rd column that we use to transpose into a position vector.
So, no we have a position on the plane we can work with.
To add an object off plane, the usual approach is to add it in front of the device’s camera where the user taps. This can easily be accomplished by building a
SCNVector3 using the camera transform.
To obtain the camera transform:
Once you have the
cameraTransform (MDLTransform), all there’s left to build the
We now have a position in 3D space for your object off-plane.
Let’s go over how to add it to the scene.
Now that we have the position of the 3D object as a vector, we can assign that to our SCNNode:
cube to the scene is all there’s left to adding your 3D content:
This should place a 3D cube in your world and your device should now track it.
The other method to add 3D content is adding raw anchors, and showing SCNNode in their position.
The idea behind this method is simple:
- Create an AR Anchor
- Add it to the AR Session
ARSCNView’s delegate method to return a
SCNNodefor an anchor
To get hold of an anchor to add:
- On Plane
ARHitTestResultobject from the hit-test method has an
anchorproperty which can be added to the session using the
- Off Plane
transformproperty, an anchor can be created like:
and can be added using the
session.add(anchor: ARAnchor) method.
Once anchors are added to the session, to display 3D content in their place, we need to implement
This method returns an SCNNode for an anchor in the session, an anchor’s identifier property can be used to distinguish between different anchors added to the session. We simply return a SCNNode in this method, for the required anchor to add 3D content.
Let’s look at how all of this comes together in code:
- Adding 3D content on plane:
Incase you’re adding anchors to the session (Method 1), implementing the delegate method like this is necessary to display 3D content:
- Adding 3D content off-plane
So, that’s about how to add 3D content on and off-plane with ARKit.
In this part we have seen how to add 3D content to your AR session, we’ll look into adding some lights and making your objects more interesting in the next part. Thanks for following along, and feel free to leave any feedback!
Index of the series of ARKit posts: