AR Development with ARKit 8
Augmented reality is no longer a futuristic concept—it’s a mainstream tool that can turn a flat screen into an interactive 3‑D playground. With Apple’s ARKit 8, developers get a richer set of sensors, improved world‑tracking, and new APIs that make it easier to blend virtual objects with the real world. In this guide we’ll walk through the core workflow, explore two practical projects, and sprinkle in pro tips that will help you ship polished AR experiences faster.
Getting Started: The ARKit 8 Toolbox
Before you dive into code, make sure your development environment meets the baseline requirements. ARKit 8 runs on iOS 16 or later and needs a device with an A12 Bionic chip or newer (iPhone SE 2 or later, iPad Pro 3rd gen, etc.). Xcode 15 provides the latest templates, and you’ll want to enable the RealityKit and ARKit frameworks in your project settings.
At its heart, ARKit 8 introduces three game‑changing features: Depth API enhancements for better occlusion, Location Anchors that let you pin content to real‑world coordinates, and Improved Body Tracking that works with the iPhone’s TrueDepth camera. Understanding these pillars will shape how you design your AR scenes.
Core Classes You’ll Use
- ARView – The primary view that renders both RealityKit entities and SceneKit nodes.
- ARWorldTrackingConfiguration – Configures plane detection, scene reconstruction, and more.
- ARAnchor and its subclasses (e.g.,
ARPlaneAnchor,ARLocationAnchor). - ARCoachingOverlayView – Guides users through the initial world‑mapping step.
Most AR apps start by creating an ARView, attaching a configuration, and then responding to delegate callbacks when ARKit discovers a surface or a location anchor. The following sections break down this flow with concrete examples.
Example 1: Placing a Simple 3‑D Model on Detected Planes
Our first project is a classic “place‑a‑cube” demo. It demonstrates plane detection, hit‑testing, and how to attach a ModelEntity to a real‑world surface. The code is deliberately minimal so you can focus on the ARKit pipeline.
import UIKit
import RealityKit
import ARKit
class SimpleARViewController: UIViewController, ARSessionDelegate {
var arView: ARView!
override func viewDidLoad() {
super.viewDidLoad()
arView = ARView(frame: view.bounds)
view.addSubview(arView)
// 1️⃣ Configure world tracking with horizontal plane detection
let config = ARWorldTrackingConfiguration()
config.planeDetection = [.horizontal]
arView.session.run(config)
// 2️⃣ Add a tap gesture recognizer for placing objects
let tap = UITapGestureRecognizer(target: self, action: #selector(handleTap(_:)))
arView.addGestureRecognizer(tap)
}
@objc func handleTap(_ gesture: UITapGestureRecognizer) {
let location = gesture.location(in: arView)
// 3️⃣ Perform a hit‑test against detected planes
if let result = arView.raycast(from: location, allowing: .estimatedPlane, alignment: .horizontal).first {
// 4️⃣ Create a simple cube entity
let mesh = MeshResource.generateBox(size: 0.1)
let material = SimpleMaterial(color: .systemTeal, isMetallic: true)
let cube = ModelEntity(mesh: mesh, materials: [material])
// 5️⃣ Anchor the cube at the hit‑test position
let anchor = AnchorEntity(world: result.worldTransform.translation)
anchor.addChild(cube)
arView.scene.addAnchor(anchor)
}
}
}
Notice how the code follows a clear five‑step pattern: configure, detect, interpret user input, create a virtual object, and finally anchor it. The raycast call replaces the older hitTest API, offering more accurate depth estimation thanks to ARKit 8’s enhanced scene reconstruction.
Pro tip: Enable config.sceneReconstruction = .meshWithClassification if you need a full 3‑D mesh of the environment. This gives you per‑pixel classification (floor, wall, ceiling) and can dramatically improve placement realism.
Real‑World Use Case: Interior Design Apps
Imagine a furniture retailer that wants customers to visualize a sofa in their living room. By extending the previous example, you can load a high‑resolution glTF model, apply realistic materials, and even let users scale the object with pinch gestures. The same pipeline—plane detection → hit‑test → anchor → entity—remains unchanged, but you swap the simple cube for a commercial‑grade asset.
To keep the UI responsive, load large models asynchronously using ModelEntity.loadModelAsync. This prevents the main thread from stalling while the asset decompresses, a common pitfall in production AR apps.
Async Model Loading Example
func placeFurniture(at transform: simd_float4x4) {
// Load a glTF sofa model in the background
ModelEntity.loadModelAsync(named: "sofa.glb")
.sink { loadCompletion in
switch loadCompletion {
case .failure(let error):
print("Failed to load model: \(error)")
case .finished:
break
}
} receiveValue: { modelEntity in
// Apply a subtle shadow to ground the object
modelEntity.generateCollisionShapes(recursive: true)
let anchor = AnchorEntity(world: transform.translation)
anchor.addChild(modelEntity)
self.arView.scene.addAnchor(anchor)
}
.store(in: &cancellables)
}
By using Combine’s sink operator, you keep the UI thread free while the model loads, and you get a clean error‑handling path. The generateCollisionShapes call also enables physics‑based interactions later on, such as dragging the sofa across the floor.
Example 2: Location‑Based AR with ARKit 8
ARKit 8’s Location Anchors let you pin content to latitude/longitude coordinates, opening the door to outdoor experiences, tourism guides, and location‑aware games. The API works best with the new ARGeoTrackingConfiguration, which fuses GPS, Wi‑Fi, and visual‑inertial odometry for sub‑meter accuracy in many urban environments.
Below is a minimal implementation that places a virtual billboard at the Eiffel Tower’s coordinates. When the user points their device toward the landmark, the billboard appears anchored in the correct spot, regardless of where the user started.
import UIKit
import RealityKit
import ARKit
import CoreLocation
class GeoARViewController: UIViewController, ARSessionDelegate {
var arView: ARView!
let targetLocation = CLLocationCoordinate2D(latitude: 48.8584, longitude: 2.2945) // Eiffel Tower
override func viewDidLoad() {
super.viewDidLoad()
arView = ARView(frame: view.bounds)
view.addSubview(arView)
// 1️⃣ Geo‑tracking configuration
let geoConfig = ARGeoTrackingConfiguration()
geoConfig.planeDetection = [.horizontal, .vertical]
geoConfig.worldAlignment = .gravityAndHeading
arView.session.run(geoConfig, options: [])
// 2️⃣ Add coaching overlay for smoother onboarding
let coaching = ARCoachingOverlayView()
coaching.session = arView.session
coaching.autoresizingMask = [.flexibleWidth, .flexibleHeight]
coaching.goal = .horizontalPlane
coaching.activatesAutomatically = true
view.addSubview(coaching)
// 3️⃣ Listen for location anchor updates
arView.session.delegate = self
}
// ARSessionDelegate callback
func session(_ session: ARSession, didAdd anchors: [ARAnchor]) {
for anchor in anchors {
guard let locationAnchor = anchor as? ARLocationAnchor else { continue }
// Verify we’re close enough to the target location
let distance = locationAnchor.coordinate.distance(to: targetLocation)
if distance < 20 { // within 20 meters
placeBillboard(at: locationAnchor.transform)
}
}
}
func placeBillboard(at transform: simd_float4x4) {
// Simple textured plane as a billboard
let mesh = MeshResource.generatePlane(width: 2.0, height: 1.0)
let material = UnlitMaterial(color: .white)
material.baseColor.texture = try! TextureResource.load(contentsOf: URL(string: "https://example.com/billboard.png")!)
let billboard = ModelEntity(mesh: mesh, materials: [material])
let anchor = AnchorEntity(world: transform.translation)
anchor.addChild(billboard)
arView.scene.addAnchor(anchor)
}
}
The ARGeoTrackingConfiguration automatically creates ARLocationAnchor objects as the device moves. In the delegate callback we filter anchors by distance to our point of interest, then attach a simple billboard. This pattern scales nicely: you can maintain an array of target locations and spawn different assets (e.g., audio guides, 3‑D sculptures) as users approach each spot.
Pro tip: Always requestNSLocationWhenInUseUsageDescriptionin yourInfo.plistand handle the user’s privacy choice gracefully. If location services are disabled, fall back to a standard plane‑detection flow rather than presenting a blank screen.
Advanced Interaction: Gestures, Physics, and Occlusion
ARKit 8’s depth API now supports person segmentation and environment occlusion out of the box. By enabling config.environmentTexturing = .automatic and config.sceneReconstruction = .meshWithClassification, virtual objects can be correctly hidden behind real‑world obstacles, dramatically increasing immersion.
To make the experience feel tangible, combine gestures with physics. Below is a concise snippet that adds pinch‑to‑scale and pan‑to‑move for any selected ModelEntity. The physics engine ensures objects don’t intersect the floor or each other.
extension ARView {
func enableObjectManipulation() {
// Pinch gesture for scaling
let pinch = UIPinchGestureRecognizer(target: self, action: #selector(handlePinch(_:)))
addGestureRecognizer(pinch)
// Pan gesture for translation
let pan = UIPanGestureRecognizer(target: self, action: #selector(handlePan(_:)))
addGestureRecognizer(pan)
}
@objc func handlePinch(_ gesture: UIPinchGestureRecognizer) {
guard let entity = entity(at: gesture.location(in: self)) else { return }
let scale = Float(gesture.scale)
entity.scale = SIMD3(repeating: scale)
if gesture.state == .ended { gesture.scale = 1.0 }
}
@objc func handlePan(_ gesture: UIPanGestureRecognizer) {
let location = gesture.location(in: self)
guard let result = raycast(from: location, allowing: .existingPlaneGeometry, alignment: .any).first,
let entity = entity(at: location) else { return }
// Move entity to new world position
entity.transform.translation = result.worldTransform.translation
}
}
Notice the use of entity(at:), a convenience method that returns the topmost ModelEntity under a touch point. By coupling gestures with ray‑casts against existing plane geometry, you keep objects glued to real surfaces while still allowing free movement.
Occlusion with People Segmentation
If your app features avatars or virtual assistants, you’ll want them to appear correctly behind people walking in front of the camera. Enabling ARBodyTrackingConfiguration and setting the renderOptions on ARView achieves this in a few lines:
let bodyConfig = ARBodyTrackingConfiguration()
arView.session.run(bodyConfig)
arView.renderOptions.insert(.disablePersonOcclusion) // false = enable occlusion
When .disablePersonOcclusion is not set, ARKit automatically masks out the region occupied by a detected person, letting your virtual content appear naturally behind them. This is especially useful for AR games where characters interact with the real world in a believable way.
Performance Considerations
AR experiences are CPU‑ and GPU‑intensive, so optimizing for frame rate is non‑negotiable. Here are three quick checks before you ship:
- Limit the number of active anchors. Each anchor adds a node to the scene graph; prune anchors that are far away or no longer needed.
- Use lightweight materials. Unlit or simple physically‑based materials render faster than complex shaders.
- Batch geometry. Combine static meshes into a single
ModelEntitywhen possible to reduce draw calls.
ARKit also provides a ARFrame.debugOptions flag that shows CPU/GPU usage in real time. Turn it on during development to spot bottlenecks early.
Pro tip: When targeting older devices (e.g., iPhone SE 2), disable sceneReconstruction and stick to basic plane detection. The extra mesh processing can drop the frame rate below 30 fps, which feels sluggish in AR.
Testing and Debugging Strategies
Because AR relies on the physical environment, testing on a single device can be misleading. Use the Xcode “Simulated AR” mode to verify basic logic without moving a phone around. For more realistic testing, create a small “test suite” of common environments: a plain table, a cluttered desk, and an outdoor patio. Capture screenshots of the ARFrame debug overlay to compare tracking quality across scenes.
When you encounter “tracking lost” warnings, check the following:
- Is the lighting adequate? ARKit prefers 500–1000 lux.
- Are there enough visual features? Plain white walls can confuse the visual‑inertial system.
- Is the device’s motion too rapid? Sudden accelerations can cause temporary pose estimation errors.
Adding a ARCoachingOverlayView (as shown earlier) helps guide users to optimal scanning motions, reducing the frequency of tracking loss.
Deploying to the App Store
Before you submit, make sure your app complies with Apple’s AR guidelines. Include a clear description of the AR functionality in the App Store metadata, and provide screenshots that showcase the experience in action. If you use location anchors, you must also include a privacy policy that explains how location data is collected and stored.
Finally, test the binary on at least two physical devices with different hardware capabilities. Use TestFlight to gather feedback on usability, especially around onboarding flows like the coaching overlay.
Conclusion
ARKit 8 empowers developers to craft immersive, context‑aware experiences with relatively little boilerplate. By mastering plane detection, location anchors, and the new depth APIs, you can build everything from simple product visualizers to sophisticated outdoor tours. Remember to keep the user’s environment in mind—good lighting, clear surfaces, and smooth onboarding are as important as the code you write. With the patterns, code snippets, and pro tips shared here, you’re ready to turn those ideas into real‑world AR applications that delight users and stand out in the App Store.