How to Build an ARKit Scene in SwiftUI

iOS 17+ Xcode 16+ Advanced APIs: ARKit / RealityKit Updated: May 11, 2026

TL;DR

SwiftUI has no native AR view, so you wrap RealityKit's ARView in a UIViewRepresentable struct. Configure an ARWorldTrackingConfiguration inside makeUIView, run the session, and place ModelEntity anchors via raycasting on tap.

import SwiftUI
import RealityKit
import ARKit

struct ARSceneView: UIViewRepresentable {
    func makeUIView(context: Context) -> ARView {
        let arView = ARView(frame: .zero)
        let config = ARWorldTrackingConfiguration()
        config.planeDetection = [.horizontal, .vertical]
        arView.session.run(config)
        return arView
    }
    func updateUIView(_ uiView: ARView, context: Context) {}
}

struct ContentView: View {
    var body: some View {
        ARSceneView().ignoresSafeArea()
    }
}

Full implementation

The complete example adds a Coordinator as the ARView's delegate so you can react to session events, wires up a tap gesture recognizer to perform a raycast against detected planes, and places a semi-transparent RealityKit box entity wherever the user taps. A SwiftUI overlay shows a status message that updates via a published property on the coordinator.

import SwiftUI
import RealityKit
import ARKit

// MARK: - State shared between the coordinator and SwiftUI
@Observable
final class ARSceneState {
    var statusMessage: String = "Move your device to detect surfaces"
    var placedCount: Int = 0
}

// MARK: - UIViewRepresentable wrapper
struct ARSceneView: UIViewRepresentable {

    var state: ARSceneState

    func makeCoordinator() -> Coordinator {
        Coordinator(state: state)
    }

    func makeUIView(context: Context) -> ARView {
        let arView = ARView(frame: .zero)

        // World tracking with plane detection + scene depth on LiDAR devices
        let config = ARWorldTrackingConfiguration()
        config.planeDetection = [.horizontal, .vertical]
        if ARWorldTrackingConfiguration.supportsSceneReconstruction(.mesh) {
            config.sceneReconstruction = .mesh
        }
        arView.session.run(config)
        arView.session.delegate = context.coordinator

        // Store reference so the coordinator can raycast
        context.coordinator.arView = arView

        // Tap gesture to place objects
        let tap = UITapGestureRecognizer(
            target: context.coordinator,
            action: #selector(Coordinator.handleTap(_:))
        )
        arView.addGestureRecognizer(tap)

        // Debug plane overlay (helpful during dev)
        arView.debugOptions = [.showAnchorOrigins]

        return arView
    }

    func updateUIView(_ uiView: ARView, context: Context) {}
}

// MARK: - Coordinator (ARSessionDelegate + gesture target)
extension ARSceneView {
    final class Coordinator: NSObject, ARSessionDelegate {
        let state: ARSceneState
        weak var arView: ARView?

        init(state: ARSceneState) { self.state = state }

        // Called when new anchors are added (e.g., detected planes)
        func session(_ session: ARSession, didAdd anchors: [ARAnchor]) {
            let planeCount = anchors.compactMap { $0 as? ARPlaneAnchor }.count
            if planeCount > 0 {
                Task { @MainActor in
                    self.state.statusMessage = "Tap a surface to place an object"
                }
            }
        }

        // Tap: raycast against detected geometry, place a box
        @objc func handleTap(_ sender: UITapGestureRecognizer) {
            guard let arView else { return }
            let location = sender.location(in: arView)

            // Prefer mesh geometry; fall back to estimated plane
            let results = arView.raycast(
                from: location,
                allowing: .estimatedPlane,
                alignment: .any
            )
            guard let first = results.first else { return }

            // Build a small coloured box
            let mesh = MeshResource.generateBox(size: 0.05, cornerRadius: 0.004)
            let material = SimpleMaterial(
                color: UIColor.systemIndigo.withAlphaComponent(0.85),
                isMetallic: true
            )
            let entity = ModelEntity(mesh: mesh, materials: [material])
            entity.generateCollisionShapes(recursive: false)

            let anchor = AnchorEntity(world: first.worldTransform)
            anchor.addChild(entity)
            arView.scene.addAnchor(anchor)

            Task { @MainActor in
                self.state.placedCount += 1
                self.state.statusMessage = "\(self.state.placedCount) object(s) placed"
            }
        }
    }
}

// MARK: - SwiftUI screen
struct ARSceneScreen: View {
    @State private var sceneState = ARSceneState()

    var body: some View {
        ZStack(alignment: .top) {
            ARSceneView(state: sceneState)
                .ignoresSafeArea()

            // Heads-up status banner
            Text(sceneState.statusMessage)
                .font(.footnote.weight(.semibold))
                .foregroundStyle(.white)
                .padding(.horizontal, 16)
                .padding(.vertical, 8)
                .background(.ultraThinMaterial, in: Capsule())
                .padding(.top, 60)
                .animation(.easeInOut, value: sceneState.statusMessage)
        }
    }
}

#Preview {
    ARSceneScreen()
}

How it works

UIViewRepresentable bridge — ARView is a UIKit view, so makeUIView constructs and configures it; updateUIView stays empty because the AR session manages its own state frame-by-frame.
ARWorldTrackingConfiguration — Running the session with planeDetection: [.horizontal, .vertical] lets ARKit identify flat surfaces. On LiDAR-equipped devices the optional sceneReconstruction = .mesh gives pixel-precise geometry for more accurate raycasting.
Coordinator as ARSessionDelegate — The Coordinator class (created once by makeCoordinator) receives session(_:didAdd:) callbacks. It pushes status updates back to SwiftUI via @Observable ARSceneState on the main actor, keeping the view layer reactive without manual DispatchQueue calls.
Raycasting on tap — arView.raycast(from:allowing:alignment:) projects a screen-space point into world space, returning hit results sorted by distance. Using .estimatedPlane works even before a full plane anchor is confirmed.
RealityKit entity placement — A ModelEntity is added as a child of an AnchorEntity(world:) at the raycast's worldTransform, pinning it to real-world coordinates rather than to the camera.

Variants

Load a USDZ model instead of a primitive

// In Coordinator.handleTap, replace the mesh/material block:
do {
    // "toy_car.usdz" must be in the app bundle
    let entity = try await ModelEntity(named: "toy_car")
    entity.scale = SIMD3(repeating: 0.3)
    entity.generateCollisionShapes(recursive: true)

    let anchor = AnchorEntity(world: first.worldTransform)
    anchor.addChild(entity)
    await MainActor.run { arView.scene.addAnchor(anchor) }
} catch {
    print("Failed to load model: \(error)")
}

Image-based anchor (AR Quick Look / marker)

Swap ARWorldTrackingConfiguration for ARImageTrackingConfiguration and load reference images from AR Resources in your asset catalogue. Set config.trackingImages to that group, then in session(_:didAdd:) cast the anchor to ARImageAnchor and place content relative to its transform. Useful for packaging inserts, book covers, or product labels that should trigger 3D overlays.

Common pitfalls

Missing NSCameraUsageDescription crashes the app on launch. Add the key to Info.plist or your target's Privacy settings in Xcode 16 before running on a real device — ARKit won't show a helpful error, it just terminates.
Never call session.run from updateUIView. SwiftUI can call updateUIView many times per second; restarting the session there resets tracking constantly. Initialise once in makeUIView and update configuration only when a tracked property actually changes, guarded by an equality check.
Raycasting silently returns empty results on the simulator. The iOS Simulator has no camera, so ARView.raycast always returns []. Always test AR features on a physical device; add a guard with a user-facing message if ARWorldTrackingConfiguration.isSupported returns false.
VoiceOver accessibility. ARView has no built-in accessibility tree. Add a transparent AccessibilityElement SwiftUI overlay with an .accessibilityLabel describing the scene state, and announce placement events with UIAccessibility.post(notification: .announcement, argument:) from the coordinator.

Prompt this with Claude Code

When using Soarias or Claude Code directly to implement this:

Implement an ARKit scene in SwiftUI for iOS 17+.
Use ARKit and RealityKit (ARView, ARWorldTrackingConfiguration,
ModelEntity, AnchorEntity, raycasting).
Wrap ARView in UIViewRepresentable with a Coordinator as ARSessionDelegate.
On tap, raycast against detected planes and place a 3D box entity.
Use @Observable for session status updates to the SwiftUI layer.
Make it accessible (VoiceOver announcements on placement).
Add a #Preview with realistic sample data.

In the Soarias Build phase, paste this prompt into the implementation step to scaffold the full AR feature — including the Info.plist camera permission key — in a single pass before iterating on entity design.

FAQ

Does this work on iOS 16?

The UIViewRepresentable + ARView approach works back to iOS 13. However, this guide targets iOS 17+ because it relies on the @Observable macro (introduced in iOS 17) for state sharing between the coordinator and SwiftUI. If you need iOS 16 support, replace @Observable ARSceneState with an ObservableObject / @Published class and pass it as an @EnvironmentObject or binding instead.

How do I remove placed objects?

Keep an array of AnchorEntity references in ARSceneState. To delete the last one call arView.scene.removeAnchor(anchor) from the coordinator. For bulk removal, iterate the array and call removeAnchor on each, then clear the array. RealityKit automatically deallocates the underlying Metal resources when the last reference drops.

What's the UIKit equivalent?

In a pure UIKit app you subclass UIViewController, add an ARView as a full-screen subview in viewDidLoad, configure the session in viewWillAppear, and pause it in viewWillDisappear. The SwiftUI UIViewRepresentable wrapper mirrors this lifecycle: makeUIView ≈ viewDidLoad, and you should pause the session by overriding dismantleUIView(_:coordinator:) — a static method on the representable — to avoid leaking the camera resource when the view is removed from the hierarchy.

Last reviewed: 2026-05-11 by the Soarias team.