Table of Contents
Have you heard about those cool smartphone apps that blend digital objects into the real world? That‘s augmented reality (AR), and it‘s skyrocketing in popularity these days.
Unity makes it surprisingly easy to start building your own AR apps. With a few setup steps, their AR Foundation toolkit allows you to detect planes, visualize data overlays, and place interactive 3D models.
In this guide, I‘ll walk you step-by-step through configuring Unity and deploying to mobile. Stick with me, and you‘ll be well on your way to becoming an AR developer!
What Exactly is Augmented Reality Anyway?
Augmented reality refers to computer-generated sensory input like graphics, GPS data, or audio layered over real world environments. AR apps run on devices with cameras like phones or tablets and overlay contextual digital information as you move the device around.
For example, furniture retailer IKEA created an AR app that lets you preview true-to-scale models of their furniture in your own home before you buy. You simply browse items in the app, tap one you like, point your phone at a room, and see a 3D rendering placed in your actual space.
AR is expanding rapidly into fields like social media, medicine, industrial design, and more. Global AR app revenue hit $2.8 billion in 2021 and could grow to a $72 billion market by 2030 according to some projections.
Now is a great time to start developing your AR skills!
Introducing Unity AR Foundation
Unity provides several tools for building augmented and virtual reality apps including their purpose-built AR Foundation framework.
AR Foundation handles all the low-level integration with AR devices and platforms through modular plugins. That means you can focus on creating compelling experiences rather than wrestling with compatibility issues or hardware quirks.
Some key capabilities provided by Unity‘s AR Foundation include:
- Environmental Mapping: Detect feature points and planar surfaces in physical spaces
- Occlusion: Digital objects realistically interact with real-world geometry
- Light Estimation: Virtually placed objects have lighting applied as if truly present
- Anchors: Attach objects to tracked reference points
- Hit Testing: Cast rays to precisely place objects on detected planes
Let‘s explore a hands-on project to see these in action!
Developing Our AR App in Unity
Alright my friend, let‘s build your first AR application together with Unity.
We‘ll walk through:
- Setting up Unity and packages
- Configuring the AR environment
- Detecting trackable planes and points
- Placing a 3D model with raycasting
- Building to your Android device
Follow along on your own Unity install, and you‘ll gain practical experience for constructing production-grade apps.
Getting Set Up with Unity
First, head over to Unity‘s website and grab the Unity Hub installer. This acts as a central interface for managing Unity versions and builds.
Run the installer, open up Hub, and add the 2020 Long-Term Support editor release. Make sure to check Android Build Support too. This will ensure we can actually deploy to our phones later!
Next, click the New Project button and create a new 3D application with default settings. Once the editor opens up, we can start configuring our AR project.
Importing AR Foundation and Setting Up the Scene
Within Unity‘s Project window, choose Window > Package Manager to open the package repository.
Here we can import the essential AR Foundation package along with the ARCore XR Plugin for Android mobile support. Install both of these critical AR packages now.
With the packages added, Unity gains all kinds of new AR-centric components and features. We need to construct an initial scene first though.
Open up the Hierarchy tab and add an AR Session game object. The session orchestrates the lifecycle and data flow of our AR experience.
Then add an AR Session Origin child object to transform between the device‘s AR coordinate system and Unity‘s world coordinate grid.
Configure the ARCamera subcomponent too – disable MSAA antialiasing for performance, and set a near clipping plane around 0.1 meters for good AR resolution.
We‘ll also want visual helpers when positioning virtual objects. Expand ARSessionOrigin and enable the Point Cloud Manager and Plane Manager while checked into Runtime Overrides.
Lastly, drag the prefabbed AR Default Point Cloud and AR Default Plane helpers into the Managers we enabled.
Whew, that was a fair bit of configuration! But now we have an AR foundation established. Let‘s see it in action.
Visualizing Trackable Surfaces
If you enter Play Mode now and point your Scene camera around the environment, you should see yellow rectangles representing surfaces and an orange point cloud mapping out features.
As you move around, AR Foundation detects horizontal and vertical planes suitable for placing objects. It also senses key points around textures or edges.
We can leverage these plane detections to spawn 3D models realistically anchored in the surroundings.
AR visualizations help place digital objects realistically [Image credit: Unity]
Raycasting 3D Models into the Environment
Now let‘s dynamically spawn a 3D model leveraging those planar surface detections.
First, download any 3D asset you like and import it into your Unity project‘s Assets folder. I chose this free military drone model.
With your model imported, drag an instance into the Hierarchy panel. Then make it a Prefab by dragging it from Hierarchy to Assets in the Project window.
Next, we‘ll write a small script to handle raycasting and spawning. When you tap on the screen, we‘ll shoot a ray from the camera, detect the closest plane hit point, and instantiate our prefab there.
Here are the raycasting steps in RaycastDetectAndPlace.cs:
//Raycast against trackables on input touch
var hits = m_RaycastManager.Raycast(inputRays, trackableTypes, UnityEngine.XR.ARSubsystems.TrackableType.Planes);
//Ray hit a plane, spawn our prefab
if (hits.Count > 0) {
//Get hit pose and spawn our model there
var hitPose = hits[0].pose;
Instantiate(m_PlacedPrefab, hitPose.position, hitPose.rotation);
}Attach this script to our ARSessionOrigin. Then set the public Prefab field to our drone model asset from earlier.
When you enter Play Mode now and tap on plane detections in the Scene camera, your 3D model will spawn anchored realistically!
Building and Running the App on Android
Alright, we have a functioning small-scale AR experience. Now it‘s time to run it on an actual mobile device!
Open Build Settings in Unity and switch platform to Android, then click Switch Platform to auto-install dependencies.
Also enable the ARCore backend in XR Plugin Management to leverage Google‘s AR platform.
Finally, connect an Android phone with USB debugging enabled. Back in Build Settings, select Build and Run to compile and deploy direct to your phone.
After launching, tap around surfaces in your environment to spawn drones! Our AR app running on an actual smartphone!
Optimizing Performance of Mobile AR Apps
Augmented reality apps require heavy-duty processing. Here‘s some quick performance wins to maintain high frame rates:
- Simplify 3D models to minimize polygon counts
- Reduce particle counts on visual effects
- Limit unnecessary physics calculations
- Disable anti-aliasing features like MSAA
- Lower frame rates from 60 FPS to 30 FPS cap
I also highly recommending profiling Unity projects to pinpoint optimization opportunities. The built-in profiler exposes stats for CPU/GPU load, render throughput, memory allocations and more.
If you incorporate optimizations diligently throughout development, you can achieve fluid 60 frames-per-second AR experiences.
Expanding Our AR Development Skills
Congrats friend! You now have firsthand experience building an AR application harnessing Unity‘s incredible platform.
This project guided you through:
- Importing AR Foundation packages
- Managing AR session lifecycle
- Detecting planes and feature points
- Placing prefab objects with raycasting
- Building and running apps on Android devices
Of course we‘ve still only scratched the surface of professional-grade AR development. A few ideas for taking your skills even further:
- Expand to multiuser experiences on ARCloud
- Integrate spatial computing features like occlusion and physics
- Design more complex interactions using the XR Interaction Toolkit
- Port experiences to more platforms like HoloLens or smart glasses with Unity
If you found this interesting, I highly recommend Unity‘s book on AR projects which explores these advanced concepts and more.
At this point though, you should feel empowered to start constructing your own mobile AR apps leveraging Unity‘s incredible platform. Now go wow some people with spatially blended worlds!
Let me know if any other questions come up. Happy to help however I can on your AR developer journey.