Handheld AR Onboarding

  • 2019–2020

  • Role: UX/UI, Motion, Visual Design

  • Freelance — Team: Designer, Engineer, PM

A UX framework consisting of animated instructions to provide guidance on how to set up and placing content in mobile AR apps. The UX animations are open source and available on the Unity Asset store for AR developers and content creators to use.

a sample AR mobile app using the design framework






Initial Idea & Concept

I work on a lot of mobile AR apps and prototypes that revolved around placing 3D content in the environment. I found that a common first step required users to first scan their surroundings and understand how to place the augmentation in their world before they can interact with it. I solved this problem by creating a clear and simple user flow that guides users with this initialization process. The UX consist of three parts for users, including teaching them when and how to scan, when and where to place, and provides visual feedback with each of these steps.

Onboarding is necessary to improve usability, especially for those experiencing AR applications for the first time. It’s important to teach users how to engage with the technology and make sure their experiences go as smoothly as possible. I wanted to share this framework with developers and other creators who are working on similar types of mobile AR projects, to use as a quick resource. This way AR creators and developers can incorporate this kind of UI guide at the start of their app, and continue to focus on content development and creating immersive experiences.







UX decisions applied with interaction design principles







visual design of the framework guides







Projects Using AR Onboarding

sculptural typography with reflections


reflective bean sculpture

Bauhaus Bauspiel blocks


poké ball with a custom reticle







Step 01 : Finding a Surface

Both ARKit and ARCore SDKs requires the users to move their device slowly in order to properly detect feature points and planes in the environment. The UI animation is meant to guide users as they move their device horizontally and vertically around their space. Once a plane is detected the animation will fade out.

Initially my animation only showed the device moving horizontally. After conducting user tests, I found that the animation didn't convey enough information to instruct users how to successfully detect planes and feature points with each try. Having an iterative flow from designing the visuals, exporting the assets for the mobile app, and testing these ideas, I found the final animation works well for users.







Step 02 : Plane Rendering

When exploring different types of plane rendering, I found that feathering the edges of the planes, rather than using hard edges, made the planes appear more integrated on various surfaces. I created several patterns for custom plane visualization, that can match any app aesthetic or the style of a brand.

Even though this type of visualization is important in the current state of the technology, it can be designed to feel integrated with the rest of the application and content.







Step 03 : Tap to Place

The next animated instruction indicates to users that they can tap anywhere on their device to place AR content. I also designed a 3D reticle that exists in world space and attaches to found planes. This reticle helps users place content accurately and precisely, as it's a clear visual location indicator. When prototyping, the reticle has proved to be extremely helpful when working on apps without features that allow you to edit (like reposition, rotate, delete functions) after placing the AR content.










Available in AR Foundation

AR Foundation is Unity's cross platform framework that allows developers to build both ARKit and ARCore apps from a single project. The sample project includes all the necessary pieces that creators and developers need to start building AR apps quickly and efficiently includes ways to demonstrate plane visualization, object placement, etc. My UI/UX mobile AR framework is included in the AR Foundation Github sample repository. This work has been featured in events and keynotes including the Game Developers Conference 2019, Unite Shanghai 2019, and Unite Seoul 2019, as well as the Handheld AR Ecosystem blog post.






V2 — UX Animation Updates

The goal is to cover all the unique tracking types for mobile AR: plane tracking, image tracking, object tracking, face tracking, and body tracking. These animations were released as a more robust project on the Unity Asset Store. Improvements include more consistent visual style for all animations, as well as exporting animations to transparent video for easier playback and file management.