September 29, 2021
How to apply UX design principles in augmented reality
If you are a UX/UI designer who builds user experiences in digital environments, chances are you will be working with augmented reality sooner than you think. As AR applications rapidly break into the mainstream, making the user feel in control of a product becomes even more critical in user experience design.
This article breaks down the role that user experience design principles play in augmented reality application development, with a specific focus on UI design.
The article is based on a presentation by our senior software engineer, Gökhan Özdemir, for the “UX for AR” webinar. Watch the full recording here.
What is UX design for augmented reality?
User Experience Design, or UX, is the process of designing a product that considers the needs of the users and makes the user flow as seamless and intuitive as possible. Good UX always starts with putting the user at the center of the design process. It also relies on the principles of human psychology and empathy.
Now, what about UX for AR? In augmented reality apps, success means offering a great user experience through a seamless blend of hardware and software.
Augmented reality experiences are overlaid on the real environment, so the user experience is spatial and highly contextual. It makes designing UX for AR more challenging as designers need to think through spatial experiences. Getting it wrong can mean users have a less than stellar experience – and no one wants that.
User design can be tricky. Designing for a new technology that is only getting traction? Even trickier! Let’s explore the role of user experience (UX) design in AR applications — how to think through your user experience as a designer and navigate the technical decisions when creating an AR app.
You will learn how to create a compelling user experience for your AR application that considers the physical space and natural human interaction.
Five pillars of UX design for augmented reality
Users prefer to interact with elements of an interface discreetly, not to be reminded of what the interface contains. This is different from the traditional user experience (UX) associated with conventional websites and mail applications. The UX for augmented reality (also known as 3D user interface) concept emphasizes interaction and visual interest above all else. Users are interested in entering the virtual space and are not distracted by surroundings that are not real.
Our five common UX design pillars for AR will help you define the considerations you’ll need to make when designing your UI and experiences for virtual objects.
Kick-off your design process by considering these criteria:
- UI (User Interface)
While it’s crucial to consider the first two pillars (environment and movement) designing for AR, the last three (onboarding, interaction, and UI) are equally crucial for both 3D and traditional 2D screen space UI.
As augmented reality experiences are spatial and always interconnected with the real world, the environment plays a key role in the design process. The environment can be broken down into four most common categories of space, defined by the distance from the user.
Examples of AR in the intimate space include face filters (like Snapchat or Instagram), hand tracking, or hand augmentations if you use a head-mounted AR display.
Moving to personal space, augmented reality experiences might feature real objects, people, or the area around you. Featured in the video below, you can see a learning-focused AR experience that uses educational models to animate the chemistry concepts through an interactive digital layer.
Another example of using augmented reality in personal space is popular table-top and card games and augmented packaging. Think augmented pizza boxes or collectible cards with augmented characters that interact with each other.
Discover more: Wizzzi uses AR to gamify loyalty program for retailers
Next up is the social space. If you pan the camera further away, you will target the area that can be occupied by other people, unlike in a personal space where you have more privacy. This space segment is widely used for multiplayer AR games or augmenting objects on a scale, from the furniture to monuments and buildings.
In many cases, AR experiences in public space are anchored to specific locations with enough area to place an augmentation or sites that should be tracked in AR. The mumok AR experience in Vienna is a perfect example of the AR in public space where the entire building is tracked, using the Wikitude Scene Tracking feature.
The success of any new product or service directly depends on how well it integrates with today’s users’ minds — both physically and psychologically. Movement makes the next UX design pillar. When you design the experience, you want to use the area around the user most of the time.
As smartphones and head-worn devices give a limited view into the environment, the designer’s primary task is to guide the user. By including the navigation elements on the screen, you will be steering the user’s gaze, helping them get around and move along the experience.
While you are visually guiding the user, it’s essential to keep in mind not to dictate to go in specific directions. This might lead to unwanted hiccups in the experience or even cause accidents.
The next pillar we are going to explore is the user onboarding. Creating user-friendly and engaging augmented reality experiences can be a challenge. It’s not enough to just put some markers around your location or overlay some information on top of an image. You need to understand what the user is looking at and how they are using it. When creating the AR experiences, keep in mind that the most important thing for your users is not accuracy but usability.
Another factor to consider is that different devices have various technical limitations in supporting AR features. Markerless AR, for instance, would require the user to move the device around, so that computer vision algorithms could detect different feature points across multiple poses to calculate surfaces.
The scanning process takes no time for newer devices with an in-built LiDAR sensor (like iPad Pro). But for other devices, your users might appreciate a comprehensive onboarding UI. The pop-up menu or instructions should guide the user on the following steps to successfully launch and run an AR experience.
To launch a tracking algorithm, you might want to use a sketched silhouette of the desired object to provide a clue on the shape and pose to prompt the user to align the view with the real object. Read more about the Alignment Initializer feature in our documentation.
Taking the onboarding offline, sometimes the physical methods like signage are used to communicate about the AR app, provide a QR code for quick download and mark the exact standpoint for an optimal experience.
Once the AR experience is launched, we are transitioning to another UX design staple – interaction. During this phase, your user will benefit from the intuitive and responsive interaction. When designing for touch, you are most likely be using these most common gestures and prompts:
- Tap to select
- Drag starting from the center of the object to translate
- Drag starting from edge of the object to rotate
- Pinch to scale
Responsive interaction means taking into account the distance from the desired objects to the camera, which will define how easy or difficult it is for the user to interact with it. To facilitate the interaction with farther placed objects, consider increasing the sphere’s bounding box to make it less dependent on the distance to the camera.
Minimizing input via finger might also be a good idea, especially when designing for tablet users. As most of the tablets are held with two hands, some UI or interaction elements placed in the middle of the screen will be very hard to interact with. Instead, use gaze input like triggering intro, interactions, or buttons in the augmented space by looking at them long enough. You might know this from VR where you don’t have any controllers and experiences are mostly driven by gaze.
Consider using accessibility features, especially if you are designing for a broader audience. This way, you let the user rotate or reset the position of an augmentation instead of walking around it.
UI (User Interface)
The final principle we want to highlight is UI, which consists of augmented and traditional screen space. Depending on the use case, you will be using them interchangeably. While UI in the augmented space boosts immersion as the user perceives it as part of the experience, screen space UI is sometimes easier to read and interact.
Designing with humans in mind
AR can improve people’s lives simply by allowing them to experience something that wasn’t possible before. Applying UX principles to AR can help designers create experiences that are clear, integrate easily into daily life, and create powerful emotional responses.
The guidelines we’ve shared aren’t magic bullets, but they do place fundamental guidance around where designers should be focusing their attention when crafting an experience for a user of any age.