Mary Huang
Star Wars: Tales From the Galaxy's Edge

ILMxLAB
2020-2021
UI Lead

"Star Wars: Tales from the Galaxy's Edge is an action-adventure virtual reality experience from ILMxLAB that lets you travel across the galaxy to the planet of Batuu and live out your own unforgettable story."

ILMxLAB Experience Launch Page

As the UI Lead, I was responsible for end-to-end design for all in-game UI features. Menus, typography, icons, AR-in-VR hud elements, custom UI shaders, tutorials, localization, settings, motion graphics. I also designed all the mission progression, challenges, journal, and worked with game designers to untangle the spaghetti of nonlinear story progression. Shipped in Unreal Engine.

I was the first UI/UX designer ever hired at ILM. I shipped all the UI for the first launch in November 2020 in just 5 months, working mostly with the art director, game director, and engineering leads. For our follow-up release 10 months later, I maintained and improved core features and also expanded on augmenting storytelling with more in-world UI.

Achieving "AR in VR"

With so many different mechanics to represent, I championed a philosophy of 'less is more'.

I built an in-world spatial UI system using a collection of shaders with different parameters. The primary goal was to keep it minimal and avoid too many elements visible at a time. Being selective with these parameters allowed me to create differentiation between game mechanics.

  1. Distance culling. Most elements were set to only be visible if the player was within 3 meters.
  2. Overlay. Some elements (like the compass target marker) needed to be visible on top of all geometry.
  3. Stay constant size in viewframe = scale up the element from a distance.
  4. Small animations add a lot: fade in on approach or constant rotation.

All-Kit. 'augmented' tool icons appear just when the player presses the joystick. In-world helper icons for puzzles appear at 3 m.

The compass. I removed competing features to make an objective marker that is located in world space at a target point, draws on top of everything, and maintains a minimum size.

In-world shop UI. The credits display animates as the player approaches the counter.

Inventory Pouch

The number of inventory items increased in the second release. Working closely with the gameplay engineer, we designed this expandable inventory that balanced the need for quick access with the need to hold many quest items.

We gave priority to weapons in the 'mini' slots. Later we also added an 'organize' mode to make those actions more efficient. I also gathered feedback about desired improvements, and worked with our gameplay engineering lead to ship the shoulder holsters feature, allowing an item to be stored on the player's back.

The inventory pouch was one of the most complicated UI mechanics. The fun idea of turning a conventionally 2D UI element into a 3D object doesn't scale well. Diagetic elements have to follow laws of reality and industrial design, thus inheriting all the problems of real-life objects. If we had more inventory items in a future release, we would have moved most of the feature to the 2D Pause Menu.

Tutorials

Game mechanics need to be paced out for learning and building your character.

In VR, a lot of common features are stuck to your body, or mapped to controller buttons that you can't see. I recommended that we allow the player to attach each object and have a moment to learn how to use it.

I worked with the level designer and gameplay engineer to build this intro sequence and the 20+ tutorials spread across this level.

The inventory pouch was extra tricky, with a lot of back and forth on where it should be. It's on the chest, so we emphasize it in the beginning, reinforcing the action of picking up an item and stashing it in the pouch.

IG88: Being a Droid

I created a unique HUD UI system to give a distinctively assassin droid play experience. The monochrome green and geometric elements reference vintage arcade graphics and real-life military HUD displays.

Working with the script, I created the "scan" feature and corresponding Aurebesh translation animation to punctuate jokes and plot themes.

Small details like the audio bars, rotating targeting reticles, and hmd-oriented scanlines are subtle special touches to give that "being inside a droid" experience.

To keep those draw calls performant, I animated all motion graphics with meshes and shaders using timeline nodes in Unreal.

Play it, it's super fun in VR.

Compass Navigation & Gauntlet

The gauntlet held 3 main play features: Holocalls, Scanning, and Compass.

The compass proved to be the most important game element to guide the player to the objective, and this feature proved extra problematic as a diagetic VR element.

Challenges:

  • A 3D arrow mounted to your wrist is way more complicated than a 2D element on a flat screen.
  • Our levels have a lot of winding paths and ‘doglegs’ due to rendering and level streaming requirements. No long clear sight lines.
  • This feature had to work across every single mission across multiple different levels.


    • Shipping It:
  • I condensed a wide range of ideas from different game designers.
  • I removed competing features and made an objective marker that is located in world space at a target point.
  • Audited all the missions and collaborated with level designers to get it all working.

    • I also improved usability and ergonomics on the 3D gauntlet design, working with our gameplay engineering lead and art director.

    Tldr: The initial version had toggle switches, which were impossible to use. The hand models are giant, especially for short people or kids, so I carefully nudged the position of the gauntlet and buttons to make it work. As a petite female, I'm good at testing for the range of shorter people.

    Settings...Everyone's favorite menu feature!

    I made the settings easily accessible from both the main menu and the pause menu. Many settings have states dependent on other setting states, so keeping the design files well-documented saved a lot of engineering headache.

    We shipped some highly requested settings in a patch after launch to appease experience VR players who expect much more detailed control over their play experience.