ISCA 2021 Tutorial

ILLIXR: Illinois Extended Reality Testbed

Enabling Augmented and Virtual Reality Architecture and Systems Research

Friday, June 18, 2021

9 AM - 12 PM EDT

 

Overview

Augmented, Virtual, and Mixed Reality, collectively known as Extended Reality (XR), are emerging technologies that will impact most aspects of our lives, ranging from education and science to social interactions and entertainment. However, modern XR systems are unable to fully realize the potential of XR, owing to a severe performance, power, and quality gap that exists between the state-of-the-art systems of today and the idealized XR systems of tomorrow. Closing this gap and designing systems that are several orders of magnitude faster, smaller, and more energy efficient than modern systems is challenging for two main reasons: 1) the diversity and complexity of domains and tasks within XR is immense, and 2) state-of-the-art XR systems are closed source and closely guarded by a few key industry players.

In order to enable architecture and systems research in XR, we developed ILLIXR, the Illinois Extended Reality Testbed, available at illixr.github.io. ILLIXR is the first fully open-source full system testbed for XR, and contains several state-of-the-art XR components, connected via an efficient, flexible, and modular runtime framework. ILLIXR is OpenXR compliant, enabling it to run any OpenXR-based application, including those developed using game engines, on a real headset such as Project North Star. Finally, ILLIXR provides both component- and system-level metrics for both performance and quality, enabling end-to-end experimentation and optimization.

ILLIXR is an end-to-end system that can be used by computer architects, system designers, compiler writers, and algorithms researchers to not only innovate within their own research areas, but to co-design different parts of the system, and understand how their ideas impact the final user experience. Such cross-layer innovations are increasingly important for efficient system design as we deal with the end of Moore's law. Examples of research being conducted with ILLIXR include selection and generation of compute and memory acceleration primitives, full-system SoC design, inter-accelerator communication, programming languages and OSes for domain-specific systems, real-time scheduling, device-edge-cloud partitioning of workloads, and many other aspects of designing future XR systems in particular and domain-specific systems in general.

This tutorial will first provide an overview of the XR domain, followed by a deep dive of ILLIXR and its components, runtime, and metrics. We will show a hands on demo of ILLIXR, including setup, build process, and usage with actual OpenXR applications on a real headset. We will conclude with an overview of gained insights and ongoing research and development.

Target Audience

The target audience for this tutorial is the entire ISCA community. ILLIXR enables research in end-to-end design of future XR systems in particular and domain-specific systems in general.

Resources

"Exploring Extended Reality with ILLIXR: A new Playground for Architecture Research," Muhammad Huzaifa, Rishi Desai, Samuel Grayson, Xutao Jiang, Ying Jing, Jae Lee, Fang Lu, Yihan Pang, Joseph Ravichandran, Finn Sinclair, Boyuan Tian, Hengzhi Yuan, Jeffrey Zhang, and Sarita Adve

NVIDIA GTC'21 talk and slides.

Tentative Schedule

9:00 - 9:10 AM | Introductions

9:10 - 10:25 AM | XR and ILLIXR overview

10:25 - 10:30 AM | Break

10:30 - 11:10 AM | Hands-on sessions and demos

11:10 - 11:15 AM | Break

11:15 - 11:45 AM | Results from ILLIXR and ongoing research and development

11:45 - 12:00 PM | Open discussion

Organizers

Sarita Adve (University of Illinois at Urbana-Champaign)

Muhammad Huzaifa (University of Illinois at Urbana-Champaign)

Ying Jing (University of Illinois at Urbana-Champaign)

Yihan Pang (University of Illinois at Urbana-Champaign)

Boyuan Tian (University of Illinois at Urbana-Champaign)