Thursday, December 12, 2024

Real-World Experiments Reveal Indoor AR Challenges

- Advertisement -

Osaka University researchers have discovered why AR fails indoors and proposed a solution—find out how it could change everything!

Testing conditions: (a) The LiDAR and monocular cameras are blocked to ensure only specific sensors are utilized. Note that the IMU is always utilized. (b) There are three patterns for the complexity of the environment. Wall: a white wall with few visual features; Shelf corner: includes books in a bookshelf providing several visual features, as the red dots show. Crowded: the environment includes miscellaneous clutter to increase the visual features. (c) Different brightness levels. 7 lux is the expected brightness in a movie theater, whereas 200 lux is a well-lit office space. (d) Different movement types. The XY-stage is specially designed to conduct repeatable experiments on the floor plane. Credit: 2024 Yamaguchi et al., Experience: Practical Challenges for Indoor AR Applications, ACM MobiCom '24
Testing conditions: (a) The LiDAR and monocular cameras are blocked to ensure only specific sensors are utilized. Note that the IMU is always utilized. (b) There are three patterns for the complexity of the environment. Wall: a white wall with few visual features; Shelf corner: includes books in a bookshelf providing several visual features, as the red dots show. Crowded: the environment includes miscellaneous clutter to increase the visual features. (c) Different brightness levels. 7 lux is the expected brightness in a movie theater, whereas 200 lux is a well-lit office space. (d) Different movement types. The XY-stage is specially designed to conduct repeatable experiments on the floor plane. Credit: 2024 Yamaguchi et al., Experience: Practical Challenges for Indoor AR Applications, ACM MobiCom ’24

Augmented reality (AR) apps on smartphones are top-rated. They let you see things like how furniture would look in your home, help you use maps better, or let you play fun games. A famous example is Pokémon GO, where you catch digital creatures with your phone. But if you try to use AR apps inside a building, they might not work because the technology has trouble without a strong GPS signal.

After conducting thorough experiments with smartphones and users, researchers from Osaka University have uncovered the detailed reasons behind these problems and proposed a possible solution.

- Advertisement -

To evaluate the performance of these systems, researchers conducted case studies, including setting up a virtual classroom in an empty lecture hall, where participants were tasked with arranging virtual desks and chairs in an optimal layout.

113 hours of experiments and case studies were conducted across 316 patterns in real-world settings. The goal was to identify and analyze AR failure modes by selectively disabling specific sensors and altering environmental and lighting conditions.

The study revealed that virtual elements in augmented reality often “drift” within the scene, which can cause motion sickness and diminish the sense of realism. The findings showed challenges with visual landmarks, which are hard to detect from a distance, at extreme angles, or in low-light conditions. LiDAR can also be unreliable, and the inertial measurement unit (IMU) accumulates errors at high and low speeds over time. To overcome these issues, the researchers suggest using radio-frequency-based localization, such as ultra-wideband (UWB) sensing, as a possible solution.

Ultra-wideband (UWB) functions similarly to WiFi or Bluetooth, with popular applications including Apple AirTag and Galaxy SmartTag+. Unlike vision-based landmarks like QR codes or AprilTags, radio-frequency localization is less influenced by lighting, distance, or line of sight. 

Researchers anticipate that in the future, UWB or other sensing technologies such as ultrasound, WiFi, BLE, or RFID could be combined with vision-based methods, resulting in significantly enhanced augmented reality applications.

Reference: Experience: Practical Challenges for Indoor AR Applications, DOI: 10.1145/3636534.3690676

Nidhi Agarwal
Nidhi Agarwal
Nidhi Agarwal is a journalist at EFY. She is an Electronics and Communication Engineer with over five years of academic experience. Her expertise lies in working with development boards and IoT cloud. She enjoys writing as it enables her to share her knowledge and insights related to electronics, with like-minded techies.

SHARE YOUR THOUGHTS & COMMENTS

EFY Prime

Unique DIY Projects

Electronics News

Truly Innovative Electronics

Latest DIY Videos

Electronics Components

Electronics Jobs

Calculators For Electronics