︎
GeoquestAR

Tags: Augmented Reality, NREAL glasses, Interactive Storytelling, Location-Based Gaming
Time: two weeks (2024)
This project was built at 2024 MIT Reality Hack in a team of 5. It recieved 2 awards and was brought to AWE 2024 for demo:
  • Best Use of Meshy
  • WINNER Best Use of XREAL

INSPIRATION
GeoQuest AR was inspired by the desire to blend the exploratory spirit and exergame nature of geocaching with the digital interactivity and health incentives of Pokémon GO. By incorporating AR, we enhance the journey to the geocaching destination. We call this experience 'Geoquesting'.

WHAT IT DOES
A geoquesting app prototype that guides users on an interactive journey filled with quests and challenges. Successfully completing these tasks leads to the final destination and physical geocaching activities.

Core features:
  • GPS to AR Alignment: Utilizes Mapbox SDK to convert GPS data to scene coordinates.
  • NREAL SDK Interaction: Demonstrates prototype interactions with support from XREAL.
  • Dynamic Object Generation: Uses Meshy.AI for runtime object generation and Trilib to download GLB files over the network, enabling digital item placement at geocache destinations.
 
Tech Specification:
  • Unity Engine [2022.3.24f1]
  • XR Plugin Management [4.4.0]
  • Xreal MR SDK [2.2.0]
  • Mapbox SDK [2.0.1]
  • Meshy.AI API
  • OpenAI Unity [2.0] (by Sercan Altundas)
  • Mixamo animations
  • Trilib for glb support


[Project link at MIT Reality Hack]

[Demo at AWE 2024]
We modify the 3 stages of interactions before user reaches the wayoint to fit the theme of AWE.

1. Say an object you want to leave as a "cache" for future users (Whisper Audio-to-text, Meshy Text-to-3D. 2. Point to shoot bubbles down (NREAL Hand-tracking)).

3. Dance with "Palmer Luckey" (Meshy Text-to-3D, Mixamo animation). 4. At the waypoint, retrieve the 'cache' you created earlier and explore the 'caches' left by other users.

Somehow we got Palmer Luckey to try our app...