NVIDIA XR AI Brings Real Time AI Agents to AR Glasses
NVIDIA has launched NVIDIA XR AI in public beta, a developer library that embeds AI agents directly into AR glasses and extended reality devices. The platform is designed to help developers build hands-free, context-aware AI systems that work in real-world environments like factories, hospitals, and research labs.
Unlike chatbots or desktop copilots, NVIDIA XR AI processes live inputs from AR hardware, including video, audio, depth, pose, and sensor data, then connects those inputs to AI models, enterprise data, and reasoning tools. The result is an agent that can perceive its surroundings, retrieve knowledge, and guide a user through complex tasks without interrupting their workflow.
The platform integrates four key capabilities: multimodal perception from AR and XR devices, enterprise knowledge retrieval through NVIDIA NeMo Retriever, support for reasoning models including NVIDIA Nemotron and NVIDIA Cosmos Reason, and agent orchestration through the NVIDIA NeMo Agent Toolkit. Infrastructure runs across NVIDIA DGX Spark, DGX Station, and RTX PRO systems.
Several major organizations are already putting NVIDIA XR AI to work. Siemens is exploring how the platform can help factory engineers troubleshoot equipment and access maintenance information in real time through smart glasses. Rana, an AutoBio company, is deploying its LabOS system built on NVIDIA XR AI at Stanford and Princeton to guide researchers through gene-editing and stem cell experiments, hands-free, step by step.
In clinical settings, the Surreality Lab at the University of Pittsburgh Medical Center demonstrated how the platform can assist surgical teams with context-aware information without cluttering the surgeon’s field of view. VITURE has integrated NVIDIA XR AI into a wearable interface aimed at industrial workers. Innoactive is using it to capture and preserve spatial context during automotive design reviews. Atlantic Studios is even using the technology to let audiences explore an immersive Titanic scan through voice-guided discovery.
NVIDIA XR AI is positioning itself as the foundational layer for a new class of AI workers, ones that operate not on screens, but in the physical world alongside people. Developers can access the library and resources at the NVIDIA XR AI developer page.
