Research

Research priorities

Embodied, multimodal, and adaptive human–AI collaboration

As automation changes technical work, people need capabilities in system oversight, maintenance, diagnostics, safe human–robot collaboration, and exception handling. We engineer XR environments to investigate how people and AI develop these capabilities together.

1. 

   01 · Highest priority

   Embodied AI collaborators and hybrid intelligence

   Our first priority is investigating spatially present AI partners in technical XR simulations: agents that can demonstrate procedures, co-manipulate objects, observe behavior, and adapt during a shared task.

   We study collaborative agency, shared spatial attention, and mutual intelligibility: how a person and an AI partner make their intentions and actions understandable to each other.

   Evaluation focus: collaborative fluency, adaptive co-presence, and distributed knowledge growth.

2. 

   02 · Assessment and adaptation

   Multimodal real-time skill assessment and adaptive scaffolding

   We investigate how hand tracking, head pose, interaction events, timing, and error patterns can inform assessment of procedural expertise, cognitive load, and performance in technical tasks.

   Our aim is to connect domain-specific task evidence with scenario adaptation and timely scaffolding. Performance data and expertise from technical partners ground the assessment.

   Evaluation focus: procedural skill, error patterns, assessment validity, and the effect of adaptive support.

3. 

   03 · Technical teaming

   Human–robot and human–AI teaming in digital twins

   We design XR training environments for practicing safe collaboration, troubleshooting, exception handling, and system oversight with virtual robots or automated cells.

   This direction connects technician pathways with human-centered automation, including explainable feedback and opportunities for people to question or override AI recommendations.

   Evaluation focus: skill transfer to technical settings, trust calibration, human oversight, and system-level outcomes.

The framework enables the research

Our web-based XR stack uses Babylon.js or Three.js with WebXR, targeting Meta Quest-class devices. Shared conversation, voice, avatars, and interaction capabilities provide a foundation across the platform family.

Conversational agents, generative 3D tools, and content authoring help us build and iterate. We use generated assets as inputs to interaction-ready, part-aware scenarios; our research contribution centers on spatial collaboration, domain-grounded adaptation, and evidence of performance and transfer.

Explore the platform family and project portfolio →