← Back to Projects

A Sensory Experience

2025

Immersive VR Installation

Inverse Kinematics, Unity, Whole-body 3D tracking, VR, Meta SDK, eye-tracking

Installation

A Sensory Experience Through the Entanglement of Dynamic Virtual Memories

Marta Shilova, Hui-Ting Hong, Chu-Yin Chen, Ray-Kuang Lee

ACM International Joint Conference on Pervasive and Ubiquitous Computing 2025

Intricate Memories is an immersive and interactive virtual reality installation that explores how bodily sensations, gaze, and multimodal feedback affect the participant's sense of presence through a responsive avatar. The system integrates real-time body tracking, eye tracking, and generative AI to enable a dynamic entanglement between the participant and their avatar, whose appearance evolves based on emotional and sensory stimuli. By combining multiple input modalities and sensory outputs, this VR space supports non-linear interactions guided by gaze, allowing the avatar's body to act as a medium of somatic memory. The project examines how three types of memory : somatic, mental, and collective artificial, reshape presence and body representation in immersive environments. It also critically examines how media representations, especially those shaped by disinformation, can alter our affects.

A Sensory Experience — 1

Team & Partners

  • Avatar shader/morphologies|Marta SHILOVA
  • Unity, full-body IK, eye-gaze interaction system|Hui-Ting Hong
  • Real-time AI image generation pipeline|Hui-Ting Hong
  • Touch Designer, real-time visual processing|Marta SHILOVA
  • Video Montage|Marta SHILOVA
  • Developed as a part of the Orchid International Collaboration Project led by Pr. Chu-Yin CHEN.

OFFICIAL TRAILER