A MULTI SENSORY MEDIA ART PERFORMANCE CONTROLLED BY NEURAL FUNCTION

Melbourne Knowledge Week Festival, Melbourne Meat Market, AUSTRALIA

the loop places the human body at the centre of a digital experience. This project investigates human-computer integration. We create a feedback loop between people and machines, forming a cycle of information that is shaped by a collusion between active brainwaves and real-time multi-sensory art. the loop involves sight, sound, smell, taste and touch.

the loop features two performers: a live musician/programmer, and a dancer. An EEG headset reads electrical activity that’s occurring inside the dancer’s brain, and sends this data to our custom designed system. Our system transforms real-time brain activity into abstract audio-visuals, it also provides aromatic, gustatory and haptic feedback to the dancer. The dancer’s neural function controls the sensory environment.

The dancer’s movement is tracked and their body is projection mapped with imagery effected by their neural function. Their brain function is also visualised and projected onto a large media surface at the rear of the stage. The dancer’s neural function also controls a vibrating vest that they are wearing and it effects the release of aromatic and gustatory (taste-based) stimuli.

When the dancer hears and sees the audio-visuals, and when they feel, taste and smell the sensory stimuli that is triggered from their neural function, this has an impact on their brain, and this impact then stimulates new brain function that, in-turn, changes the audio, visuals, aromas, gustatory and haptic effects. Therefore, a cyclical feedback loop is created between the human body and a real-time reactive multi-sensory environment.

PluginHUMAN: Betty Sargeant & Justin Dwyer. THE LOOP a multi-sensory performance controlled by real-time neural function

PluginHUMAN: Betty Sargeant & Justin Dwyer. THE LOOP a multi-sensory performance controlled by real-time neural function

CREDITS

Dr Betty Sargeant ~ Artist

Justin Dwyer ~ Artist

Andrew Ogburn + Justin Dwyer ~ Music

Penny Baron ~ Director and Choreographer

Dr Kate Hunter ~ Assistant Director

Thanks to Florian ‘Floyd’ Mueller and the Exertion Games Lab, Monash University, Australia.

RELATED PROJECTS:

PluginHUMAN DREAM 2.2 - Interactive brainwave art installation. Betty Sargeant and Justin Dwyer.
DREAM 2.2
PluginHUMAN: Betty Sargeant & Justin Dwyer. Inter-Dream: a VR Lullaby Machine
INTER-DREAM
PluginHUMAN: Betty Sargeant & Justin Dwyer. BREATHE: LED immersive installation.
BREATHE