IRCAM Gestural Description of Sounds

From XDIF

Jump to: navigation, search


Contents

Overview

Hypothesis

The type of gestural description - causal vs morphological - is linked to the level of identification of a sound?

Our hypothesis is based on a recent article (Lemaitre et al, JEP 2010)

  • A clear influence of the causal uncertainty of the sounds:
    • the more the sounds are identifiable, the more they are grouped together because the events are similar
    • the less the sounds are identifiable, the more they are grouped together because of acoustical similarities
  • A clear influence of the expertise of the participants: expert listeners are more keen on grouping sounds on the basis of their acoustical properties, while naive listeners group together similar events

Hence our hypothesis is that gesture strategies chosen by people while they are listening to environmental sounds can roughly be put into two categories:

  • First, people try to mimic action they guess it has produced the sound (Symbolic)
  • Second, people try to follow synchronously the temporal evolution of some sound features such as loudness, pitch etc. (Morphologic)

Thus, the gesture strategies are related to the ability of a participant to identify or not the cause of a sound.


Experiment

Task: Performing free-hand movements the more associated and synchronised to the listened sounds.


Set-Up and Storage

  • Motion-Capture system: Art-Track
    • One marker on the hand
  • Stereophonic diffusion

Recordings

  • Motion capture data: positions (x,y,z) and orientations (pitch, roll, yaw)
  • Audio recrdings (synchronised to the gesture)
  • Video recordings (synchronised to the gesture)


Collaborators

Project from Baptiste Caramiaux PhD (IRCAM, IMTR Team) with Tommaso Bianco (IRCAM, IMTR Team), Patrick Susini (IRCAM, PDS Team), Frédéric Bevilacqua (IRCAM, IMTR Team) and Norbert Schnell (IRCAM, IMTR Team)

Personal tools