© Dr. Giusy Caruso, pianist & artist researcher
This artistic research aims at studying the process of my interpretation and practice of the 72 Etudes Karnatiques pour piano by the French composer Jacques Charpentier (1933-2017) and at investigating the “mirroring” relation between musical intentionality and gesture of a piano performance.
I empirically analysed this process by using technology to understand how my artistic practice could influence changes in my interpretation of the piece (musical intentionality) and in my performance gesture.
In this way, this artistic research would contribute to the established interpretative tradition on contemporary music and open new perspectives on the methodological approaches for contemporary music performers engaged in artistic research.
I worked on my interpretation and performance of the 72 Etudes Karnatiques by Jacques Charpentier (1933-2017) by experimenting with new strategies for practicing in collaboration with the composer and Indian music experts.
ph. Alain Machelidon (2017) “Jacques Charpentier & Giusy Caruso”
Focusing on the exploration of my piano practicing and specifically on the relation between musical intentionality and performance gesture (embodied music cognition), my artistic research entails both the artistic and scientific perspective.
The analysis of my performance practice was conducted by traditional methods (observation and evaluation of my video and audio recordings)and by the innovative use of technology as an “augmented mirror” for monitoring music performance, in particular the Motion Capture System (MoCap).
The aim of this study (see Caruso et al., 2016) is to go beyond the local musical score level by paying attention to the stylistic and artistic interpretations that the performer develops during the artistic process. In this approach, the performer-based analysis is supported by technology as a “mirror” (technology-enhanced mirror) that provides top-down and bottom-up enhanced feedback regarding the performance (Motion and Audio analysis).
Qualitative/top-down approach= written self-description and self-evaluation of audio and video recordings
Quantitative/bottom-up approach= objective analysis of audio and motion recordings
The feedback is re-injected in the artistic process that defines and shapes the expressive intention of the performer. Technology used as an “augmented mirror” thus becomes a tool in a reinforcement dynamic within the artistic process to better understand the artistic interactions that drive the expressive performance intentions.
In 2015 coached by Dr. Luc Nijs, I started working with the Motion Capture System at the IPEM Lab – University of Ghent – to study the evolution of my gestural approach to the music and the different levels of embodiment and expressivity (Leman 2007). Despite the traditional 2D-video recordings, the 3D camera technologies offer the most promising approach by providing a 3D-visualization of the body in performance in an avatar, together with objective data like displacement, velocity, quantity of motion, on a 3D-body posture and movement.
Prof. Dr. Marc Leman, director of IPEM, presents “Motion Capture of a Piano Performance”
Data are other ways to represent and “talk” about music performance.
Here is a comparison of the motion data, specifically the vertical displacement (y) of my right hand (green line), my right elbow (blue line) and my head (yellow line) during the execution of this musical passage from the 72 Etudes Karnatiques, taken as a point of reference. The first graph represents my initial gestural approach to the music and the second graph the developed approach after a period of three years practicing with the composer and Indian music experts. The graph of developed approach shows similar trajectories in the right hand, right elbow and head, therefore a major coordination of all the three parts of the body, which are more pronounced (developed level of embodiment)clearly replicating the musical shape, in particular the accented-notes.
To inspect the audio waveform, I used Sonic Visualiser, an open source application for viewing, analysing and annotating music audio files. The use of Sonic Visualiser enables musicians to select fragments on which to apply the wanted analysis through a set of plug-ins (in the vamp plugin format) to be added to the software in function of the analytical demands.
By importing a waveform to the software and by applying specific plug-ins, it is possible to obtain straightforwardly the note onset (directly from the software or through manual annotation), timing, dynamics (like the energy of the signal approximated by computing the root-mean-square, RMS) and spectrograms over the given segment.
To correlate the qualitative descriptions and quantitative data (motion capture and audio data), I used the multimodal annotation software ELAN.
In ELAN, it is possible to import the video recordings, the audio tracks and data gathered from the motion capture (displacement, velocity and acceleration) plotted in graphical representations. This screen shot, taken from one of my Elan projects, shows the parallel configuration of four layers. The quantitative measurements are displayed in the following order from up downward (1) the video, (2) the motion tracks of head, right hand, and right elbow, (3) the audio waveforms, while the qualitative de-scriptions (4) are shown below in the segmented annotations.
The interdisciplinary approach of my methodology is summarized in a triangular artistic research system (TARS).
In this triangular artistic research system (TARS), artistic research is placed in the middle as a synthesis (see the blue arrows) of the interplay between theory, practice and production, which are manifestations of the entire artistic process (see the red arrows). Practice and production furnish the concrete territory of experiences where theoretical concepts are applied. Vice versa, theory concerns the knowledge and the strategies for reflecting, documenting and disseminating an investigation through practice and production. A mutual influence happens also between practice and production because practice is finalized to production and production is used as a feedback for practice. All these three domains entail the common objective of documenting and disseminating the artistic process as an artistic research.