Session 8

Presenters: Roger Alsop, Yichen Wang, Jesse Austin-Stewart

02:30pm: Roger Alsop Lost Horizons, an immersive computer-based work integrating text, visual imagery, spatialized sound in performance

This paper discusses the effect of lockdown in Melbourne on the authors/artists, who are from very different cultures and histories, and how this effect may be represented through artistic works. It aims to offer insight to ways of communicating what cannot be communicated using traditional forms of artistic creativity, and to explore alternative ways to connect to an emotional state through art and technology. This has significance in developing creative approaches to integrating diverse art forms within a specific intention, to express the impact of isolation caused by lockdown for the artists.

03:00pm: Yichen Wang Sonic Sculptural Staircase in Highly Immersive and World Aware Augmented Reality

Modern head-mounted augmented reality technology has now advanced to the level that it can be said to enable a Highly Immersive and World Aware (HIWAAR) experience. Critical to providing such an experience is the ability of some modern systems, like the Microsoft HoloLens, to capture video information of the surrounding physical environment in real time and to build models of that environment which enable a user to be “world aware” in the augmented world. At the same time, such systems can provide graphical images and audio that surround and augment the user experience in a highly immersive manner. Works that utilize such HIWAAR technology for the creation of augmented multi-sensory experience have been claimed to be effective in activating users’ engagement and enhancing art experience. However, few works have specifically considered how augmented art experiences should be designed and fewer have provided rigorous evaluations of the effectiveness of such experiences.

My work explores the affordances of HIWAAR technology with the respect to sonic-tactile interaction in providing an augmented sculptural staircase appreciation experience. I argue that such powerful use of technology prompts a more active, embodied, and emotionally engaging experience to people in discovering, interpreting and acquiring knowledge. I demonstrate this by developing and evaluating an HIWAAR installation, Sonic Sculptural Staircase (SSS). My work commences using interviews with art professionals seeking for design inspirations for the HIWAAR augmented sonic sculpture (or sculptural elements) experience. Drawing on results from these interviews, the SSS was designed and developed. Two different experience conditions, the informational experience and the experiential experience, are introduced in SSS to explore ways in which sound can be integrated with visuals and interaction for people to understand and appreciate a sculptural staircase. Results from user evaluation show that the SSS presented a pleasant augmented sculptural staircase appreciation experience along with enhanced awareness of the physical piece, on both experience conditions over time. Furthermore, preliminary evidence from follow-up studies appear to show that the experiential experience, which utilized real-time generated electronic music and field recordings mapping related context, has a more lasting impact than the informational experience on the way that participants view the sculptural staircase in their daily lives.

Based on my work, I conclude the aggregated sonic, visual and tactile HIWAAR experience is far more perceptually immersive and effective in communicating compared to a simple visual experience. Positive results revealed from user evaluations preliminarily support the effective-ness of utilizing such technology for the design of other augmented multi-sensory experiences for relevant communities.

03:30pm: Jesse Austin-Stewart Multichannel Monophonic Spatial Application

Exploring space within sound works has led to the creation of many varied spatial loudspeaker systems. The technology needed to create these systems is often beyond the means of individuals to recreate in their personal home or studio environment. To combat this, there have been a variety of applications designed to allow individuals to compose spatially for these systems in their own home by allowing them to listen to a simulation (often in stereo) of how their work would sound over a particular loudspeaker system. These applications are often designed, with particular systems in mind or at least place the listener in a fixed listening position. These applications don’t give the listener/composer the opportunity to simulate novel loudspeaker systems.

This paper will review some of the applications designed to allow listeners/composers to simulate various loudspeaker arrays and will also introduce a new application designed by the author that allows for the simulation of novel 2-dimensional loudspeaker systems of up to 64 loudspeakers from various listening positions. In designing an application to simulate novel loudspeaker configurations, composition for novel non-traditional loudspeaker systems becomes much more accessible. The newly designed application also makes for simpler engagement with particular spatial composition approaches for those who are hard of hearing.