VR10–Whimsies Attic

We had a practical session about mixing and mastering for our project this week. The immersion, Distance Attenuation Curves and reverb should be looked on carefully. I think everything regarding into modulation, frequency and eq should be done well in the daws.

We have different loading mods in VR headset:

Compressed: Assets with compressed loading mode are loaded into memory in compressed format when loaded, and then decompressed when played. This requires less memory than decompressed loading mode, but requires slightly more CPU when playing assets.

Decompressed: Assets with decompressed loading mode are completely decompressed into memory when loaded. This uses more memory than streaming loading mode and compressed loading mode, but requires very little CPU when playing assets. This loading mode is most commonly used for short assets that are played frequently and for mobile platforms.

Streaming: Assets with streaming loading mode are continuously loaded piecemeal from disk into a small memory buffer while playing. This substantially reduces the memory required to load and play assets. However, most platforms can only support a single-digit number of simultaneously playing streams, as streaming assets require constant disk access while playing. Streaming loading mode is most commonly used for music and background ambiences. By default, assets ten seconds or longer are automatically set to use streaming loading mode. Other assets default to compressed loading mode. You can change this behavior in the assets tab of the preferences dialog.

“It appears that the sense that time is flowing may be a more complicated phenomenon than is generally thought. There are actually two components to the flow of time (Davies, 1996) that should be distinguished. The commonly used expression, “flow of time,” refers to the experiential phenomenon of events seeming to move from the future into the present and then into the past, a moving present that seems to flow like a river, hence the term (Segal, 2004) . ”

Excerpt From: “The Flow of Time as a Perceptual Illusion”. 

92 GRUBER AND BLOCK Foundalis (2008), Hameroff (2003), Penrose (1989) and Smart (1980). 

Mixing for Virtual Reality

Mixing is the process of blending sounds together. In linear mediums such as music
and movies mixing involves finding the right volume level and panning for each of the
tracks, as well setting up reverb.
For interactive experiences the soundscape is more dynamic; the volume level and
panning is dependent on the direction and distance of the sounds. The tools for
mixing interactive non-VR experiences are typically automatic panning based on
direction, volume controlled by distance based curves; and there are a variety of
dynamic reverb solutions.
In virtual reality the panning is replaced with HRTF which provides more accurate
directional cues than panning. It’s also possible to achieve more accurate distance
cues with careful consideration of how the volume changes over distance and how
reverb is treated.

Immersion

Properly spatialize sound sources.
• Create soundscapes that are neither too dense nor too sparse.
• Avoid user fatigue.
• Use suitable volume levels comfortable for long-term listening.
• Design with appropriate room and environmental effects.

Distance Attenuation Curves

The rule of thumb for physically accurate distance attenuation is: “a doubling of
distance is a halving of intensity”. For example if a sound is set to full volume (0dB)
when it’s 5 meters away, it would be -6dB when it’s 10 meters away, and -12dB
when it’s 20 meters away, and so on. Sometimes this attenuation model does not
produce the desired result, in these cases it’s necessary to bend the laws of physics
a little to achieve the desired experience.

Reverb

Apart from volume, another essential distance cue is reverb. When a sound is very
far away we hear a lot more reverb relative to the direct sound, whereas, when a
sound is very close we hear more of the direct sound and very little reverb.
Controlling the amount of reverb per sound is a critical component to creating the
perception of distance.

Book Review

“One component of immersion that is dominant is that players report losing their sense of time passing and indeed this can be viewed as both a good and bad aspect of the overall experience of playing video games”.(pp.3)

“We describe two experiments where participants played a maze game and the degree of immersion was manipulated by adding music.”

“Thus overall, immersion in games is a cognitive experience constructed within the mind of a player. It does not need to be an extreme experience and whilst about being “in a game” this is not meant in any physical or transported sense but rather about a players thoughts and feelings being more involved in the game than anything else.”(pp.7)

“The findings suggested that increased immersion was altering players’ ability to re produce a given duration whilst doing a task but this could simply be due to increasing complexity of the tasks in the experimental set-up.”(pp.14)

“The main goal in this experiment is to investigate the relationship between levels of immersion and time estimates by manipulating the immersiveness of a game using music.” (pp.17)

“It is immediately apparent that immersion in the experimental condition, that is with music, is actually lower than in the control condition.”(pp.22)

“Previous studies have suggested that the addition of music could increase immersion however, we have also found that how much a person likes a game can dominate their immersion over and above other factors ” (pp.25)

“If the effect is due to immersion, the explanation would be that players were drawn more into the game by the music and hence, despite the game not requiring more attention, players were not allocating attention to the task of monitoring time.” “A more cautious interpretation is that music draws away attentional resources whether relevant to the task in hand or not. The more likeable the music, the more attention it attracts and hence the less a player is able to reliably track the passing of time. In this sense, it is possible that the addition of music sets up one of the conditions for immersion, namely that players become less aware of time and hence promotes the possibility of becoming immersed.” “But as discussed earlier, any alteration of the game play itself alters the information processing requirement of the game and is therefore known to affect time estimation independently to immersion. What is needed is some way to alter the context of the game without altering the game. It might be possible to achieve this by altering the motivation of players through the use of prizes or cash bonuses for high scores though it is not known if this is a mechanism that does increase immersion.” “In summary, music added to a game is able to cause duration to be under-estimated in the prospecive paradigm but not the retrospective paradigm. That is, music reduces the experienced duration of playing a game but not the remembered duration. What is more, the addition of music can make playing more or less immersive depending on whether the music is liked or not. It may even be the case that music is able to set up the situation where immersion is more likely to occur. This is a complex picture and the work described is only just scratching the surface of what might actually be going on with music in video games. What does seem clear though is that this whole area would benefit from the investment of a lot more time.”(pp.35-38)

“In the decades before video technology became available, the presentation of artwork and music became an increasingly important component of the creative process, a spatial interest that called into question conventional concert and art venue decorum.” (pp.2)

“The interest of video artist-composers in spatialising their work has encouraged historians to place the genre within a sculptural, installational lineage.”(pp.8)

“Sonic Spaces The spatialisation of sound can be understood in various ways: the environment in which music is performed; where the players are positioned; and how the audi ence is placed. With the developments in surround sound and stereophony, the spatiality of live music, home sound systems and the cinematic experience has become commonplace, while personal music devices and locative media enable a continual acoustical re-mooding of a listener’s environment.”(pp.14)

“As music moved into the space of the listener, similar spatial shifts began to occur in art exhibition and it is possible to chart for the art gallery an almost simultaneous transition from physically neutral space to interactive environment. ”（pp.48）

“The idea of spatial reconfiguration and impermanence distinguishes installation from other types of art that are transported, already completed, into the exhibition space. Rather, installation works often occupy, and therefore activate, the entire neutral void of the gallery space. Such expansion requires a rethink of traditional gallery organisation, in which artworks, sometimes by many different authors, are separated from one another by frames or, at least, by wall space. ” (pp.82)

“Video’s ability to resculpt the gallery space in real time links it to the spatial expansions of both music and art and the multimedia spaces of expanded cinema. If we decentre the discussion of video from object to the environment in which video pieces reside, it is possible to trace for it an assortment of ancestries that enhance its intermedial qualities. On closer inspection, then, it is possible to revoice the assertion in my introduction to suggest that video art-music represents the culmination of both intermedial and spatial experimentation. ” (pp.105)

Review:

Time Perception, Immersion and Music in Video Games, the relationship between music, immersion and time perception in video games is explored:

1. Time perception and immersion. Immersion in video games frequently causes the player to lose track of time. This cognitive sensation of immersion, which the player creates in their head, may be both a positive and a detrimental part of the gaming experience. 

2. Music is a component of immersion. Previous studies have demonstrated that the addition of music can enhance immersion, but enjoyment of the game can also influence immersion.

3. How music affects how we see time. The player’s capacity to faithfully recreate a certain period while doing a task can change due to the heightened immersion caused by music. Music may attract attention, making players less aware of time and promoting immersion. The level of enjoyment of music can affect the player’s ability to track the passage of time.

‘Towards Space’, the focus is on the relationship between the musical and spatial aspects of the creative process:

1. The presentation of art and music in the creative process grew in importance before the development of video technology. The conventional protocol for concerts and art venues was put to the test by this spatial interest.

2. The spatialization of video art and music: Video artist-composers expressed interest in the spatialization of their work, and as a result, the spatialization of sound may be understood in terms of the setting, the player’s position, and the audience’s location.

3. The transformation from physically neutral places to interactive environments happens similarly in art exhibitions as music penetrates the listener’s area. This change in music and art at the same time reflects the changing spatial dynamics.

4. Spatial reconfiguration and installation art: The necessity to reconsider conventional methods of organising galleries, in which artworks are frequently separated by frames or wall spaces, is what distinguishes installation art from other art genres.

5. Video art, music, and spatial expansion: Video has the power to instantly transform gallery settings, connecting it to the way that both music and art may take up more space. It is feasible to discern the video works’ intermediate and spatially experimental nature by focusing on the context in which they are positioned.