VR - The evolution of interaction

Written by

Virtual reality (VR) is impressive. Anybody who has spent any time using it will tell you how amazing it felt to fire an arrow from a virtual bow or how incredible it was to watch a blue whale swim past, almost close enough to touch. Amazing! Incredible! Genuinely, it really is. But what happens once you’ve exhausted these more disposable encounters and just want to watch a video or listen to music? What could VR mean for the common experiences we engage in regularly?

For your daily stroll along a plank, you have Richie’s Plank Experience. For your daily stroll along a plank, you have Richie’s Plank Experience.

With the introduction of every new technology comes the inevitable period of porting, recreating old media for a new medium. This usually involves taking the original content and adding another layer of output or input. Radio newsreaders would read directly from the newspapers, TV shows would record with the same hosts, in the same theatres as the radio shows that preceded them, and many a touch-screen volume control began life as a recreation of analogue dials on audio equipment long gathering dust in people’s attics.

Over time, as ideas begin to evolve, the content starts to fit the shape of the new container. The challenge is, VR is rather an amorphous container. By exploring past experiences and the changes they have endured, might we be able to avoid the pitfalls associated with designing for a new technology? I have been tasked with furthering the idea and have future plans of using this as a foundation for designing more intuitive VR products.

Virtual Literature

Writing was invented during the early bronze age and has itself been dramatically reinvented by technological advancements. Automated printing brought written media to the masses and in turn the act of recreational reading. What new opportunities does VR present for writing?

Until more recently, with the introduction of e-readers like the Kindle, the process of reading a book remained relatively unchanged. You may have enjoyed a children’s book with interactive sound-effects; cow goes moo, chicken goes cluck, fish goes… Or have read Alan Partridge’s memoir while listening to the accompanying Spotify playlist, but rarely are these more than detracting interjections. A worthwhile addition should complement the primary activity and not contend with it.

Audiobooks, along with the text being read/acted aloud, will sometimes include subtle background effects or music to create an atmosphere. One of the many things VR is good for, is creating an atmosphere. The shape of literature within VR, whether text is present or not, could stretch into and way beyond words, encompassing the reader in the story, with a mood or feeling and not necessarily a defined scene. Or alternatively, it might well just be an oversized book!

Virtual Music

The recording of music and how it is distributed has dictated how we consume it. The basic idea of an album structure still exists from the days of vinyl LPs, and while the production of music more often follows this structure, the decrease in cost of digital storage and streaming has literally shuffled things up. Can VR accommodate this new way of consuming music or does it struggle with the unpredictability of personal music curation?

Visual pairing with music is hardly anything new. It was one of the first things to appear with the introduction of “talky” cinema. Walt Disney produced a whole series of Silly Symphonies as accompaniments to musical pieces, and later the feature length Fantasia. Skip forward to 1956 and you have Tony Bennett claiming to have created the first “music video”. I think it is fair to say that, nowadays, music and music videos rarely harmonise. To really hear a song you don’t need to watch the video. Real-time music visualisation became a reality in the 70s and by the 90s it received widespread attention due to the proliferation of home computers. It may be not be the most aesthetically pleasing, but it was at least synchronised.

Enjoy listening to music while travelling down a nauseating, never-ending tunnel of colour. Enjoy listening to music while travelling down a nauseating, never-ending tunnel of colour.

Recorded music videos definitely have a place in VR. However, it is real-time visualisation that holds the most interesting possibilities for music. As mentioned, atmosphere can be most effective when projected in a subtle, unobtrusive way. With that in mind the inclusion of real-time sound manipulation could add to this experience or equally risk detracting from it.

Virtual Imagery

Video has long utilized electronic displays and has seen the technology advance from an 8x8 pixel display to the 3840x2160 pixel 4K displays (soon to be followed by 8k) we now have in our homes. It wasn’t until the ubiquity of higher resolution screens that digital photography fully joined the party.

If television screens are any indication of how we like to view content, bigger is better. However, a decreasing amount of content is actually being viewed on television screens. Different types of content are often suited to, or sometimes limited to, different screen types. VR is capable of mimicking all these screen types, and more importantly capable of removing the “screen” entirely. Just because you now have space to view content on an IMAX sized screen from your living room sofa doesn’t necessarily mean you should, or at least not everything.

360º video in VR is a great example of porting. Instead of the mundanely dragging the viewport to change your orientation, you can now do so by awkwardly rotating your whole body on the spot. As users can’t control their movement on any other axis fixed rotation can be quite disorientating, and more comfortably enjoyed from the safety of a swivel chair. This control of the narrative makes the story less focused with 360º of imagery vying for your attention. This is not to say user-directed experiences are not valid and may well fit better within VR than on a flat plane, nonetheless there is still space to evolve.

It is very difficult to introduce extra stimuli to an experience without diluting the primary narrative, particularly when the content is ever changing, like video. Ultra hi-res photography and other static imagery could benefit far more from the use of a full-peripheral or 360º field of view. Larger, more sprawling stories could be told by allowing the viewer navigate through or zoom into an image and choose a focus for themselves, then given the opportunity to follow a new path in their own time. Flat images start to feel less so when the edges aren’t visible or smoothly transitioned between. Google Earth is a prime example of how static images can be stitched together to create something far greater than the sum of its parts. And then there’s that little thing called three-dimensional depth, though stereoscopic 3D is hardly anything new and predates James Cameron’s Avatar by almost 170 years.

Edit. Now, I am going to have to eat my words slightly. While I still agree that most 360º offerings have felt limp, I was very impressed by the recently released Gorillaz video for Saturnz Barz (Spirit House). Produced by Passion Pictures Animation in collaboration with Google Spotlight Stories, the video uses scale and a flowing narrative that encourages you to follow it instead of choosing your own direction. It doesn’t bombard or overwhelm and, very confidently, uses the space to strengthen a single path rather than branch it off. This really demonstrates the need for clear direction, not just of the content, but of the audience. Leading with visual flow, non-disorientating editing and 3D audio can all help tackle this, we may require a fair bit of hand-holding while trying to adjust.

Virtual Learning

The emphasis of VR use has been placed on recreational activities, but those in the health and teaching sectors have also shown interest. Being able to visualise ideas or principles is key to developing and understanding them. Personal computers have become an important teaching aid within schools, how valuable could VR prove to be as an education tool?

It may be safer to practice or experiment in VR first It may be safer to practice or experiment in VR first

Learning through interaction is an effective way to initially grasp a subject, and for practical reasons this is often using crude representation rather than the genuine article. In VR there are far less restrictions and far more opportunities to tailor learning experiences around the pupil, the teacher and the subject in question. Interactive sandboxed spaces allow for trial and error without the worry of real consequence, which could prove to be detrimental in some cases, like learning to drive a car.

Forza Horizon 2 from Microsoft Studios Forza Horizon 2 from Microsoft Studios

A Virtual Evolution

Many of the possible applications mentioned, more so the recreational ones, focus on setting a tone or mood. Exploring atmosphere alone is a very safe use of VR often keeping familiar content or setting as its base and layering additional stimuli on top. Through our exploration of VR so far, we are beginning to see that what eventually becomes of these experiences may prove to be so far from the original content that it appears completely beyond recognition, and far outside the realm of possibility in our natural reality. This post is a brief sample of my findings and thoughts, which our VR&D team here will continue to build upon, stretch to their limit and eventually conceptualise around.

Progress in VR is inevitable given enough time, but it doesn’t need to progress at a snail’s pace. It is important that we use our past knowledge of content and its relationship with a particular technology to move forward, but it is also important that we learn from our mistakes and not simply emulate what came before. The consistent thing across all VR hardware is the effect of a spherical and encompassing field of view. This is the virtual space provided, make use of it.

Professor Xavier Professor Xavier

Every few hundred millennia, evolution leaps forward. – Professor Xavier

Ready to talk to us? Interested in visiting?
Contact or
call us on +44 (0)203 215 0041