Live entertainment, anyone remember that? Theatres and concert venues are either laying closed or repurposed into the much-needed vaccination centres as we hurtle in the middle of 2021. I do remember heading to the O2 arena on 1st February 2020 to see the Kaiser Chiefs and Razorlight. We nearly didn’t go as there were certainly worrying looking things around the ‘Rona’, the show was great, but the anthem “I predict a riot” was soon to play out in supermarket aisles in the surreal rush that many engaged in over toilet paper. Watching our other show tickets get rescheduled for months later and now years seems like a never-ending process but in the words of one of the bands we had lined up to see “The show must go on” and that is something I recently experienced thanks to the RSC delivering their online performance with Dream.

Virtual scenes, actors and spectators

Dream was a limited run set of live 50-minute performances delivered online by the RSC as part of a partnership with Manchester International Festival, Marshmallow Laser Feast, Philharmonia Orchestra and Portsmouth university. At its heart this show, inspired by Shakespeare’s A Midsummer Night’s Dream, was set in a virtual forest, rendered using the Unreal game engine. The twist over some of the other virtual events that have sprung up, was that the main character of Puck, a human shaped collection of rocks, was representing live full body motion capture of the performer. The virtual camera was also manually operated in the capture space. Usually, virtual cameras are either under the control of the viewer or following virtual dolly tracks. Motion capture has traditionally been done in films and games to be used in post-production to create characters, this was real time.

Another twist, you may have seen footage of actors and green screens where they are talking to something they cannot see, or a ball on a stick, that represents where the CGI will be slotted in later. In Dream the primary performers wore VR headsets to be able to see the environment and one another. As an audience we were not in VR, but using a web browser, to see what the producers wanted us to see. Just to recap, it was a streaming video on the web of people in VR performing live full body movements. The event came in two flavours for the audience, a free to watch one or a £10 ticketed version. The latter offered a degree of viewer interaction, where at certain points in the story we would drop virtual fire flies onto a map and these would appear in the scene to light Pucks way. We were also treated to a behind the scenes live Q&A after the performance too.

Camera tricks and set changes

Live theatre often has set changes, new backdrops and props wheeled in an out in a variety of clever ways. Having a virtual set means instant changes can happen, but it also means that subtle alterations can be made to further the storytelling. Dream sees an ever-increasing storm start to rage, initially the forest is calm and relaxing. A mixture of visual and audio alterations tells this story, but it is also fuelled by the live acting, a change from flowing movements to a more nervous or cowering demeanour and scared voices. The motion capture is happening in a physical space, so unlike the virtual one being shown it has limitations, i.e. walls. The characters can’t just run on for 100m as they might in a on location film shoot, they have to use stage craft to loop around the space. This is where the live virtual camera helps, as it is not looking at a stage, it is in it, so a combination of the camera person moving around virtual trees and rocks, and the actors can make the space seem larger and more varied than it otherwise would. A big flat empty floor to run around with a VR headset is now a relatively simple experience, and how many of us use VR for games and meetings.

This performance saw the performers climb up rocks and trees, and some were hovering in mid-air too. The production achieved this with some things as simple as other people lifting the performer up, bearing in mind the performer is in VR and blind to the physical world, that plays both ways as anyone walking around in the physical space and not in a mocap suit is invisible to the system. It looks like they also had some actual blocks and boxes, mapped to the position of virtual objects placed in the space, again invisible to us, and the actor can only see the virtual rock, but will know there is a thing to climb. Virtual reality arcade experiences use this to great effect, a set is built with door arches, walls, seats etc that are perfectly mapped to the virtual world, which means in VR you can see the doorway on a space station, and even physically rest against it and feel it as you poke your head around looking for the bad guys, this real-world haptic experience is an incredible addition to any experience. For the Dream performance there is also another option that they may have been using because the world is virtual the placing of the “floor” is too. Again a mixture of camera and character movement can soon have a character up in the bows of a tree, the performer still has their feet firmly on the ground but the scene has effectively moved vertically in virtual space. Clever stuff I am sure you agree, though these blends of physical and virtual get quite freaky at times.

All the feels

This is all very clever and some innovative twists technically but was the show any good. The short answer is yes it was. For the entire show I was transfixed and intrigued. There was a charm and a warmth and a nervousness for us all to be seeing a live performance, it could have just as easily been a recording, but it wasn’t, it felt live because it was. The interactivity elements were a little flat in how you chose where to drop fireflies, but I felt I was with my fellow viewers, we were virtually holding up our lighters or phone flashlights in the crowd. When I originally saw this as described as a VR performance, I was hoping it would be in VR, that I would be there, but it made sense for the production to choose how I saw things and also for it to be accessible to as many people as possible just on a web browser. The meet the performers at the end was also another nice touch to see, hear and hence feel their total engagement with delivering the show. The way entertainment is delivered is, like everything else, rapidly advancing and the potential for new ways to create the right feelings and emotions in a audience blending the old and the new is very exciting.