What “VR” Gets Wrong About Reality, Virtual and Otherwise

This arcade doesn’t smell like an arcade. Somehow, I’m certain of that. The sounds I immediately recognize, though; walking into that clamor of chance, sport, cinema, and carnival; chiptunes and spring-backed buttons like treasure itself. I roll the joystick in my palm, knock it back and forth, grip it to move one way, let go to stop. I’m a little drunk on two thoughts, a false one, that I’ve been here before, and an obvious one, that the quarters fit perfectly into the glowing coin slot. The walls are lined with old console games, the kind only collectors would care about. I wonder how long this place will stay in business. I’m on a date and we’re having fun. It’s 2017.

I got a PS2 on Christmas Day, 2001. At the time, it felt like the most valuable thing in our house, and in a way, it was; anyone other than my aunt and uncle buying it for my brother and me would have been irresponsible. When I ask my parents about it, they smile, remembering our hysterical reactions. I remember JAK AND DAXTER and looking at my hands on the DualShock controller, thinking how daunting a task it would be to learn all these buttons. A triangle, a circle, a square, an X? What did that have to do with anything? And these two analog sticks-did they really expect me to walk and look at the same time? When things got too difficult that day, I handed the controller over to my dad. A system formed around this: when it was too difficult for my brother, he’d hand it to me.

It’s very important to me now, that controller, and the space it held between us. Not just for how we took care of each other through it (although, spoiler, it sort of is just that), but for what it suggests about the ways we relate to games. As we slouch towards more and more “immersive” media experiences, we’re going to need to reckon with this fact: that between the game and the player is always a third thing. For now, it’s the twin-stick controller. Once it was the Arcade. If Silicon Valley has their way, it’ll be a VR headset and motion control, and perhaps a gaming headset under 100 thrown in for good measure. At the rate the technology is advancing, it seems well within the realm of possibility to imagine that one day soon, that third thing might as well be nothing at all. Well, I’m skeptical; morally yes, but even practically-because when it gets down to it, that third thing is, basically, the whole thing.

IKIRU, Akira Kurosawa. 1952.

Image Source

Gaming went from an activity to a personal identifier right around the time that I realized I was good at them. Unsurprisingly, this was also the time where defining yourself seemed of an utmost importance. It was 2007, MODERN WARFARE had just come out, it was my first foray into playing online, and I could hold my own. I was better than my dad and (mostly) my brother. We all played, enough that the DualShock 3 often broke: left analog sticks wiggled free like a loose tooth; the circle button lost its bounce; triggers got “slippery.” For co-op, we’d barter over who’d get the more beat up controller. The slightest loss in tuning could throw off your entire play. I’d go across the street to play HALO 3 and fumble with the translation. What’s with these letters? Why is the analog stick not symmetrical? The DualShock, I understood fully, knew it in intuitive ways. I was fluent in it.

When it comes to controllers, there’s really only one company you can count on taking risks: Nintendo. However, Nintendo of America president Reggie Fils-Aime might walk that back, considering in 2012 he said about the relative failure of the WiiU console, “this industry is all about content.” For those of us that remember when the Wiimote briefly took over the world, that’s a curious sentiment. But in a retrospective for that Wii-dominated generation, Stephen Totillo sees motion control culpable for both its boom and bust: “The simplicity of motion controls may have engaged new audiences, but, compared to traditional button-and-stick controllers, they disengaged their players…What had been triggered by the digital input of a button had become the hand-waving of a gesture. The former was clear. The latter was confusing.” So is the controller or the game library guilty for the Wii’s decline? Well, both. Form dictates content. Imprecise control and low graphical ability limited the kinds of games you could make on the Wii, and while grandpa liked his WiiSports, you and all your friends loved MODERN WARFARE.

PLAY IT BY TRUST, Yoko Ono. 1966.

Image Source

The same is happening for VR, although without even that first accessibility-sparked boom. Even the most extreme VR evangelist has to concede that the technology is not living up to the hype. Naturally, venture capitalists have a chart to assure us why this is all part of the plan. While we may very well be just a few years off from VR making a smartphone-sized impact on the home and marketplace, that impact feels, at the moment, actually the size of a smartphone. There’s a lot of good reasons for this, namely steep prices and a lack of must-play games (like the Wii). But there’s also a deeper, ideological flaw to whole enterprise. It starts with “immersion” and ends with “mind-body dualism.”

Naughty Dog game designer Richard Lemarchand, in his GDC talk “Attention, Not Immersion,” questions the uses of the word for talking about video games. On one level, it just doesn’t make sense. To say someone is immersed in a video game suggests that they have somehow imagined themselves to be inside it-but what exactly is it? If we mean the video game environment, then what the player is actually in is the character avatar, and of that, Lemarchand is very skeptical: “The claim that you can make someone forget who they are, and start to believe that they’re someone else, just by enveloping them in pictures and sounds of another place and time, doesn’t seem quite right to me. It doesn’t do justice to the game artisans who create experiences that get and hold our attention in many different, ingenious ways. And even if we could pull off this trick, of making you believe that you are someone else-would we really want to?” He makes a distinction between TV-based games and VR, conceding that the latter is a better fit for the definition of immersive. But this doesn’t invalidate his larger point that immersion is out-and-out an ineffective term for designing good interactive media. When it comes to VR, the concept of “immersion” is what we’re packing into the pipe before ripping ourselves off into The Future.

Memory is a kind of action, a kind of time travel

Image Source

Above is a diagram of philosopher Henri Bergson’s concept of memory. The P plane is, essentially, the world. AB is our unconscious memories. S, or the summit, is our perception of our body. The theory is just one part of the larger work MATTER AND MEMORY, which means to reconcile the old metaphysical issue of mind-body dualism. Instead, Bergson proposes, the difference is only a matter of time. He writes that:

“We tend to scatter ourselves over AB in the measure that we detach ourselves from our sensory and motor state to live in the life of dreams; we tend to concentrate ourselves in S in the measure that we attach ourselves more firmly to the present reality, responding by motor reactions to sensory stimulation. In point of fact, the normal self never stays in either of these extreme positions; it moves between them, adopts in turn the positions corresponding to the intermediate sections, or, in other words, gives to its representations just enough image and just enough idea for them to be able to lend useful aid to the present action.” (162)

Memory, then, is mobile, oscillating from the fields of pure unconscious to the immediacy of what we call our bodies, where it informs our actions. Apart from validating Lemarchand’s skepticism of players ever becoming fully absorbed into digital identities, I bring up Bergson to show the irreducibility of that third thing which, as you might have figured by now, is a bit larger than just the apparatuses that mediate our experience of games. So if it’s not just a game controller, what is the third thing? It’s the ground beneath our feet. It’s the awareness of our face which is still our face. It’s something we hold. The virtual, in mind or pixel, is always rooted in the tactile-the sensory, the summit of point S. We can, as Bergson suggests, imagine that we concentrate ourselves in one. But really we’re just moving between them; we are never reduced into one or the other.

When people ask me my favorite gaming moment, I’ll probably say the suicide mission in MASS EFFECT 2. As an explanation, I might talk about the character writing, the effectiveness of loyalty side missions, the worldbuilding and lore that inform the climactic mission. But really, it’s just because I was jumping up and down. I remember being in that moment, sweating at the prospect of losing members of my team. I can remember remembering that this jumping up and down was the kind of spazzy thing I’d do at age nine playing NIGHTFIRE with my brother and how silly I must look, 18 years old, doing it all by myself in the middle of the night. This must be fun. This is how we experience games (and maybe everything)-a shivering emotional response that is itself the molding of a new link onto the chain of memories you’ve been assembling your entire life. I can see the links in the chain clearly: Me at 18, me at 10, and my brother at seven. It’s why I question the merits of putting a screen over my eyes: we have to consider what gets obscured. Like Lemarchand says, “even if we could pull off this trick, of making you believe that you are someone else-would we really want to?” Sometimes, maybe. But afterwards I’ll still be me. And it’s me and all my memories that guide my eye and hand, that make me feel.

In his attention-centric theory of game design, Lemarchand recognizes a need for a light touch. Like any artwork, a game must foster some sense of openness to encourage reflection; an overly-tight hold on our attention closes this off, leading to fatigue, or mindless addiction. He asks, “Have you ever stopped to watch the sunset in MINECRAFT, or to admire a view in UNCHARTED?” Those moments are vital, not only to our wellbeing, but our ability to enjoy a game. It’s an effective design choice, and an ethical one. To that end, as Lemarchand meant for the theory to be applicable for all media, I think we should reconsider the attention and subsequent resources we’re giving to VR. We could all stand, perhaps, to open up its grip on our idea of the future. Imagine if we devoted a fraction of it to (pretty much anything actually important, but let’s stay on topic) accessibility efforts like the AbleGamers Foundation, which provides customized controllers to kids with disabilities. If the DualShock is an aesthetically remarkable object where form sublimely meets function, here’s an organization doing that on a person-by-person basis.

If you can move your tongue or blink your eye, you can play with help of the Versatility Gaming Controller Configurator

Image Source

I’m not completely writing off VR. But it’s a fact that the way we talk and think about it, apart from being conceptually false, leads to poorly designed games, emotionally inert experiences, and an overall narrow vision for the future of media. That said, if game designers use the correct simulated data and listen to what their players want, there is way to do it right: in KEEP TALKING AND NOBODY EXPLODES, a player enters a VR world with a ticking time bomb, while her friends have the instructions to defuse it. This virtual reality is not about an environment, but the walls and ceilings of one-it’s the difference between believing a game is a room and that we want to be more inside it (immersed, that dirty word), versus believing a game is four walls and we want to understand ourselves in relationship to them (in this case, how to communicate in spite of walls). On the single-player side, NYU student project WHEN WE DIE is using the walls of virtual reality to craft meditative spaces. Here, players are guided to contemplate mortality in an abstract yet natural space inspired by the infra-red photography of Richard Mosse. The isolation, discomfort, and reduced sensory experience of the VR headset is all part of the design. Both these games are aware that between the player and the game is a third thing. For the future of the medium, true innovation will be a matter of deepening that third thing, making it more accessible, more interesting, more human-not wiping it out of existence. Like Bergson says, that’s not how existence works anyway.

This is what innovation looks like. A couple months after Christmas, the PS2 gets moved into our room. My brother and our might-as-well-be-cousins are playing co-op deathmatch on NIGHTFIRE deep into the night, as our parents drink and laugh outside. Someone gets accused of screen-peeking. Tension is high, people are frustrated. The parents help us make a device- poster-board and string, it slides into place, bisecting the TV like a drawbridge visor. You can actually snipe, if you want. We enter our virtual worlds more separated. But really, not at all: one person is lying on the floor, another is sitting on the couch; we’re all in the same room, holding the same kind of controller, looking at the same TV. The only thing that’s truly our own is the ways we will remember these moments. I wonder what we would recall, if the screen was around our eyes and our bodies in different places. Which room are we immersed in?

Jimmy Evans

Jimmy Evans is a guest contributor here at Crossfader. He writes about rules, spaces, and feelings because sometimes they confuse him. In seventh grade he cut his hair to fit in and he’s been selling out ever since.