The desire to sink into a world that is not quite our own is hardly a new one. Nor is the idea that technology might help us accomplish that goal. Nearly a hundred years ago in 1935, the sci-fi story Pygmalion’s Spectacles envisioned exactly such a tale. In the short story, author Stanley Weinbaum described a pair of ‘goggles’ that allowed the wearer to view and physically experience a fictional story. Given that the story was written several decades before the first video games, its similarity to what we might consider very modern Virtual Reality products is incredible.
Achieving ‘true’ immersion in reality, however, remains an elusive prospect. Recent advances in technology have brought us closer than ever to that goal, but how close are we really? To answer that question, we need to examine the technology currently available.
Virtual Reality Headsets
While we might think of VR as a modern technology, it’s actually been around since the 1950s. Early machines like the Sensorama, which combined various technologies to stimulate the senses of up to four ‘riders’, were more akin to theme park 3D motion simulators than the PSVR 2, but it wasn’t long before personalised setups were developed. The 1968 Sword of Damocles, for instance, is generally regarded as the first head-mounted VR device. It was an unwieldy bit of kit. Its headset was so heavy it had to be attached to the ceiling and users couldn’t move while wearing it. However, it did allow users to view different perspectives when they turned, setting up the first hints of the technology we see today.
The PS VR2 is arguably the most advanced VR system available to a general audience.
The following decade saw the introduction of flight simulators, which remain a major pillar of modern, if specialised, immersion. The ’70s also had the launch of VIDEOPLACE, the first truly interactive platform capable of responding to a ‘player’s’ movements.
The next fifty years saw a tremendous leap in technological development. From rough, wireframe models and approximate movement controls using cameras, we’ve now arrived at Virtual Reality headsets capable of rendering interactive experiences in extremely high definition. These headsets can adjust a game’s visuals in accordance with how a player moves and turns their heads, provide surround sound audio, and track their arm movements using handheld controllers. The result is that a player can really feel like they’re within an entirely different world. Even better, it allows them to influence that world with their own physical movements.
The Drawbacks
Modern VR headset are incredibly impressive pieces of kit, but they’re not without their faults. Far and away their greatest issue is their cost – or, more specifically, their cost relative to their value. The new PSVR2, for instance, is hitting shelves with a price tag of $549.99, making it roughly equivalent to a modern console.
The difference, however, is in what you get for your money. PC and console gamers benefit from a vast range of games developed by innumerable different teams, offering a huge amount of variety. Contrastingly, the games available on VR headsets are usually relegated to traditionally designed games that have been ported to VR as more of an afterthought than as the intended audience. For example, it was big news for VR gamers several years ago when developer Bethesda announced that they were going to port two of their biggest games – Skyrim and Fallout 4 – over to virtual reality. While these games offer extensive worlds for players to explore in VR, neither of them were designed for the platform. As a result, both suffered major technical issues when they were first released. Even now, after five years of patches, Fallout 4 VR still has Mixed reviews on Steam because of its technical issues.
That’s not to say there aren’t any games designed with the advantages of VR in mind. 2020’s Half-Life: Alyx was almost universally praised in every area of development, in particular how it used the medium of VR to enhance the player experience. As a title designed specifically for that medium, it was able to take advantage of player interactivity in a way ported games could not.
Fallout 4 VR was widely criticised on release due to its bugs.
It’s something of a vicious cycle. Players don’t want to spend huge amounts of money on an under-supported platform, so developers are disinclined to produce games specifically for a medium only accessible to a few. Time and reducing equipment costs can soothe these issues, but it has been slow going.
There is also an argument to be made about how immersive these headsets truly are. While they can allow you to see and interact with a game, there are substantial limitations to both your movement in space – i.e. ability to walk around – and the physical feedback you receive. The headset itself can’t solve such issues, which is where peripheral technology comes in.
VR Peripherals
Broadly, VR headsets suffer from two major drawbacks when it comes to the level of immersion they provide: they don’t offer the player any tactile (touch) feedback and they substantially limit a player’s movement in physical space. The latter of these issues is particularly troublesome, as it is one of the primary reasons why some VR users suffer from motion sickness during play sessions. Their eyes convince their brain that they’re moving, while in reality they’re standing still.
To solve these problems, developers have turned away from headsets to produce ‘peripherals’. These devices exist separately from the headset, but work together with it to provide a more rounded experience. To start, let’s examine how games can provide physical feedback from in-game contact.
Haptic Suits
For tactile feedback, you have the option of what’s known as a haptic suit or vest. These are wearable items, typically designed to be wrapped around the torso, that use various mechanisms to simulate contact in accordance with a game.
As with VR, haptic suits are older than you might expect. The first prototypes started appearing in the ‘90s, using the vibrations caused by sound to mimic punches and kicks in a fighting game. More recent offerings such as the 3rd Space Vest used compressed air to achieve a more substantial effect, delivering 30 pounds per square inch of pressure and, as the developers explained, was designed to make players feel like they had been ‘shot for real‘ while playing FPS games.
The Teslasuit also offers a haptic glove to simulate touch.
At present, one of the most advanced forms of a haptic suit is the Teslasuit. The machine uses motion capture, biometric sensors, EMS, and TENS to stimulate sensations. The result is a tremendous level of tactile feedback, but at the cost of an unobtainable price tag for most. Even the developers have suggested that this technology may never be affordable at a consumer level as it’s too complex to simplify for the mass market. Instead, the Teslasuit has been proposed for a range of medical uses in patients with sensory or movement disabilities. That being said, that the technology exists at all is the first step towards its becoming more broadly accessible, even if a gaming equivalent could still be decades away.
The Movement Problem
One of the other core issues with VR headsets is that they tether you to a relatively small real-world space. This does actually have some advantages. Most people don’t want to inadvertently walk into walls while playing and it’d be impossible to find someone with the level of physical fitness required to keep up with gaming protagonists who sprint for hours at a time. By eliminating the need for extensive physical movement, such issues are entirely avoided. At the same time, the lack creates a substantial disconnect between what a player is seeing and what they’re physically experiencing.
How, then, to make players feel like they’re physically moving with their character while remaining within a small play space? One solution comes from Virtuix, a company founded in 2013 with the intention of facilitating what they call VR locomotion. Over the last few years, they’ve developed ‘omni-directional treadmills‘. These are precisely what they sound like: tracks that allow you to run in any direction without actually moving.
These tracks may look like something from science fiction, but recent developments have made them substantially more obtainable than you might expect. Indeed, the ‘Omni Pro’ system has been available for use in commercial locations since 2016. Now, the company is in the process of developing and launching a home system alternative, the Omni One.
Virtuix, and other companies like them, offer a surprisingly neat solution for solving the locomotion disconnect of VR gaming. You are locked into a single physical location, but you’re still completely free to move in any direction. It does, however, leave the issue of fitness. Many modern videogame protagonists possess an inhuman level of physical fitness in order to smooth out gameplay friction. It would get pretty old if you had to keep pausing a game so your character could catch their breath. Instead, games almost always ignore the issue. It’s simply accepted that protagonists can run, jump, and climb without needing to worry about the practicalities of human endurance.
As a result, technology like the Omni One will likely only be truly useful in smaller-scale, more realistic titles. No matter how good the game, it’s not going to be easy to convince players to hike their way across huge maps like that of Red Dead Redemption 2. That’s not necessarily a bad thing, but it certainly limits its applications in the tremendously diverse field of gaming.
Brain-Computer Interfaces
The hard truth is that many games stray so far from reality, there’s no feasible solution to physically interact with them in a direct parallel. Even with all modern VR technology, the limits of our own physicality will always act as a barrier. That’s where Brain-Computer Interfaces, or BCIs, come in.
Of all the technologies discussed in this article, playing a video game with your mind is certainly the most far-fetched. And yet, despite sounding like it’s just sprung fully formed from an episode of Star Trek, the beginnings of this technology are already available and functioning in the real world. In fact, the first attempts at such a system date back to 1973. These early prototypes, which persist to the present day, relied on using a non-invasive electroencephalogram (EEG) to detect brain activity and translate it into computer controls.
In the decades since, such controls have been refined to a tremendous level. In 2012, researchers at Graz University of Technology demonstrated using a BCI to play World of Warcraft, albeit somewhat slowly. More recently, Twitch user Perrikaryal has been streaming a BCI playthrough of the notoriously difficult Elden Ring. At present, she’s able to attack and use items with the BCI, but still relies on traditional controls for movement. In the future, however, she hopes to make a full transition to hands-free.
As a BCI can rely entirely on brain interactions for control, it conveniently bypasses any fitness issues with locomotion. There’s no need for muscular stimulation and as a result, the physical abilities of the player become immaterial. This is actually a priority of the technology; as with almost all the technologies now being adapted to VR gaming, BCIs are primarily being developed with the intention of assisting people with disabilities. Enabling them to work with nothing more than neural stimulation is their primary purpose.
Being able to control games with nothing but your mind is incredible, but control is only one aspect of immersion. For a game to truly bring you into its world, it needs to be able to influence you too. Just as haptic suits facilitate tactile feedback, a true BCI experience needs to convince you that you can see and feel the world of the game. That, as it turns out, is a little more complicated.
Neural Feedback
Although BCIs being used to control games is an entirely realistic proposition, neural feedback is a rather different proposition. That is, the technology stimulating activity in your brain rather than the other way around. Theoretically doing so would allow a game to generate sensations of sight and touch that would allow the player to believe they really were in that world.
Again, as fictional as it might sound, this technology isn’t outside the realms of possibility. For several decades now, developments in human robotics and neuroprosthetics have shown promise in sending electronic signals to the brain to simulate physical sensations. These have included allowing blind people to ‘see’ and amputees to feel touch using a brain-controlled prosthetic. While far from flawless, the technology certainly exists.
Researchers at the University of Utah pioneered an arm prosthetic capable of ‘feeling’ touch.
That said, the level of connectivity these devices require is reliant on physical implants. It’s one thing to ask a player to wear a headset that tracks their brain activity, it’s quite another to ask them to undergo surgery. Unless such technology could be adapted for non-invasive use, its applications in the gaming community are almost entirely nil.
In the meantime, while it’s clear that this technology is still a long way from the kind of immersion promised by Pygmalion’s Spectacles, it is still a substantial step in that direction. Unfortunately, the necessity of surgical implants might not be the greatest hurdle BCIs face. Even non-invasive alternatives could still pose harm to their users.
The Risks
For many, the idea of letting a computer get inside their head and changing their awareness of reality is the start of a horror film. And while the robot apocalypse is unlikely to pose a threat, it’s not wrong to be wary of this technology. The human brain is an incredibly complex organ. Using computers to actively change our perception could have any number of unintended or unpredictable ramifications.
Even simply using BCIs for controlling a game has been shown to impact brain activity. One study carried out in 2020 investigated how using a BCI to control a gem-swapping game over the course of four weeks impacted the subjects’ attention, working memory, and memory retrieval. While the changes observed were seen as positive, they demonstrated that this technology certainly has the potential to cause harm. In particular, the authors of the study noted that participants reported increased levels of mental fatigue as a result of playing.
Despite being non-invasive, even EEG-based BCIs could pose a health risk.
The study is far from a definitive ruling on the subject. It only involved fifteen participants and didn’t investigate any long-term effects, but it may indicate future concerns. More than anything, it demonstrates that any form of BCI that implements neural feedback will require extensive testing before it reaches the market. Video game equipment is always subject to some safety testing, largely centering around the electronics contained within, but a device with the power to influence a player’s physical and mental health would require an entirely different approach. Whether game developers are interested in crossing that hurdle or not is a question still up for debate.
The Future
Trying to predict what the technology of the future will look like has always been something of a losing game. Identifying the next big leap forwards is almost impossible for anyone not directly working in that particular field. That said, it’s clear that immersion technology is currently in the middle of a boom. Though this is largely driven by technologies being designed to aid patients with physical disabilities, there is a substantial overlap with equipment that can facilitate immersive gaming.
At present, that has brought us VR technology that grows increasingly convincing all the time, while becoming more and more accessible to a wider audience. VR might have its limitations, but it remains the standout option for an immersive gaming experience.
Whether the future really will see BCIs enabling players to experience games within the confines of their own minds can’t be certain. That researchers are investigating such possibilities is encouraging, but not conclusive. There are still a lot of significant hurdles to their development that need to be overcome before we could ever conceive of something like that existing in the broader gaming community. For the moment, the possibility has to remain in the realm of science fiction.
Considering that video games didn’t exist as a broadly accepted medium only a few decades ago, the progress developers have made in this area is almost unimaginable. It’s also an indication that technology will likely progress much faster than we might expect. With modern developments in VR gaming, we’re hardly doing badly for ourselves as it is. While truly immersive gaming is still frustratingly out of reach, it might end up being a lot closer than you might think.
