You press a light switch and the light comes on. What could be simpler than that. But notice something. As the light comes on, you probably have a feeling that, somehow, you caused that to happen. This experience of ‘being … Continue reading
Tag Archives: virtual reality
At the edges of awareness
Imagine this. Following a brain injury you lie in a hospital bed and from the outside you appear to be totally unconscious. You don’t respond to anything the doctors or your family say, you make no voluntary movements, and although you still go to sleep and wake up there seems to be nobody at home. But your ‘inner universe’ of conscious awareness still remains, perhaps flickering and inconsistent, but definitely there. How could anyone else ever know, and how could you ever communicate with your loved ones again?
Two new radio dramas, The Sky is Wider and Real Worlds, engage with these critical questions by drawing on the cutting edge of the neurology and neuroscience. Recent advances have enabled researchers to not only diagnose ‘residual’ awareness following severe brain injuries, but also to open new channels of communication with behaviourally unresponsive patients. The key medical challenge is to distinguish between the so-called ‘vegetative state’ in which there truly is no conscious awareness, from ‘minimally conscious’ or ‘locked-in’ conditions where some degree of consciousness persists (even normal consciousness, in the locked-in state), even though there are no outward signs.
Linda Marshall Griffith’s drama The Sky is Wider takes inspiration from an ‘active approach’ in which the neurologist asks questions of the patient and monitors their brain activity for signs of response. In a classic study from about 10 years ago, Adrian Owen and his team asked behaviourally unresponsive patients to imagine either walking around their house or playing tennis, while their brains were scanned using functional MRI (which measures regional metabolic activity in the brain). These questions were chosen because imagining these different behaviours activates different parts of the brain, and so if we see these selective activations in a patient, we know that they have understood and are voluntarily following the instructions. If they can do this, they must be conscious. It turns out that between 10-20% of patients behaviourally diagnosed as being in the vegetative state can pass this test. Equally important, this same method can be used to establish simple communication by (for example) asking a patient to imagine playing ‘tennis’ to answer ‘yes’ and walking around a house to answer ‘no’.
These developments represent a revolution in clinical neurology. Current research is increasing the efficiency of active approaches by using the more portable electroencephalography (EEG) instead of bulky and expensive MRI. ‘Passive’ techniques in which residual consciousness can be inferred without requiring patients to perform any task are also rapidly improving. These methods are important because active approaches may underestimate the incidence of residual awareness since not all conscious patients may understand or be able to follow verbal instructions.
Alongside these scientific developments we encounter pressing ethical questions. How should we treat patients in these liminal states of awareness? And given a means of communication, what kinds of questions should we ask? The Sky is Wider explores these challenging ethical issues in a compelling narrative which gives dramatic voice to the mysterious conditions of the vegetative and minimally conscious states.
In Real Worlds, Jane Rogers takes us several years into the future. Communication with behaviourally unresponsive patients is now far advanced and is based on amazing developments in ‘virtual reality’. The clinical context for this drama is the ‘locked-in syndrome’ where a patient may have more-or-less normal conscious experiences but completely lack the ability to move. In Real Worlds, a locked-in patient transcends these limitations by controlling a virtual reality avatar directly using brain signals. These avatars inhabit virtual worlds in which the avatars of different people can interact, while the ‘real’ person behind each may remain hidden and unknown.
This drama deliberately inhabits the realm of science fiction, but there is solid science behind it too. The development of so-called ‘brain computer interfaces’ (BCI) is moving fast. These interfaces combine brain imaging methods (like EEG or fMRI, or sometimes more ‘invasive’ methods’ in which electrodes are inserted directly into the brain) with advanced machine learning methods to perform a kind of ‘brain-reading’. The idea is to infer, from brain activity alone, intended movements, perceptions, and perhaps even thoughts. These decoded ‘thoughts’ can then be used to control robotic devices, or virtual avatars. In some cases, a person’s own body might be controlled via direct stimulation of muscles. Progress in this area has been remarkably rapid. In a landmark but rather showy example, the Brazilian neuroscientist Miguel Nicolelis used a BCI to allow a paralysed person to ‘kick’ the first ball of the 2014 football world cup, through brain-control of a robotic avatar. More recently, brain-reading methods have allowed a paralysed man to play Guitar Hero for the first time since his injury.
The other technology highlighted in Real Worlds is virtual reality (VR), which – thanks to its enormous consumer potential – is developing even more rapidly. All the major technology and AI companies are getting in on the act, and VR headsets are finally becoming cheap enough, comfortable enough, and powerful enough to define a new technological landscape. Here at the Sackler Centre for Consciousness Science at the University of Sussex, we are exploring how VR can help shed light on our normal conscious experience. In one example, we use a method called ‘augmented reality’ (AR) to project a ‘virtual’ body into the real world as seen through a camera mounted on the front of a VR headset. This experiment revealed how our perception of what is (and what is not) our own body can be easily manipulated, indicating that our experience of ‘body ownership’, which is so easy to take for granted, is in fact continuously and actively generated by the brain. In a second example, we developed a method called ‘substitutional reality’ in which a VR headset is coupled with panoramic video and audio taken from a real environment, manipulated in various ways. The resulting experiences are much more immersive than current computer-generated virtual environments and in some cases people cannot distinguish them from actually ‘real’ environments.
Just as in the first drama, ethical questions risk outpacing the science and technology. As VR becomes increasingly immersive and pervasive, its potential to impact our real lives is ever more powerful. While benefits are easy to imagine – for instance in bringing distant relatives together or enabling remote experiences of inaccessible places – there are also legitimate concerns. High on the list would be what happens if people become increasingly unable to distinguish the real world from the virtual, whether in the moment or (more plausibly) in their memories. And what if they progressively withdrew from ‘reality’ if the available virtual worlds became more appealing places to be? Of course, simple dichotomies are unhelpful since VR technologies are part of our real worlds, just like mobile phones and laptop computers. Jane Rogers’ Real Worlds explores these complex ethical issues by imagining VR as a future treatment – perhaps ‘prosthesis’ would be a better word – for the disorders of consciousness like those encountered in The Sky is Wider.
Together, these dramas explore the human and societal consequences of existing and near-future clinical technologies. With artistic license they ask important questions that scientists and clinicians are not yet equipped to address. Ultimately, I think they convey an optimistic message, that we can understand and treat – if not cure – severely debilitating conditions that may otherwise have remained undiagnosed let alone treated. But they also lead us to consider, not just what we could do, but what we should do.
The Sky is Wider (written by Linda Marshall Griffiths) and Real Worlds (written by Jane Rogers) were produced by Nadia Molinari for BBC Radio 4. I acted as the scientific consultant. The original ideas were formulated during a 2014 Wellcome Trust ‘Experimental Stories’ workshop in a conversation between myself, Nadia, and Linda.