At the edges of awareness

Imagine this. Following a brain injury you lie in a hospital bed and from the outside you appear to be totally unconscious. You don’t respond to anything the doctors or your family say, you make no voluntary movements, and although you still go to sleep and wake up there seems to be nobody at home. But your ‘inner universe’ of conscious awareness still remains, perhaps flickering and inconsistent, but definitely there. How could anyone else ever know, and how could you ever communicate with your loved ones again?

Two new radio dramas, The Sky is Wider and Real Worlds, engage with these critical questions by drawing on the cutting edge of the neurology and neuroscience. Recent advances have enabled researchers to not only diagnose ‘residual’ awareness following severe brain injuries, but also to open new channels of communication with behaviourally unresponsive patients. The key medical challenge is to distinguish between the so-called ‘vegetative state’ in which there truly is no conscious awareness, from ‘minimally conscious’ or ‘locked-in’ conditions where some degree of consciousness persists (even normal consciousness, in the locked-in state), even though there are no outward signs.


Brain activity during mental imagery, in a behaviourally unresponsive patient and in a  control subject.  Source: MRC via The Guardian

Linda Marshall Griffith’s drama The Sky is Wider takes inspiration from an ‘active approach’ in which the neurologist asks questions of the patient and monitors their brain activity for signs of response. In a classic study from about 10 years ago, Adrian Owen and his team asked behaviourally unresponsive patients to imagine either walking around their house or playing tennis, while their brains were scanned using functional MRI (which measures regional metabolic activity in the brain). These questions were chosen because imagining these different behaviours activates different parts of the brain, and so if we see these selective activations in a patient, we know that they have understood and are voluntarily following the instructions. If they can do this, they must be conscious. It turns out that between 10-20% of patients behaviourally diagnosed as being in the vegetative state can pass this test. Equally important, this same method can be used to establish simple communication by (for example) asking a patient to imagine playing ‘tennis’ to answer ‘yes’ and walking around a house to answer ‘no’.

These developments represent a revolution in clinical neurology. Current research is increasing the efficiency of active approaches by using the more portable electroencephalography (EEG) instead of bulky and expensive MRI. ‘Passive’ techniques in which residual consciousness can be inferred without requiring patients to perform any task are also rapidly improving. These methods are important because active approaches may underestimate the incidence of residual awareness since not all conscious patients may understand or be able to follow verbal instructions.

Alongside these scientific developments we encounter pressing ethical questions. How should we treat patients in these liminal states of awareness? And given a means of communication, what kinds of questions should we ask? The Sky is Wider explores these challenging ethical issues in a compelling narrative which gives dramatic voice to the mysterious conditions of the vegetative and minimally conscious states.


In Real Worlds, Jane Rogers takes us several years into the future. Communication with behaviourally unresponsive patients is now far advanced and is based on amazing developments in ‘virtual reality’. The clinical context for this drama is the ‘locked-in syndrome’ where a patient may have more-or-less normal conscious experiences but completely lack the ability to move. In Real Worlds, a locked-in patient transcends these limitations by controlling a virtual reality avatar directly using brain signals. These avatars inhabit virtual worlds in which the avatars of different people can interact, while the ‘real’ person behind each may remain hidden and unknown.

This drama deliberately inhabits the realm of science fiction, but there is solid science behind it too. The development of so-called ‘brain computer interfaces’ (BCI) is moving fast. These interfaces combine brain imaging methods (like EEG or fMRI, or sometimes more ‘invasive’ methods’ in which electrodes are inserted directly into the brain) with advanced machine learning methods to perform a kind of ‘brain-reading’. The idea is to infer, from brain activity alone, intended movements, perceptions, and perhaps even thoughts. These decoded ‘thoughts’ can then be used to control robotic devices, or virtual avatars. In some cases, a person’s own body might be controlled via direct stimulation of muscles. Progress in this area has been remarkably rapid. In a landmark but rather showy example, the Brazilian neuroscientist Miguel Nicolelis used a BCI to allow a paralysed person to ‘kick’ the first ball of the 2014 football world cup, through brain-control of a robotic avatar. More recently, brain-reading methods have allowed a paralysed man to play Guitar Hero for the first time since his injury.

The other technology highlighted in Real Worlds is virtual reality (VR), which – thanks to its enormous consumer potential – is developing even more rapidly. All the major technology and AI companies are getting in on the act, and VR headsets are finally becoming cheap enough, comfortable enough, and powerful enough to define a new technological landscape. Here at the Sackler Centre for Consciousness Science at the University of Sussex, we are exploring how VR can help shed light on our normal conscious experience. In one example, we use a method called ‘augmented reality’ (AR) to project a ‘virtual’ body into the real world as seen through a camera mounted on the front of a VR headset. This experiment revealed how our perception of what is (and what is not) our own body can be easily manipulated, indicating that our experience of ‘body ownership’, which is so easy to take for granted, is in fact continuously and actively generated by the brain. In a second example, we developed a method called ‘substitutional reality’ in which a VR headset is coupled with panoramic video and audio taken from a real environment, manipulated in various ways. The resulting experiences are much more immersive than current computer-generated virtual environments and in some cases people cannot distinguish them from actually ‘real’ environments.


A ‘virtual reality’ hand, part of a Sackler Centre study to explore the mechanisms underlying experiences of body ownership.  VR programming by Dr. Keisuke Suzuki.

Just as in the first drama, ethical questions risk outpacing the science and technology. As VR becomes increasingly immersive and pervasive, its potential to impact our real lives is ever more powerful. While benefits are easy to imagine – for instance in bringing distant relatives together or enabling remote experiences of inaccessible places – there are also legitimate concerns. High on the list would be what happens if people become increasingly unable to distinguish the real world from the virtual, whether in the moment or (more plausibly) in their memories. And what if they progressively withdrew from ‘reality’ if the available virtual worlds became more appealing places to be? Of course, simple dichotomies are unhelpful since VR technologies are part of our real worlds, just like mobile phones and laptop computers. Jane Rogers’ Real Worlds explores these complex ethical issues by imagining VR as a future treatment – perhaps ‘prosthesis’ would be a better word – for the disorders of consciousness like those encountered in The Sky is Wider.

Together, these dramas explore the human and societal consequences of existing and near-future clinical technologies. With artistic license they ask important questions that scientists and clinicians are not yet equipped to address. Ultimately, I think they convey an optimistic message, that we can understand and treat – if not cure – severely debilitating conditions that may otherwise have remained undiagnosed let alone treated. But they also lead us to consider, not just what we could do, but what we should do.

The Sky is Wider (written by Linda Marshall Griffiths) and Real Worlds (written by Jane Rogers) were produced by Nadia Molinari for BBC Radio 4. I acted as the scientific consultant. The original ideas were formulated during a 2014 Wellcome Trust ‘Experimental Stories’ workshop in a conversation between myself, Nadia, and Linda.

The science of selfhood

lorna-zoe-wanamaker-by-johan-persson2-1200x800.jpgZoë Wanamaker as Lorna in Nick Payne’s Elegy.

“The brain is wider than the sky,
For, put them side by side,
The one the other would contain,
With ease, and you besides”

Emily Dickinson, Complete Poems, 1924

What does it mean to be a self? And what happens to the social fabric of life, to our ethics and morality, when the nature of selfhood is called into question?

In neuroscience and psychology, the experience of ‘being a self’ has long been a central concern. One of the most important lessons, from decades of research, is that there is no single thing that is the self. Rather, the self is better thought of as an integrated network of processes that distinguish self from non-self at many different levels. There is the bodily self – the experience of identifying with and owning a particular body, which at a more fundamental level involves the amorphous experience of being a self-sustaining organism. There is the perspectival self, the experience of perceiving the world from a particular first-person point-of-view. The volitional self involves experiences of intention of agency, of urges to do this-or-that (or, perhaps more importantly, to refrain from doing this-or-that) and of being the cause of things that happen.

At higher levels we encounter narrative and social selves. The narrative self is where the ‘I’ comes in, as the experience of being a continuous and distinctive person over time. This narrative self – the story we tell ourselves about who we are – is built from a rich set of autobiographical memories that are associated with a particular subject. Finally, the social self is that aspect of my self-experience and personal identity that depends on my social milieu, on how others perceive and behave towards me, and on how I perceive myself through their eyes and minds.

In daily life, it can be hard to differentiate these dimensions of selfhood. We move through the world as seemingly unified wholes, our experience of bodily self seamlessly integrated with our memories from the past, and with our experiences of volition and agency. But introspection can be a poor guide. Many experiments and neuropsychological case studies tell a rather different story, one in which the brain actively and continuously generates and coordinates these diverse aspects of self-experience.

The many ways of being a self can come apart in surprising and revealing situations. For example, it is remarkably easy to alter the experience of bodily selfhood. In the so-called ‘rubber hand illusion,’ I ask you to focus your attention on a fake hand while your real hand is kept out of sight. If I then simultaneously stroke your real hand and the fake hand with a soft paintbrush, you may develop the uncanny feeling that the fake hand is now, somehow, part of your body. A more dramatic disturbance of the experience of body ownership happens in somatoparaphrenia, a condition in which people experience that part of their body is no longer theirs, that it belongs to someone else – perhaps their doctor or family member. Both these examples involve changes in brain activity, in particular within the ‘temporo-parietal junction’, showing how even very basic aspects of personal identity are actively constructed by the brain.

Moving through levels of selfhood, autoscopic hallucinations involve seeing oneself from a different perspective, much like ‘out of body’ experiences. In akinetic mutism, people seem to lack any experiences of volition or intention (and do very little), while in schizophrenia or anarchic hand syndrome, people can experience their intentions or voluntary actions as having external causes. At the other end of the spectrum, disturbances of social self emerge in autism, where difficulties in perceiving others’ states of mind seems to be a core problem, though the exact nature of the autistic condition is still much debated.

When it comes to the ‘I’, memory is the key. Specifically, autobiographical memory: the recollection of personal experiences of people, objects, and places and other episodes from an individual’s life. While there are as many types of memory as there are varieties of self (for example, we have separate memory processes for facts, for the short term and the long term, and for skills that we learn), autobiographical memories are those most closely associated with our sense of personal identity. This is well illustrated by some classic medical cases in which, as a result of surgery or disease, the ability to lay down new memories is lost. In 1953 Henry Moliason (also known as the patient HM) had large parts of his medial temporal lobes removed in order to relieve severe epilepsy. From 1957 until his death in 2008, HM was studied closely by the neuropsychologist Brenda Milner, yet he was never able to remember meeting her. In 1985 the accomplished musician Clive Wearing suffered a severe viral brain disease that affected similar parts of his brain. Now 77, he frequently believes he has just awoken from a coma, spending each day in a constant state of re-awakening.

Surprisingly, both HM and Wearing remained able to learn new skills, forming new ‘procedural’ memories, despite never recalling the learning process itself. Wearing could still play the piano, and conduct his choir, though he would immediately forget having done so. The music appears to carry him along from moment to moment, restoring his sense of self in a way his memory no longer can. And his love for his wife Deborah seems undiminished, so that he expresses an enormous sense of joy on seeing her, even though he cannot tell whether their last meeting was years, or seconds, in the past. Love, it seems, persists when much else is gone.

For people like HM and Clive Wearing, memory loss has been unintended and unwanted. But as scientific understanding develops, could we be moving towards a world where specific memories and elements of our identity can be isolated or removed through medical intervention? And could the ability to lay down new memories ever be surgically restored? Some recent breakthroughs suggest these developments may not be all that far-fetched.

In 2013, Jason Chan and Jessica LaPaglia, from Iowa State University showed that specific human memories could indeed be deleted. They took advantage of the fact that when memories are explicitly recalled they become more vulnerable. By changing details about a memory, while it was being remembered, they induced a selective amnesia which lasted for at least 24 hours. Although an important advance, this experiment was limited by relying on ‘non-invasive’ methods – which means not using drugs or directly interfering with the brain.

More recent animal experiments have shown even more striking effects. In a ground-breaking 2014 study at the University of California, using genetically engineered mice, Sadegh Nabavi and colleagues managed to block and then re-activate a specific memory. They used a powerful (invasive) technique called optogenetics to activate (or inactivate) the biochemical processes determining how neurons change their connectivity. And elsewhere in California, Ted Berger is working on the first prototypes of so-called ‘hippocampal prostheses’ which replace a part of the brain essential for memory with a computer chip. Although these advances are still a long way from implementation in humans, they show an extraordinary potential for future medical interventions.

The German philosopher Thomas Metzinger believes that “no such things as selves exist in the world”. Modern neuroscience may be on his side, with memory being only one thread in the rich tapestry of processes shaping our sense of selfhood. At the same time, the world outside the laboratory is still full of people who experience themselves – and each other – as distinct, integrated wholes. How the new science of selfhood will change this everyday lived experience, and society with it, is a story that is yet to be told.

Originally commissioned for the Donmar Warehouse production of Elegy, with support from The Wellcome Trust.  Reprinted in the programme notes and in Nick Payne’s published script.

Tracing the edges of consciousness


As a scientist, consciousness has always fascinated me. But understanding consciousness is not a project for science alone. Throughout history, philosophers, artists, storytellers, and musicians have all wondered about the apparent miracle of conscious awareness. Even today, while science might give us our best shot at figuring out the brain – the organ of experience – we need, more than ever, a melding of the arts and sciences, of contemporary and historical approaches, to understand what consciousness really is, to grasp what we mean by, as Mark Haddon eloquently puts it, “Life in the first person.”

This quote comes from Haddon’s beautiful introductory essay to a major new exhibition at the Wellcome Collection in London. Curated by Emily Sargent, States of Mind: Tracing the edges of consciousness “examines perspectives from artists, psychologists, philosophers and neuroscientists to interrogate our understanding of the conscious experience”. Its a fantastic exhibition, with style and substance, and I feel very fortunate to have been involved as an advisor from its early stages.

What’s so special about consciousness?

Consciousness is at once the most familiar and the most mysterious aspect of our existence. Conscious experiences define our lives, but the private, subjective, and what-it-is-likeness of these experiences seems to resist scientific enquiry. Somehow, within each our brains the combined activity of many billions of neurons, each one a tiny biological machine, is giving rise to a conscious experience. Your conscious experience: right here, right now, reading these words. How does this happen? Why is life in the first person?

In one sense, this seems like the kind of mystery ripe for explanation. Borrowing again from Mark Haddon, the raw material of consciousness is not squirreled away deep inside an atom, its not happening 14 billion years ago, and its not hiding out on the other side of the universe. It’s right here in front of – or rather behind – our eyes. Saying this, the brain is a remarkably complex object. It’s not so much the sheer number of neurons (though there about 90 billion). It’s the complexity of its wiring: there are so many connections, that if you counted one every second it would take you 3 million years to finish. Is it not possible that an object of such extraordinary complexity should be capable of extraordinary things?

People have been thinking about consciousness since they’ve been thinking at all. Hippocrates, the founder of modern medicine, said: “Men ought to know that from the brain, and from the brain only, arise our pleasures, joys, laughter and jests, as well as our sorrows, pains, griefs and tears … Madness comes from its moistness.” (Aristotle, by the way, got it wrong, thinking the brain hadn’t much to do with consciousness at all.)

Fast forward to Francis Crick, whose ‘astonishing hypothesis’ in the 1990s deliberately echoed Hippocrates: “You, your joys and your sorrows, your memories and your ambitions … and so on … are in fact no more than the behaviour of a vast assembly of nerve cells and their associated molecules”. Crick, who I was lucky enough to meet during my time in America, was working on the neurobiology of consciousness even on the day he died. You will see some of his personal notes, and his perplexing plasticine brain models, in States of Mind.

L0080096 Descartes: view of posterior of brain

Descartes: view of posterior of brain, from De Hominem. Wellcome Collection

A major landmark in thinking about consciousness is of course Descartes, who in the 17th Century distinguished between “mind stuff” (res cogitans) and “matter stuff” (res extensa), so giving rise to the now infamous mind-body problem and the philosophy of dualism. Its a great thrill for to see an original copy of Descartes’ De Homine as part of this exhibition. Its modern incarnation as David Chalmers’ so-called ‘hard problem’ has recently gained enough cultural notoriety even to inspire a Tom Stoppard play (though for my money Alex Garland’s screenplay for Ex Machina is the more perspicuous). The idea of the hard problem is this: Even if we knew everything about how the operations of the brain give rise to perception, cognition, learning, and behaviour a problem would still remain: why and how any of this should be associated with consciousness at all? Why is life in the first person?

Defining consciousness

How to define consciousness? One simple definition is that for a conscious organism there is something it is like to be that organism. Or, one can simply say that consciousness is what disappears when we fall into a dreamless sleep, and what returns when we wake up or start dreaming. A bit more formally, for conscious organisms there exists a continuous (though interruptible) stream of conscious scenes – a phenomenal world – which has the character of being subjective and private. The material in States of Mind can help us encounter these ideas with a bit more clarity and force, by focusing on the edges – the liminal boundaries – of consciousness.

First there is conscious level: the difference between being awake and, let’s say, under general anaesthesia. Here, neuroscience now tells us that there is no single ‘generator’ of consciousness in the brain, rather, being consciousness depends on highly specific ways in which different parts of the brain speak to each other. Aya Ben Ron’s film of patients slipping away under anaesthesia is a beautiful exploration of this process, as is the whole section on ‘SLEEP | AWAKE’.

Then there is conscious content: what we are conscious of, when we are conscious. These are the perceptions, thoughts, and emotions that populate our ever-flowing stream of awareness. Here, current research is revealing that our perceptual world is not simply an internal picture of some external reality. Rather, conscious perception depends on the brain’s best guesses, or hypotheses, about the causes of sensory data. Perception is therefore a continuously creative act that is tightly bound up with imagination, so that our experience of the world is a kind of ‘controlled hallucination’, a fantasy that – usually, but not always – coincides with reality. The material on synaesthesia in States of Mind beautifully illuminates this process by showing how, for some of us, these perceptual fantasies can be very different – that we all have our own distinctive inner universes. You can even try training yourself to become a ‘synaesthete’ with a demo of some of our own research, developed for this exhibition. Many thanks to Dr. David Schwartzman of the Sackler Centre for making this happen.


Alphabet in Colour: Illustrating Vladimir Nabokov’s grapheme-colour synaesthesia, by Jean Holabird.

Finally there is conscious self – the specific experience of being me, or being you. While this might seem easy to take for granted, the experience of being a self requires explanation just as much as any other kind of experience. It too has its edges, its border regions. Here, research is revealing that conscious selfhood, though experienced as unified, can come apart in many different ways. For example, our experience of being and having a particular body can dissociate from our experience of being a person with name and a specific set of memories. Conscious selfhood, like all conscious perception, is therefore another controlled hallucination maintained by the brain. The section BEING | NOT BEING dramatically explores some of these issues, for example by looking at amnesia with Shona Illingworth, and with Adrian Owen’s seminal work on the possibility of consciousness even after severe brain injury.

This last example brings up an important point. Besides the allure of basic science, there are urgent practical motivations for studying consciousness. Neurological and psychiatric disorders are increasingly common and can often be understood as disturbances of conscious experience. Consciousness science promises new approaches and perhaps new treatments for these deeply destructive problems. Scoping out further boundary areas, studying the biology of consciousness can shed new light on awareness in newborn infants and in non-human animals, informing ethical debates in these areas. Above all, consciousness science carries the promise of understanding more about our place in nature. Following the tradition of Copernicus and Darwin, a biological account of conscious experience will help us see ourselves as part of, not apart from, the rest of the universe.

L0079940 Neuronal Theory - 11312.

Santiago Ramon y Cajal, distinguishing the reticular theory (left) from the neuron doctrine (right).  From the Instituto Cajal, Madrid.

Let’s finish by returning to this brilliant exhibition, States of Mind. What I found most remarkable are the objects that Emily Sargent has collected together. Whether its Descartes’ De Hominem, Ramon y Cajal’s delicate ink drawings of neurons, or Francis Crick’s notebooks and models, these objects bring home and render tangible the creativity and imagination which people have brought to bear in their struggle to understand consciousness, over hundreds of years. For me, this brings a new appreciation and wonder to our modern attempts to tackle this basic mystery of life. Emily Dickinson, my favourite poet of neuroscience, put it like this. “The brain is wider than the sky, for – put them side by side – the one the other will contain, with ease, and you – beside.”

States of Mind is at the Wellcome Collection in London from Feb 4th until October 16th 2016 and is curated by Emily Sargent. Sackler Centre researchers, in particular David Schwartzman and myself,  helped out as scientific advisors. This text is lightly adapted from a speech I gave at the opening event on Feb 3rd 2016. Watch this space, and visit the exhibition website, for news about special events on consciousness that will happen throughout the year.

States of Mind at the Wellcome Collection


YellowPinkBlue by Ann Veronica Janssens

From October 2015 until October 2016 the Wellcome Collection in London is curating an exhibition called States of Mind: Tracing the Edges of Consciousness.  It has been launched with a brilliant piece of installation art by Ann Veronica Janssens (until 3rd Jan 2016).  In YellowPinkBlue the entire gallery space is invaded by coloured mist, to focus attention on the process of perception itself so that one becomes subsumed by the experience of seeing.  I’m excited to be contributing in various ways to States of Mind, via the Sackler Centre (more on that soon). To start with, here is the text I wrote for Janssen’s remarkable piece.

What in the world is consciousness?

Right now an apparent miracle is unfolding. Within your brain, the electrochemical activity of many billions of richly interconnected brain cells – each one a tiny biological machine – is giving rise to a conscious experience. Your conscious experience: right here, right now, reading these words.

It is all too easy to go about our daily lives, having conscious experiences, without appreciating how remarkable it is that we have these experiences at all. Ann Veronica Janssens’s piece returns us to the sheer wonder of being conscious. By stripping away many of the features that permeate our normal conscious lives, the raw fact of experiencing is given renewed emphasis.

People have wondered about consciousness since they’ve wondered about anything. Hippocrates, the Greek founder of modern medicine, rightly identified the brain as the organ of experience (though Aristotle didn’t agree). In the Renaissance, Descartes divided the universe into ‘mind stuff’ (res cogitans) and ‘matter stuff’ (res extensa), giving birth to the philosophy of dualism and the confounding ‘mind–body’ problem of how the two relate. In the 19th century, when psychology first emerged as a science, understanding consciousness was its primary objective. Though largely sidelined during the 20th century, the challenge of revealing the biological basis of consciousness is now firmly re-established for our times. Janssens’s piece reminds us of the important distinction in science between being conscious at all (conscious level: the difference between being awake and being in a dreamless sleep or under anaesthesia) and what we are conscious of (conscious content: the perceptions, thoughts and emotions that populate our conscious mind). There is also conscious selfhood – the specific experience of being me (or you). Each of these aspects of consciousness can be traced to specific mechanisms in the brain that neuroscientists, in cahoots with researchers from many other disciplines, are now starting to unravel. There are many exciting ideas in play, ranging from the dependence of conscious level on how different parts of the brain speak to each other, to understanding conscious content as determined by the brain’s ‘best guess’ of the causes of ambiguous and noisy sensory signals. Crucially, these ideas have allowed consciousness science to progress from the philosopher’s armchair to the research laboratory.

Besides the allure of basic science, there are important practical motivations for studying consciousness. Neurological and psychiatric disorders are increasingly common and can often be framed as disturbances of conscious experience. Consciousness science promises new approaches and perhaps new treatments for these scourges of modern society. New theories and experiments can also shed light on consciousness in newborns and in non-human animals, adding critical information to important ethical debates in these areas. But above all, consciousness science carries the promise of understanding more about our place in nature. Following Darwin and Copernicus, a biological account of conscious experience will help us see ourselves as part of, not apart from, the rest of the universe.

Anil Seth, Professor of Cognitive and Computational Neuroscience
Co-Director, Sackler Centre for Consciousness Science, University of Sussex

A first draft of a digital brain: The Human Brain Project’s new simulation


Today, Henry Markram and colleagues have released one of the first of a raft of substantial new results emerging from the controversial Human Brain Project (HBP). The paper, Reconstruction and Simulation of Neocortical Microcircuitry, appears in the journal Cell.

As one of the first concrete outputs emerging from this billion-euro endeavour this had to be a substantial piece of work, and it is. The paper describes a digital reconstruction of ~31,000 neurons (with ~8 million connections and ~37 million synapses) of a tiny part of the somatosensory cortex of the juvenile rat brain. What is unique about this simulation is not the number of neurons (31,000 is pretty modest by today’s standards), but the additional detail included. Simulated neurons are given specific morphological, chemical, and electrical characteristics, and are precisely positioned in 3D space so that they form biologically realistic connections. This level of detail is at the heart of the HBP strategy, and it underlies the claim that the simulation is a ‘reconstruction’ of neural tissue, not just an abstract model of neuronal connectivity.

So how good is it? Certainly, the simulation detail is extremely impressive, as is the wealth of experimental data that is accounted for. Particularly striking is the ability to predict both general features of neocortical dynamics – like the existence of ‘soloist’ and ‘chorister’ neurons – as well as to inspire specific new experiments that further validated the simulation. It is also promising that Markram & co managed to interpolate their sparse experimental data in order to fully specify the model, without losing the fidelity of the model to the real ‘target’ system.

The authors admit this is a ‘first step’ and the results are certainly intriguing. But the real question is whether the aggressively ‘bottom up’ approach of the HBP will, by itself, yield the transformational understanding of neuroscience that it has promised. Modelling work in science – whether computational or mathematical – is about finding the right level of abstraction to best explore and understand some natural principle, or test some specific hypothesis.  A model that relies on incorporating as much detail as possible could lead to a simulation that is almost as hard to understand as the target system. Jorge Luis Borges long ago noted the tragic uselessness of the perfectly detailed map in his short story ‘On Exactitude in Science’.

For this reason alone, its hard to be confident that the HBP approach — impressive as it is at the level of a tiny volume of immature cortex — will scale up to deliver real insights about how brains, bodies and environments mesh together in generating complex adaptive behaviour (and perception, and thought, and consciousness). And on the other hand, as detailed as the current simulation is, it still neglects very basic and undoubtedly important aspects of the brain – including glial cells, vasculature, receptors, and the like. This goes to show that even the most detailed simulation models still have to make abstractions. In the present model decisions about what is included and excluded seem to be made more according to practical criteria (what is possible?) than theoretically principled criteria (what are we trying to explain with this model?).

Can the HBP be extended both downwards (to encompass the so-far excluded but potentially critical details of neuronal microstructure) and upwards (to a whole brain and organism level, including sensorimotor interactions with bodies and environments)? The jury is still out. So let’s applaud this Herculean effort to simulate a tiny part of a tiny brain, but let’s also keep in mind that the HBP won’t solve neuroscience all by itself, and only time will tell whether it will play a significant role in unravelling the properties of the most complex object in the known universe.

The original article is here: Markram et al (2015). Cell 163:1-37.
Some of the above comments appear in a New Scientist commentary by Jessica Hamzelou, published 08/10/2015:  Digital version of piece of rat brain fires like the real thing.

Brain Twisters

BrainTwistBrain Twisters, the follow up to the Royal Society Prize Winning ‘EyeBenders’ is now out! Another co-production with author Clive Gifford.

Here’s the blurb: Trick your senses and baffle your brain with this crazy book of mind tricks and neuroscience information. Find out how magicians make use of “inattentional blindness” when doing magic tricks, and why you miss details that are hidden in plain sight. Discover why your memory isn’t as good as you think, and how it’s possible to remember things that never actually happened. This astonishing science book presents a wide range of brain games and mind tricks, and explains how these reveal the working processes of the brain. It will engage and entertain, and leave you wondering: do you really know your own mind?

I really enjoyed working on this book with Clive.  Its more ambitious than EyeBenders – taking on the whole of the brain rather than just optical illusions. But I think the end result works brilliantly – though let’s see what the kids think!

Can we figure out the brain’s wiring diagram?


The human brain, it is often said, is the most complex object in the known universe. Counting all the connections among its roughly 90 billion neurons, at the rate of one each second, would take about 3 million years – and just counting these connections says nothing about their intricate patterns of connectivity. A new study, published this week in Proceedings of the National Academy of Sciences USA, shows that mapping out these patterns is likely to be much more difficult than previously thought — but also shows what we need to do, to succeed.

Characterizing the detailed point-to-point connectivity of the brain is increasingly recognized as a key objective for neuroscience. Many even think that without knowing the ‘connectome’ – the brain’s wiring diagram – we will never understand how its electrochemical alchemy gives rise to our thoughts, actions, perceptions, beliefs, and ultimately to our consciousness. There is a good precedent for thinking along these lines. Biology has been galvanized by sequencing of the genome (of humans and of other species), and genetic medicine is gathering pace as whole-genome sequencing becomes fast and cheap enough to be available to the many, not just the few. Big-science big-money projects like the Human Genome Project were critical to these developments. Similar efforts in brain science – like the Human Connectome Project in the US and the Human Brain Project in Europe – are now receiving vast amounts of funding (though not without criticism, especially in the European case) (see also here). The hope is that the genetic revolution can be replicated in neuroscience, delivering step changes in our understanding of the brain and in our ability to treat neurological and psychiatric disorders.

Mapping the networks of the human brain relies on non-invasive neuroimaging methods that can be applied without risk to living people. These methods almost exclusively depend on ‘diffusion magnetic resonance imaging (dMRI) tractography’. This technology measures, for each location (or ‘voxel’) in the brain, the direction in which water is best able to diffuse. Taking advantage of the fact that water diffuses more easily along the fibre bundles connecting different brain regions, than across them, dMRI tractography has been able to generate accurate, informative, and surprisingly beautiful pictures of the major superhighways in the brain.

Diffusion MRI of the human brain.  Source: Human Connectome Project.

Diffusion MRI of the human brain. Source: Human Connectome Project.

But identifying these neuronal superhighways is only a step towards the connectome. Think of a road atlas: knowing only about motorways may tell you how cities are connected, but its not going to tell you how to get from one particular house to another. The assumption in neuroscience has been that as brain scanning improves in resolution and as tracking algorithms gain sophistication, dMRI tractography will be able to reveal the point-to-point long-range anatomical connectivity needed to construct the full connectome.

In a study published this week we challenge this assumption, showing that basic features of brain anatomy pose severe obstacles to measuring cortical connectivity using dMRI. The study, a collaboration between the University of Sussex in the UK and the National Institutes of Health (NIH) in the US, applied dMRI tractography to ultra-high resolution dMRI data obtained from extensive scanning of the macaque monkey brain – data of much higher quality than can be presently obtained from human studies. Our analysis, led by Profs. Frank Ye and David Leopold of NIH and Ph.D student Colin Reveley of Sussex, took a large number of starting points (‘seed voxels’) in the brain, and investigated which other parts of the brain could be reached using dMRI tractography.

The result: roughly half of the brain could not be reached, meaning that even our best methods for mapping the connectome aren’t up to the job. What’s more, by looking carefully at the actual brain tissue where tractography failed, we were able to figure out why. Lying just beneath many of the deep valleys in the brain (the ‘sulci’ – but in some other places too), are dense weaves of neuronal fibres (‘white matter’) running largely parallel to the cortical surface. The existence of these ‘superficial white matter fibre systems’, as we call them, prevents the tractography algorithms from detecting where small tributaries leave the main neuronal superhighways, cross into the cortical grey matter, and reach their destinations. Back to the roads: imagine that small minor roads occasionally leave the main motorways, which are flanked by other major roads busy with heavy traffic. If we tried to construct a detailed road atlas by measuring the flow of vehicles, we might well miss these small but critical branching points.

This image shows, on a colour scale, the 'reachability' of different parts of the brain by diffusion tractography.

This image shows, on a colour scale, the ‘reachability’ of different parts of the brain by diffusion tractography.

Identifying the connectome remains a central objective for neuroscience, and non-invasive brain imaging – especially dMRI – is a powerful technology that is improving all the time. But a comprehensive and accurate map of brain connectivity is going to require more than simply ramping up scanning resolution and computational oomph, a message that mega-budget neuroscience might usefully heed. This is not bad news for brain research. Solving a problem always requires fully understanding what the problem is, and our findings open new opportunities and objectives for studies of brain connectivity. Still, it goes to show that the most complex object in the universe is not quite ready to give up all its secrets.

Colin Reveley, Anil K. Seth, Carlo Pierpaoli, Afonso C. Silva, David Yu, Richard C. Saunders, David A. Leopold*, and Frank Q. Ye. (2015) Superficial white-matter fiber systems impede detection of long-range cortical connections in diffusion MR tractography. Proc. Nat. Acad. Sci USA (2015). doi/10.1073/pnas.1418198112

*David A. Leopold is the corresponding author.

Ex Machina: A shot in the arm for smart sci-fi


Alicia Vikander as Ava in Alex Garland’s Ex Machina

IT’S a rare thing to see a movie about science that takes no prisoners intellectually. Alex Garland’s Ex Machina is just that: a stylish, spare and cerebral psycho-techno-thriller, which gives a much-needed shot in the arm for smart science fiction.

Reclusive billionaire genius Nathan, played by Oscar Isaac, creates Ava, an intelligent and very attractive robot played by Alicia Vikander. He then struggles with the philosophical and ethical dilemmas his creation poses, while all hell breaks loose. Many twists and turns add nuance to the plot, which centres on the evolving relationships between the balletic Ava and Caleb (Domhnall Gleeson), a hotshot programmer invited by Nathan to be the “human component in a Turing test”, and between Caleb and Nathan, as Ava’s extraordinary capabilities become increasingly apparent

Everything about this movie is good. Compelling acting (with only three speaking parts), exquisite photography and set design, immaculate special effects, a subtle score and, above all, a hugely imaginative screenplay combine under Garland’s precise direction to deliver a cinematic experience that grabs you and never lets go.

The best science fiction often tackles the oldest questions. At the heart of Ex Machina is one of our toughest intellectual knots, that of artificial consciousness. Is it possible to build a machine that is not only intelligent but also sentient: that has consciousness, not only of the world but also of its own self? Can we construct a modern-day Golem, that lumpen being of Jewish folklore which is shaped from unformed matter and can both serve humankind and turn against it? And if we could, what would happen to us?

In Jewish folkore, the Golem is animate being shaped from unformed matter.

In Jewish folkore, the Golem is animate being shaped from unformed matter.

Putting aside the tedious business of actually building a conscious AI, we face the challenge of figuring out whether the attempt succeeds. The standard reference for this sort of question is Alan Turing’s eponymous test, in which a human judge interrogates both a candidate machine and another human. A machine passes the test when the judge consistently fails to distinguish between them.

While the Turing test has provided a trope for many AI-inspired movies (such as Spike Jonze’s excellent Her), Ex Machina takes things much further. In a sparkling exchange between Caleb and Nathan, Garland nails the weakness of Turing’s version of the test, a focus on the disembodied exchange of messages, and proposes something far more interesting. “The challenge is to show you that she’s a robot. And see if you still feel she has consciousness,” Nathan says to Caleb.

This shifts the goalposts in a vital way. What matters is not whether Ava is a machine. It is not even whether Ava, even though a machine, can be conscious. What matters is whether Ava makes a conscious person feel that Ava is conscious. The brilliance of Ex Machina is that it reveals the Turing test for what it really is: a test of the human, not of the machine. And Garland is not necessarily on our side.

Nathan (Oscar Isaac) and Caleb (Domnhall Gleeson) discuss deep matters of AI

Nathan (Oscar Isaac) and Caleb (Domnhall Gleeson) discuss deep matters of AI

Is consciousness a matter of social consensus? Is it more relevant whether people believe (or feel) that something (or someone) is conscious than whether it is in fact actually conscious? Or, does something being “actually conscious” rest on other people’s beliefs about it being conscious, or on its own beliefs about its consciousness (beliefs that may themselves depend on how it interprets others’ beliefs about it)? And exactly what is the difference between believing and feeling in situations like this?

It seems to me that my consciousness, here and now, is not a matter of social consensus or of my simply believing or feeling that I am conscious. It seems to me, simply, that I am conscious here and now. When I wake up and smell the coffee, there is a real experience of coffee-smelling going on.

But let me channel Ludwig Wittgenstein, one of the greatest philosophers of the 20th century, for a moment. What would it seem like if it seemed to me that my being conscious were a matter of social consensus or beliefs or feelings about my own conscious status? Is what it “seems like” to me relevant at all when deciding how consciousness comes about or what has consciousness?

Before vanishing completely into a philosophical rabbit hole, it is worth saying that questions like these are driving much influential current research on consciousness. Philosophers and scientists like Daniel Dennett, David Rosenthal and Michael Graziano defend, in various ways, the idea that consciousness is somehow illusory and what we really mean in saying we are conscious is that we have certain beliefs about mental states, that these states have distinctive functional properties, or that they are involved in specific sorts of attention.

Another theoretical approach accepts that conscious experience is real and sees the problem as one of determining its physical or biological mechanism. Some leading neuroscientists such as Giulio Tononi, and recently, Christof Koch, take consciousness to be a fundamental property, much like mass-energy and electrical charge, that is expressed through localised concentrations of “integrated information”. And others, like philosopher John Searle, believe that consciousness is an essentially biological property that emerges in some systems but not in others, for reasons as-yet unknown.

In the film we hear about Searle’s Chinese Room thought experiment. His premise was that researchers had managed to build a computer programmed in English that can respond to written Chinese with written Chinese so convincingly it easily passes the Turing test, persuading a human Chinese speaker that the program understands and speaks Chinese. Does the machine really “understand” Chinese (Searle called this “strong AI”) or is it only simulating the ability (“weak” AI)? There is also a nod to the notional “Mary”, the scientist, who, while knowing everything about the physics and biology of colour vision, has only ever experienced black, white and shades of grey. What happens when she sees a red object for the first time? Will she learn anything new? Does consciousness exceed the realms of knowledge.

All of the above illustrates how academically savvy and intellectually provocative Ex Machina is. Hat-tips here to Murray Shanahan, professor of cognitive robotics at Imperial College London, and writer and geneticist Adam Rutherford, whom Garland did well to enlist as science advisers.

Not every scene invites deep philosophy of mind, with the film encompassing everything from ethics, the technological singularity, Ghostbusters and social media to the erosion of privacy, feminism and sexual politics within its subtle scope. But when it comes to riffing on the possibilities and mysteries of brain, mind and consciousness, Ex Machina doesn’t miss a trick.

As a scientist, it is easy to moan when films don’t stack up against reality, but there is usually little to be gained from nitpicking over inaccuracies and narrative inventions. Such criticisms can seem petty and reinforcing of the stereotype of scientists as humourless gatekeepers of facts and hoarders of equations. But these complaints sometimes express a sense of missed opportunity rather than injustice, a sense that intellectual riches could have been exploited, not sidelined, in making a good movie. AI, neuroscience and consciousness are among the most vibrant and fascinating areas of contemporary science, and what we are discovering far outstrips anything that could be imagined out of thin air.

In his directorial debut, Garland has managed to capture the thrill of this adventure in a film that is effortlessly enthralling, whatever your background. This is why, on emerging from it, I felt lucky to be a neuroscientist. Here is a film that is a better film, because of and not despite its engagement with its intellectual inspiration.

The original version of this piece was published as a Culture Lab article in New Scientist on Jan 21. I am grateful to the New Scientist for permission to reproduce it here, and to Liz Else for help with editing. I will be discussing Ex Machina with Dr. Adam Rutherford at a special screening of the film at the Edinburgh Science Festival (April 16, details and tickets here).

Open your MIND

is a brand new collection of original research publications on the mind, brain, and consciousness
. It is now freely available online. The collection contains altogether 118 articles from 90 senior and junior researchers, in the always-revealing format of target articles, commentaries, and responses.

This innovative project is the brainchild of Thomas Metzinger and Jennifer Windt, of the MIND group of the Johanes Gutenburg University in Mainz, Germany (Windt has since moved to Monash University in Melbourne). The MIND group was set up by Metzinger in 2003 to catalyse the development of young German philosophers by engaging them with the latest developments in philosophy of mind, cognitive science, and neuroscience. Open MIND celebrates the 10th anniversary of the MIND group, in a way that is so much more valuable to the academic community than ‘just another meeting’ with its quick-burn excitement and massive carbon footprint. Editors Metzinger and Windt explain:

“With this collection, we wanted to make a substantial and innovative contribution that will have a major and sustained impact on the international debate on the mind and the brain. But we also wanted to create an electronic resource that could also be used by less privileged students and researchers in countries such as India, China, or Brazil for years to come … The title ‘Open MIND’ stands for our continuous search for a renewed form of academic philosophy that is concerned with intellectual rigor, takes the results of empirical research seriously, and at the same time remains sensitive to ethical and social issues.”

As a senior member of the MIND group, I was lucky enough to contribute a target article, which was commented on by Wanja Wiese, one of the many talented graduate students with Metzinger and a junior MIND group member. My paper marries concepts in cybernetics and predictive control with the increasingly powerful perspective of ‘predictive processing’ or the Bayesian brain, with a focus on interoception and embodiment. I’ll summarize the main points in a different post, but you can go straight to the target paper, Wanja’s commentary, and my response.

Open MIND is a unique resource in many ways. The Editors were determined to maximize its impact, so, unlike in many otherwise similar projects, the original target papers have not been circulated prior to launch. This means there is a great deal of highly original material now available to be discovered. The entire project was compressed into about 10 months from submission of initial drafts, to publication this week of the complete collection. This means the original content is completely up-to-date. Also, Open MIND  shows how excellent scientific publication can  sidestep the main publishing houses, given the highly developed resources now available, coupled of course with extreme dedication and hard work. The collection was assembled, rigorously reviewed, edited, and produced entirely in-house – a remarkable achievement.

Thomas Metzinger with the Open MIND student team

Thomas Metzinger with the Open MIND student team

Above all Open MIND opened a world of opportunity for its junior members, the graduate students and postdocs who were involved in every stage of the project: soliciting and reviewing papers, editing, preparing commentaries, and organizing the final collection. As Metzinger and Windt say

“The whole publication project is itself an attempt to develop a new format for promoting junior researchers, for developing their academic skills, and for creating a new type of interaction between senior and junior group members.”

The results of Open MIND are truly impressive and will undoubtedly make a lasting contribution to the philosophy of mind, especially in its most powerful multidisciplinary and empirically grounded forms.

Take a look, and open your mind too.

Open MIND contributors: Adrian John Tetteh Alsmith, Michael L. Anderson, Margherita Arcangeli, Andreas Bartels, Tim Bayne, David H. Baßler, Christian Beyer, Ned Block, Hannes Boelsen, Amanda Brovold, Anne-Sophie Brüggen, Paul M. Churchland, Andy Clark, Carl F. Craver, Holk Cruse, Valentina Cuccio, Brian Day, Daniel C. Dennett, Jérôme Dokic, Martin Dresler, Andrea R. Dreßing, Chris Eliasmith, Maximilian H. Engel, Kathinka Evers, Regina Fabry, Sascha Fink, Vittorio Gallese, Philip Gerrans, Ramiro Glauer, Verena Gottschling, Rick Grush, Aaron Gutknecht, Dominic Harkness, Oliver J. Haug, John-Dylan Haynes, Heiko Hecht, Daniela Hill, John Allan Hobson, Jakob Hohwy, Pierre Jacob, J. Scott Jordan, Marius Jung, Anne-Kathrin Koch, Axel Kohler, Miriam Kyselo, Lana Kuhle, Victor A. Lamme, Bigna Le Nggenhager, Caleb Liang, Ying-Tung Lin, Christophe Lopez, Michael Madary, Denis C. Martin, Mark May, Lucia Melloni, Richard Menary, Aleksandra Mroczko-Wąsowicz, Saskia K. Nagel, Albert Newen, Valdas Noreika, Alva Noë, Gerard O’Brien, Elisabeth Pacherie, Anita Pacholik-Żuromska, Christian Pfeiffer, Iuliia Pliushch, Ulrike Pompe-Alama, Jesse J. Prinz, Joëlle Proust, Lisa Quadt, Antti Revonsuo, Adina L. Roskies, Malte Schilling, Stephan Schleim, Tobias Schlicht, Jonathan Schooler, Caspar M. Schwiedrzik, Anil Seth, Wolf Singer, Evan Thompson, Jarno Tuominen, Katja Valli, Ursula Voss, Wanja Wiese, Yann F. Wilhelm, Kenneth Williford, Jennifer M. Windt.

Open MIND press release.
The cybernetic Bayesian brain: from interoceptive inference to sensorimotor contingencies
Perceptual presence in the Kuhnian-Popperian Bayesian brain
Inference to the best prediction