For thousands of years people have wondered about the mystery of consciousness. How can anything made of physical stuff – a brain, for instance – be identical to, or give rise to, a subjective experience? Despite a revival in the … Continue reading
Our new paper on ‘measuring integrated information’ is out now, open access, in the journal Entropy. It’s part of a special issue dedicated to integrated information theory.
In consciousness research, ‘integrated information theory’, or IIT, has come to occupy a highly influential and rather controversial position. Acclaimed by some as the most important development in consciousness science so far, critiqued by others as too mathematically abstruse and empirically untestable, IIT is by turns both fascinating and frustrating. Certainly, a key challenge for IIT is to develop measures of ‘integrated information’ that can be usefully applied to actual data. These measures should capture, in empirically interesting and theoretically profound ways, the extent to which ‘a system generates more information than the sum of its parts’. Such measures are also of interest in many domains beyond consciousness, through for example to physics and engineering, where notions of ‘dynamical complexity’ are of more general importance.
Adam Barrett and I have been working towards this challenge for many years, both through approximations of the measure F (‘phi’, central to the various iterations of IIT) and through alternative measures like ‘causal density’. Alongside new work from other groups, there now exist a range of measures of integrated information – yet so far no systematic comparison of how they perform on non-trivial systems.
This is what we provide in our new paper, led by Adam along with Pedro Mediano from Imperial College London.
*
We describe, using a uniform notation, six different candidate measures of integrated information (among which we count the related measure of ‘causal density’). We set out the intuitions behind each, and compare their properties across a series of criteria. We then explore how they behave on a variety of network models, some very simple, others a little bit more complex.
The most striking finding is that the measures all behave very differently – no two measures show consistent agreement across all our analyses. Here’s an example:
Diverse behavior of measures of integrated information. The six measures (plus two control measures) are shown in terms of their behavior on a simple 2-node network animated by autoregressive dynamics.
At first glance this seems worrying for IIT since, ideally, one would want conceptually similar measures to behave in similar ways when applied to empirical test-cases. Indeed, it is worrying if existing measures are used uncritically. However, by rigorously comparing these measures we are able to identify those which better reflect the underlying intuitions of ‘integrated information’, which we believe will be of some help as these measures continue to be developed and refined.
Integrated information, along with related notions of dynamical complexity and emergence, are likely to be important pillars of our emerging understanding of complex dynamics in all sorts of situations – in consciousness research, in neuroscience more generally, and beyond biology altogether. Our new paper provides a firm foundation for the future development of this critical line of research.
*
One important caveat is necessary. We focus on measures that are, by construction, applicable to the empirical, or spontaneous, statistically stationary distribution of a system’s dynamics. This means we depart, by necessity, from the supposedly more fundamental measures of integrated information that feature in the most recent iterations of IIT. These recent versions of the theory appeal to the so-called ‘maximum entropy’ distribution since they are more interested in characterizing the ‘cause-effect structure’ of a system than in saying things about its dynamics. This means we should be very cautious about taking our results to apply to current versions of IIT. But, in recognizing this, we also return to where we started in this post. A major issue for the more recent (and supposedly more fundamental) versions of IIT is that they are extremely challenging to operationalize and therefore to put to an empirical test. Our work on integrated information departs from ‘fundamental’ IIT precisely because we prioritise empirical applicability. This, we think, is a feature, not a bug.
*
All credit for this study to Pedro Mediano and Adam Barrett, who did all the work. As always, I’d like to thank the Dr. Mortimer and Theresa Sackler Foundation, and the Canadian Institute for Advanced Research, Azrieli Programme in Brain, Mind, and Consciousness, for their support. The paper was published in Entropy on Christmas Day, which may explain why some of you might’ve missed it! But it did make the cover, which is nice.
*
Mediano, P.A.M., Seth, A.K., and Barrett, A.B. (2019). Measuring integrated information. Comparison of candidate measures in theory and in simulation. Entropy, 21:17
As a scientist, consciousness has always fascinated me. But understanding consciousness is not a project for science alone. Throughout history, philosophers, artists, storytellers, and musicians have all wondered about the apparent miracle of conscious awareness. Even today, while science might give us our best shot at figuring out the brain – the organ of experience – we need, more than ever, a melding of the arts and sciences, of contemporary and historical approaches, to understand what consciousness really is, to grasp what we mean by, as Mark Haddon eloquently puts it, “Life in the first person.”
This quote comes from Haddon’s beautiful introductory essay to a major new exhibition at the Wellcome Collection in London. Curated by Emily Sargent, States of Mind: Tracing the edges of consciousness “examines perspectives from artists, psychologists, philosophers and neuroscientists to interrogate our understanding of the conscious experience”. Its a fantastic exhibition, with style and substance, and I feel very fortunate to have been involved as an advisor from its early stages.
What’s so special about consciousness?
Consciousness is at once the most familiar and the most mysterious aspect of our existence. Conscious experiences define our lives, but the private, subjective, and what-it-is-likeness of these experiences seems to resist scientific enquiry. Somehow, within each our brains the combined activity of many billions of neurons, each one a tiny biological machine, is giving rise to a conscious experience. Your conscious experience: right here, right now, reading these words. How does this happen? Why is life in the first person?
In one sense, this seems like the kind of mystery ripe for explanation. Borrowing again from Mark Haddon, the raw material of consciousness is not squirreled away deep inside an atom, its not happening 14 billion years ago, and its not hiding out on the other side of the universe. It’s right here in front of – or rather behind – our eyes. Saying this, the brain is a remarkably complex object. It’s not so much the sheer number of neurons (though there about 90 billion). It’s the complexity of its wiring: there are so many connections, that if you counted one every second it would take you 3 million years to finish. Is it not possible that an object of such extraordinary complexity should be capable of extraordinary things?
People have been thinking about consciousness since they’ve been thinking at all. Hippocrates, the founder of modern medicine, said: “Men ought to know that from the brain, and from the brain only, arise our pleasures, joys, laughter and jests, as well as our sorrows, pains, griefs and tears … Madness comes from its moistness.” (Aristotle, by the way, got it wrong, thinking the brain hadn’t much to do with consciousness at all.)
Fast forward to Francis Crick, whose ‘astonishing hypothesis’ in the 1990s deliberately echoed Hippocrates: “You, your joys and your sorrows, your memories and your ambitions … and so on … are in fact no more than the behaviour of a vast assembly of nerve cells and their associated molecules”. Crick, who I was lucky enough to meet during my time in America, was working on the neurobiology of consciousness even on the day he died. You will see some of his personal notes, and his perplexing plasticine brain models, in States of Mind.
Descartes: view of posterior of brain, from De Hominem. Wellcome Collection
A major landmark in thinking about consciousness is of course Descartes, who in the 17th Century distinguished between “mind stuff” (res cogitans) and “matter stuff” (res extensa), so giving rise to the now infamous mind-body problem and the philosophy of dualism. Its a great thrill for to see an original copy of Descartes’ De Homine as part of this exhibition. Its modern incarnation as David Chalmers’ so-called ‘hard problem’ has recently gained enough cultural notoriety even to inspire a Tom Stoppard play (though for my money Alex Garland’s screenplay for Ex Machina is the more perspicuous). The idea of the hard problem is this: Even if we knew everything about how the operations of the brain give rise to perception, cognition, learning, and behaviour a problem would still remain: why and how any of this should be associated with consciousness at all? Why is life in the first person?
Defining consciousness
How to define consciousness? One simple definition is that for a conscious organism there is something it is like to be that organism. Or, one can simply say that consciousness is what disappears when we fall into a dreamless sleep, and what returns when we wake up or start dreaming. A bit more formally, for conscious organisms there exists a continuous (though interruptible) stream of conscious scenes – a phenomenal world – which has the character of being subjective and private. The material in States of Mind can help us encounter these ideas with a bit more clarity and force, by focusing on the edges – the liminal boundaries – of consciousness.
First there is conscious level: the difference between being awake and, let’s say, under general anaesthesia. Here, neuroscience now tells us that there is no single ‘generator’ of consciousness in the brain, rather, being consciousness depends on highly specific ways in which different parts of the brain speak to each other. Aya Ben Ron’s film of patients slipping away under anaesthesia is a beautiful exploration of this process, as is the whole section on ‘SLEEP | AWAKE’.
Then there is conscious content: what we are conscious of, when we are conscious. These are the perceptions, thoughts, and emotions that populate our ever-flowing stream of awareness. Here, current research is revealing that our perceptual world is not simply an internal picture of some external reality. Rather, conscious perception depends on the brain’s best guesses, or hypotheses, about the causes of sensory data. Perception is therefore a continuously creative act that is tightly bound up with imagination, so that our experience of the world is a kind of ‘controlled hallucination’, a fantasy that – usually, but not always – coincides with reality. The material on synaesthesia in States of Mind beautifully illuminates this process by showing how, for some of us, these perceptual fantasies can be very different – that we all have our own distinctive inner universes. You can even try training yourself to become a ‘synaesthete’ with a demo of some of our own research, developed for this exhibition. Many thanks to Dr. David Schwartzman of the Sackler Centre for making this happen.
Alphabet in Colour: Illustrating Vladimir Nabokov’s grapheme-colour synaesthesia, by Jean Holabird.
Finally there is conscious self – the specific experience of being me, or being you. While this might seem easy to take for granted, the experience of being a self requires explanation just as much as any other kind of experience. It too has its edges, its border regions. Here, research is revealing that conscious selfhood, though experienced as unified, can come apart in many different ways. For example, our experience of being and having a particular body can dissociate from our experience of being a person with name and a specific set of memories. Conscious selfhood, like all conscious perception, is therefore another controlled hallucination maintained by the brain. The section BEING | NOT BEING dramatically explores some of these issues, for example by looking at amnesia with Shona Illingworth, and with Adrian Owen’s seminal work on the possibility of consciousness even after severe brain injury.
This last example brings up an important point. Besides the allure of basic science, there are urgent practical motivations for studying consciousness. Neurological and psychiatric disorders are increasingly common and can often be understood as disturbances of conscious experience. Consciousness science promises new approaches and perhaps new treatments for these deeply destructive problems. Scoping out further boundary areas, studying the biology of consciousness can shed new light on awareness in newborn infants and in non-human animals, informing ethical debates in these areas. Above all, consciousness science carries the promise of understanding more about our place in nature. Following the tradition of Copernicus and Darwin, a biological account of conscious experience will help us see ourselves as part of, not apart from, the rest of the universe.
Santiago Ramon y Cajal, distinguishing the reticular theory (left) from the neuron doctrine (right). From the Instituto Cajal, Madrid.
Let’s finish by returning to this brilliant exhibition, States of Mind. What I found most remarkable are the objects that Emily Sargent has collected together. Whether its Descartes’ De Hominem, Ramon y Cajal’s delicate ink drawings of neurons, or Francis Crick’s notebooks and models, these objects bring home and render tangible the creativity and imagination which people have brought to bear in their struggle to understand consciousness, over hundreds of years. For me, this brings a new appreciation and wonder to our modern attempts to tackle this basic mystery of life. Emily Dickinson, my favourite poet of neuroscience, put it like this. “The brain is wider than the sky, for – put them side by side – the one the other will contain, with ease, and you – beside.”
States of Mind is at the Wellcome Collection in London from Feb 4th until October 16th 2016 and is curated by Emily Sargent. Sackler Centre researchers, in particular David Schwartzman and myself, helped out as scientific advisors. This text is lightly adapted from a speech I gave at the opening event on Feb 3rd 2016. Watch this space, and visit the exhibition website, for news about special events on consciousness that will happen throughout the year.
To help promote the new journal Neuroscience of Consciousness I took part in a short Q&A with Oxford University Press to ask the question: Can neuroscience explain consciousness? Read on …
Source: Can neuroscience explain consciousness? | OUPblog
This piece was also taken up at Reddit with some interesting comments.
YellowPinkBlue by Ann Veronica Janssens
From October 2015 until October 2016 the Wellcome Collection in London is curating an exhibition called States of Mind: Tracing the Edges of Consciousness. It has been launched with a brilliant piece of installation art by Ann Veronica Janssens (until 3rd Jan 2016). In YellowPinkBlue the entire gallery space is invaded by coloured mist, to focus attention on the process of perception itself so that one becomes subsumed by the experience of seeing. I’m excited to be contributing in various ways to States of Mind, via the Sackler Centre (more on that soon). To start with, here is the text I wrote for Janssen’s remarkable piece.
What in the world is consciousness?
Right now an apparent miracle is unfolding. Within your brain, the electrochemical activity of many billions of richly interconnected brain cells – each one a tiny biological machine – is giving rise to a conscious experience. Your conscious experience: right here, right now, reading these words.
It is all too easy to go about our daily lives, having conscious experiences, without appreciating how remarkable it is that we have these experiences at all. Ann Veronica Janssens’s piece returns us to the sheer wonder of being conscious. By stripping away many of the features that permeate our normal conscious lives, the raw fact of experiencing is given renewed emphasis.
People have wondered about consciousness since they’ve wondered about anything. Hippocrates, the Greek founder of modern medicine, rightly identified the brain as the organ of experience (though Aristotle didn’t agree). In the Renaissance, Descartes divided the universe into ‘mind stuff’ (res cogitans) and ‘matter stuff’ (res extensa), giving birth to the philosophy of dualism and the confounding ‘mind–body’ problem of how the two relate. In the 19th century, when psychology first emerged as a science, understanding consciousness was its primary objective. Though largely sidelined during the 20th century, the challenge of revealing the biological basis of consciousness is now firmly re-established for our times. Janssens’s piece reminds us of the important distinction in science between being conscious at all (conscious level: the difference between being awake and being in a dreamless sleep or under anaesthesia) and what we are conscious of (conscious content: the perceptions, thoughts and emotions that populate our conscious mind). There is also conscious selfhood – the specific experience of being me (or you). Each of these aspects of consciousness can be traced to specific mechanisms in the brain that neuroscientists, in cahoots with researchers from many other disciplines, are now starting to unravel. There are many exciting ideas in play, ranging from the dependence of conscious level on how different parts of the brain speak to each other, to understanding conscious content as determined by the brain’s ‘best guess’ of the causes of ambiguous and noisy sensory signals. Crucially, these ideas have allowed consciousness science to progress from the philosopher’s armchair to the research laboratory.
Besides the allure of basic science, there are important practical motivations for studying consciousness. Neurological and psychiatric disorders are increasingly common and can often be framed as disturbances of conscious experience. Consciousness science promises new approaches and perhaps new treatments for these scourges of modern society. New theories and experiments can also shed light on consciousness in newborns and in non-human animals, adding critical information to important ethical debates in these areas. But above all, consciousness science carries the promise of understanding more about our place in nature. Following Darwin and Copernicus, a biological account of conscious experience will help us see ourselves as part of, not apart from, the rest of the universe.
Anil Seth, Professor of Cognitive and Computational Neuroscience
Co-Director, Sackler Centre for Consciousness Science, University of Sussex
Alicia Vikander as Ava in Alex Garland’s Ex Machina
IT’S a rare thing to see a movie about science that takes no prisoners intellectually. Alex Garland’s Ex Machina is just that: a stylish, spare and cerebral psycho-techno-thriller, which gives a much-needed shot in the arm for smart science fiction.
Reclusive billionaire genius Nathan, played by Oscar Isaac, creates Ava, an intelligent and very attractive robot played by Alicia Vikander. He then struggles with the philosophical and ethical dilemmas his creation poses, while all hell breaks loose. Many twists and turns add nuance to the plot, which centres on the evolving relationships between the balletic Ava and Caleb (Domhnall Gleeson), a hotshot programmer invited by Nathan to be the “human component in a Turing test”, and between Caleb and Nathan, as Ava’s extraordinary capabilities become increasingly apparent
Everything about this movie is good. Compelling acting (with only three speaking parts), exquisite photography and set design, immaculate special effects, a subtle score and, above all, a hugely imaginative screenplay combine under Garland’s precise direction to deliver a cinematic experience that grabs you and never lets go.
The best science fiction often tackles the oldest questions. At the heart of Ex Machina is one of our toughest intellectual knots, that of artificial consciousness. Is it possible to build a machine that is not only intelligent but also sentient: that has consciousness, not only of the world but also of its own self? Can we construct a modern-day Golem, that lumpen being of Jewish folklore which is shaped from unformed matter and can both serve humankind and turn against it? And if we could, what would happen to us?
In Jewish folkore, the Golem is animate being shaped from unformed matter.
Putting aside the tedious business of actually building a conscious AI, we face the challenge of figuring out whether the attempt succeeds. The standard reference for this sort of question is Alan Turing’s eponymous test, in which a human judge interrogates both a candidate machine and another human. A machine passes the test when the judge consistently fails to distinguish between them.
While the Turing test has provided a trope for many AI-inspired movies (such as Spike Jonze’s excellent Her), Ex Machina takes things much further. In a sparkling exchange between Caleb and Nathan, Garland nails the weakness of Turing’s version of the test, a focus on the disembodied exchange of messages, and proposes something far more interesting. “The challenge is to show you that she’s a robot. And see if you still feel she has consciousness,” Nathan says to Caleb.
This shifts the goalposts in a vital way. What matters is not whether Ava is a machine. It is not even whether Ava, even though a machine, can be conscious. What matters is whether Ava makes a conscious person feel that Ava is conscious. The brilliance of Ex Machina is that it reveals the Turing test for what it really is: a test of the human, not of the machine. And Garland is not necessarily on our side.
Nathan (Oscar Isaac) and Caleb (Domnhall Gleeson) discuss deep matters of AI
Is consciousness a matter of social consensus? Is it more relevant whether people believe (or feel) that something (or someone) is conscious than whether it is in fact actually conscious? Or, does something being “actually conscious” rest on other people’s beliefs about it being conscious, or on its own beliefs about its consciousness (beliefs that may themselves depend on how it interprets others’ beliefs about it)? And exactly what is the difference between believing and feeling in situations like this?
It seems to me that my consciousness, here and now, is not a matter of social consensus or of my simply believing or feeling that I am conscious. It seems to me, simply, that I am conscious here and now. When I wake up and smell the coffee, there is a real experience of coffee-smelling going on.
But let me channel Ludwig Wittgenstein, one of the greatest philosophers of the 20th century, for a moment. What would it seem like if it seemed to me that my being conscious were a matter of social consensus or beliefs or feelings about my own conscious status? Is what it “seems like” to me relevant at all when deciding how consciousness comes about or what has consciousness?
Before vanishing completely into a philosophical rabbit hole, it is worth saying that questions like these are driving much influential current research on consciousness. Philosophers and scientists like Daniel Dennett, David Rosenthal and Michael Graziano defend, in various ways, the idea that consciousness is somehow illusory and what we really mean in saying we are conscious is that we have certain beliefs about mental states, that these states have distinctive functional properties, or that they are involved in specific sorts of attention.
Another theoretical approach accepts that conscious experience is real and sees the problem as one of determining its physical or biological mechanism. Some leading neuroscientists such as Giulio Tononi, and recently, Christof Koch, take consciousness to be a fundamental property, much like mass-energy and electrical charge, that is expressed through localised concentrations of “integrated information”. And others, like philosopher John Searle, believe that consciousness is an essentially biological property that emerges in some systems but not in others, for reasons as-yet unknown.
In the film we hear about Searle’s Chinese Room thought experiment. His premise was that researchers had managed to build a computer programmed in English that can respond to written Chinese with written Chinese so convincingly it easily passes the Turing test, persuading a human Chinese speaker that the program understands and speaks Chinese. Does the machine really “understand” Chinese (Searle called this “strong AI”) or is it only simulating the ability (“weak” AI)? There is also a nod to the notional “Mary”, the scientist, who, while knowing everything about the physics and biology of colour vision, has only ever experienced black, white and shades of grey. What happens when she sees a red object for the first time? Will she learn anything new? Does consciousness exceed the realms of knowledge.
All of the above illustrates how academically savvy and intellectually provocative Ex Machina is. Hat-tips here to Murray Shanahan, professor of cognitive robotics at Imperial College London, and writer and geneticist Adam Rutherford, whom Garland did well to enlist as science advisers.
Not every scene invites deep philosophy of mind, with the film encompassing everything from ethics, the technological singularity, Ghostbusters and social media to the erosion of privacy, feminism and sexual politics within its subtle scope. But when it comes to riffing on the possibilities and mysteries of brain, mind and consciousness, Ex Machina doesn’t miss a trick.
As a scientist, it is easy to moan when films don’t stack up against reality, but there is usually little to be gained from nitpicking over inaccuracies and narrative inventions. Such criticisms can seem petty and reinforcing of the stereotype of scientists as humourless gatekeepers of facts and hoarders of equations. But these complaints sometimes express a sense of missed opportunity rather than injustice, a sense that intellectual riches could have been exploited, not sidelined, in making a good movie. AI, neuroscience and consciousness are among the most vibrant and fascinating areas of contemporary science, and what we are discovering far outstrips anything that could be imagined out of thin air.
In his directorial debut, Garland has managed to capture the thrill of this adventure in a film that is effortlessly enthralling, whatever your background. This is why, on emerging from it, I felt lucky to be a neuroscientist. Here is a film that is a better film, because of and not despite its engagement with its intellectual inspiration.
The original version of this piece was published as a Culture Lab article in New Scientist on Jan 21. I am grateful to the New Scientist for permission to reproduce it here, and to Liz Else for help with editing. I will be discussing Ex Machina with Dr. Adam Rutherford at a special screening of the film at the Edinburgh Science Festival (April 16, details and tickets here).
Image courtesy of Phil Wheeler Illustrations
Can you learn to see the world differently? Some people already do. People with synaesthesia experience the world very differently indeed, in a way that seems linked to creativity, and which can shed light on some of the deepest mysteries of consciousness. In a paper published in Scientific Reports, we describe new evidence suggesting that non-synaesthetes can be trained to experience the world much like natural synaesthetes. Our results have important implications for understanding individual differences in conscious experiences, and they extend what we know about the flexibility (‘plasticity’) of perception.
Synaesthesia means that an experience of one kind (like seeing a letter) consistently and automatically evokes an experience of another kind (like seeing a colour), when the normal kind of sensory stimulation for the additional experience (the colour) isn’t there. This example describes grapheme-colour synaesthesia, but this is just one among many fascinating varieties. Other synaesthetes experience numbers as having particular spatial relationships (spatial form synaesthesia, probably the most common of all). And there are other more unusual varieties like mirror-touch synaesthesia, where people experience touch on their own bodies when they see someone else being touched, and taste-shape synaesthesia, where triangles might taste sharp, and ellipses bitter.
The richly associative nature of synaesthesia, and the biographies of famous case studies like Vladimir Nabokov and Wassily Kandinsky (or, as the Daily Wail preferred: Lady Gaga and Pharrell Williams), has fuelled its association with creativity and intelligence. Yet the condition is remarkably common, with recent estimates suggesting about 1 in 23 people have some form of synaesthesia. But how does it come about? Is it in your genes, or is it something you can learn?
It is widely believed that Kandinsky was synaesthetic. For instance he said: “Colour is the keyboard, the eyes are the harmonies, the soul is the piano with many strings. The artist is the hand that plays, touching one key or another, to cause vibrations in the soul”
As with most biological traits the truth is: a bit of both. But this still begs the question of whether being synaesthetic is something that can be learnt, even as an adult.
There is a rather long history of attempts to train people to be synaesthetic. Perhaps the earliest example was by E.L. Kelly who in 1934 published a paper with the title: An experimental attempt to produce artificial chromaesthesia by the technique of the conditioned response. While this attempt failed (the paper says it is “a report of purely negative experimental findings”) things have now moved on.
More recent attempts, for instance the excellent work of Olympia Colizoli and colleagues in Amsterdam, have tried to mimic (grapheme-colour) synaesthesia by having people read books in which some of the letters are always coloured in with particular colours. They found that it was possible to train people to display some of the characteristics of synaesthesia, like being slower to name coloured letters when they were presented in a colour conflicting with the training (the ‘synaesthetic Stroop’ effect). But crucially, until now no study has found that training could lead to people actually reporting synaesthesia-like conscious experiences.
An extract from the ‘coloured reading’ training material, used in our study, and similar to the material used by Colizoli and colleagues. The text is from James Joyce. Later in training we replaced some of the letters with (appropriately) coloured blocks to make the task even harder.
Our approach was based on brute force. We decided to dramatically increase the length and rigour of the training procedure that our (initially non-synaesthetic) volunteers undertook. Each of them (14 in all) came in to the lab for half-an-hour each day, five days a week, for nine weeks! On each visit they completed a selection of training exercises designed to cement specific associations between letters and colours. Crucially, we adapted the difficulty of the tasks to each volunteer and each training session, and we also gave them financial rewards for good performance. Over the nine-week regime, some of the easier tasks were dropped entirely, and other more difficult tasks were introduced. Our volunteers also had homework to do, like reading the coloured books. Our idea was that the training must always be challenging, in order to have a chance of working.
The results were striking. At the end of the nine-week exercise, our dedicated volunteers were tested for behavioural signs of synaesthesia, and – crucially – were also asked about their experiences, both inside and outside the lab. Behaviourally they all showed strong similarities with natural-born synaesthetes. This was most striking in measures of ‘consistency’, a test which requires repeated selection of the colour associated with a particular letter, from a palette of millions.
The consistency test for synaesthesia. This example from David Eagleman’s popular ‘synaesthesia battery’.
Natural-born synaesthetes show very high consistency: the colours they pick (for a given letter) are very close to each other in colour space, across repeated selections. This is important because consistency is very hard to fake. The idea is that synaesthetes can simply match a colour to their experienced ‘concurrent’, whereas non-synaesthetes have to rely on less reliable visual memory, or other strategies.
Our trained quasi-synaesthetes passed the consistency test with flying colours (so to speak). They also performed much like natural synaesthetes on a whole range of other behavioural tests, including synaesthetic stroop, and a ‘synaesthetic conditioning’ task which shows that trained colours can elicit automatic physiological responses, like increases in skin conductance. Most importantly, most (8/14) of our volunteers described colour experiences much like those of natural synaesthetes (only 2 reported no colour phenomenology at all). Strikingly, some of these experience took place even outside the lab:
“When I was walking into campus I glanced at the University of Sussex sign and the letters were coloured” [according to their trained associations]
Like natural synaesthetes, some of our volunteers seemed to experience the concurrent colour ‘out in the world’ while others experienced the colours more ‘in the head’:
“When I am looking at a letter I see them in the trained colours”
“When I look at the letter ‘p’ … its like the inside of my head is pink”
For grapheme colour synaesthetes, letters evoke specific colour experiences. Most of our trained quasi-synaesthetes reported similar experiences. This image is however quite misleading. Synaesthetes (natural born or not) also see the letters in their actual colour, and they typically know that the synaesthetic colour is not ‘real’. But that’s another story.
These results are very exciting, suggesting for the first time that with sufficient training, people can actually learn to see the world differently. Of course, since they are based on subjective reports about conscious experiences, they are also the hardest to independently verify. There is always the slight worry that our volunteers said what they thought we wanted to hear. Against this worry, we were careful to ensure that none of our volunteers knew the study was about synaesthesia (and on debrief, none of them did!). Also, similar ‘demand characteristic’ concerns could have affected other synaesthesia training studies, yet none of these led to descriptions of synaesthesia-like experiences.
Our results weren’t just about synaesthesia. A fascinating side effect was that our volunteers registered a dramatic increase in IQ, gaining an average of about 12 IQ points (compared to a control group which didn’t undergo training). We don’t yet know whether this increase was due to the specifically synaesthetic aspects of our regime, or just intensive cognitive training in general. Either way, our findings provide support for the idea that carefully designed cognitive training could enhance normal cognition, or even help remedy cognitive deficits or decline. More research is needed on these important questions.
What happened in the brain as a result of our training? The short answer is: we don’t know, yet. While in this study we didn’t look at the brain, other studies have found changes in the brain after similar kinds of training. This makes sense: changes in behaviour or in perception should be accompanied by neural changes of some kind. At the same time, natural-born synaesthetes appear to have differences both in the structure of their brains, and in their activity patterns. We are now eager to see what kind of neural signatures underlie the outcome of our training paradigm. The hope is, that because our study showed actual changes in perceptual experience, analysis of these signatures will shed new light on the brain basis of consciousness itself.
So, yes, you can learn to see the world differently. To me, the most important aspect of this work is that it emphasizes that each of us inhabits our own distinctive conscious world. It may be tempting to think that while different people – maybe other cultures – have different beliefs and ways of thinking, still we all see the same external reality. But synaesthesia, along with other emerging theories of ‘predictive processing’ – shows that the differences go much deeper. We each inhabit our own personalised universe, albeit one which is partly defined and shaped by other people. So next time you think someone is off in their own little world: they are.
The work described here was led by Daniel Bor and Nicolas Rothen, and is just one part of an energetic inquiry into synaesthesia taking place at Sussex University and the Sackler Centre for Consciousness Science. With Jamie Ward and (recently) Julia Simner also working here, we have a uniquely concentrated expertise in this fascinating area. In other related work I have been interested in why synaesthetic experiences lack a sense of reality and how this give an important clue about the nature of ‘perceptual presence’. I’ve also been working on the phenomenology of spatial form synaesthesia, and whether synaesthetic experiences can be induced through hypnosis. And an exciting brain imaging study of natural synaesthetes will shortly hit the press! Nicolas Rothen is an authority on the relationship between synaesthesia and memory, and Jamie Ward and Julia Simner have way too many accomplishments in this field to mention. (OK, Jamie has written the most influential review paper in the area – featuring a lot of his own work – and Julia (with Ed Hubbard) has written the leading textbook. That’s not bad to start with.)
Our paper, Adults can be Trained to Acquire Synesthetic Experiences (sorry for US spelling) is published (open access, free!) in Scientific Reports, part of the Nature family. The authors were Daniel Bor, Nicolas Rothen, David Schwartzman, Stephanie Clayton, and Anil K. Seth. There has been quite a lot of media coverage of this work, for instance in the New Scientist and the Daily Fail. Other coverage is summarized here.
This week I’d like to highlight my new book, 30 Second Brain, published by Icon Books on March 6th. It is widely available in both the UK and the USA. To whet your appetite here is a slightly amended version of the Introduction.
[New Scientist have just reviewed the book]
Understanding how the brain works is one of our greatest scientific quests. The challenge is quite different from other frontiers in science. Unlike the bizarre world of the very small in which quantum-mechanical particles can exist and not-exist at the same time, or the mind-boggling expanses of time and space conjured up in astronomy, the human brain is in one sense an everyday object: it is about the size and shape of a cauliflower, weighs about 1.5 kilograms, and has a texture like tofu. It is the complexity of the brain that makes it so remarkable and difficult to fathom. There are so many connections in the average adult human brain, that if you counted one each second, it would take you over 3 million years to finish.
Faced with such a daunting prospect it might seem as well to give up and do some gardening instead. But the brain cannot be ignored. As we live longer, more and more of us are suffering – or will suffer – from neurodegenerative conditions like Alzheimer’s disease and dementia, and the incidence of psychiatric illnesses like depression and schizophrenia is also on the rise. Better treatments for these conditions depend on a better understanding of the brain’s intricate networks.
More fundamentally, the brain draws us in because the brain defines who we are. It is much more than just a machine to think with. Hippocrates, the father of western medicine, recognized this long ago: “Men ought to know that from nothing else but the brain come joys, delights, laughter and jests, and sorrows, griefs, despondency, and lamentations.” Much more recently Francis Crick – one of the major biologists of our time – echoed the same idea: “You, your joys and your sorrows, your memories and your ambitions, your sense of personal identity and free will, are in fact no more than the behaviour of a vast assembly of nerve cells and their associated molecules”. And, perhaps less controversially but just as important, the brain is also responsible for the way we perceive the world and how we behave within it. So to understand the operation of the brain is to understand our own selves and our place in society and in nature, and by doing so to follow in the hallowed footsteps of giants like Copernicus and Darwin.
But how to begin? From humble beginnings, neuroscience is now a vast enterprise involving scientists from many different disciplines and almost every country in the world. The annual meeting of the ‘Society for Neuroscience’ attracts more than twenty thousand (and sometime more than thirty thousand!) brain scientists each year, all intent on talking about their own specific discoveries and finding out what’s new. No single person – however capacious their brain – could possible keep track of such an enormous and fast-moving field. Fortunately, as in any area of science, underlying all this complexity are some key ideas to help us get by. Here’s where this book can help.
Within the pages of this book, leading neuroscientists will take you on a tour of fifty of the most exciting ideas in modern brain science, using simple plain English. To start with, in ‘Building the brain’ we will learn about the basic components and design of the brain, and trace its history from birth (and before!), and over evolution. ‘Brainy theories’ will introduce some of the most promising ideas about how the brain’s many billions of nerve cells (neurons) might work together. The next chapter will show how new technologies are providing astonishing advances in our ability to map the brain and decipher its activity in time and space. Then in ‘Consciousness’ we tackle the big question raised by Hippocrates and Crick, namely the still-mysterious relation between the brain and conscious experience – how does the buzzing of neurons transform into the subjective experience of being you, here, now, reading these words? Although the brain basis of consciousness happens to be my own particular research interest, much of the brain’s work is done below its radar – think of the delicate orchestration of muscles involved in picking up a cup, or in walking across the room. So in the next chapter we will explore how the brain enables perception, action, cognition, and emotion, both with and without consciousness. Finally, nothing – of course – ever stays the same. In the last chapter – ‘the changing brain –we will explore some very recent ideas about how the brain changes its structure and function throughout life, in both health and in disease.
Each of the 50 ideas is condensed into a concise, accessible and engaging ’30 second neuroscience’. To get the main message across there is also a ‘3 second brainwave’, and a ‘3 minute brainstorm’ provides some extra food for thought on each topic. There are helpful glossaries summarizing the most important terms used in each chapter, as well as biographies of key scientists who helped make neuroscience what it is today. Above all, I hope to convey that the science of the brain is just getting into its stride. These are exciting times and it’s time to put the old grey matter through its paces.
Update 29.04.14. Foreign editions now arriving!
Since this is a new blog, forgive a bit of a catch up. This is about a recent Trends Cognitive Sciences opinion article I wrote, applying the framework of predictive processing/coding to interoception, emotion, and the experience of body ownership. There’s a lot of interest at the moment in understanding how interoception (the sense of the internal state of the body) and exteroception (everything else) interact. Hopefully this will contribute in some way. The full paper is here.
ABSTRACT: The concept of the brain as a prediction machine has enjoyed a resurgence in the context of the Bayesian brain and predictive coding approaches within cognitive science. To date, this perspective has been applied primarily to exteroceptive perception (e.g., vision, audition), and action. Here, I describe a predictive, inferential perspective on interoception: ‘interoceptive inference’ conceives of subjective feeling states (emotions) as arising from actively-inferred generative (predictive) models of the causes of interoceptive afferents. The model generalizes ‘appraisal’ theories that view emotions as emerging from cognitive evaluations of physiological changes, and it sheds new light on the neurocognitive mechanisms that underlie the experience of body ownership and conscious selfhood in health and in neuropsychiatric illness.
As always, a pre-copy-edited version is here.
I have a new ‘Discussion’ paper just out in the journal Cognitive Neuroscience. Right now there is just the target paper – eventually it will appear with published commentaries and my response. The basic idea is to bring together, in a formal theoretical framework, ideas from Bayesian predictive processing and ‘enactive’ sensorimotor theory. The new theory explains ‘perceptual presence’ in terms of the counterfactual richness of predictive representations, and it can also explain the absence of such presence in important cases like synaesthesia.
(A pre-copy-edit version can be obtained here)
ABSTRACT: Normal perception involves experiencing objects within perceptual scenes as real, as existing in the world. This property of “perceptual presence” has motivated “sensorimotor theories” which understand perception to involve the mastery of sensorimotor contingencies. However, the mechanistic basis of sensorimotor contingencies and their mastery has remained unclear. Sensorimotor theory also struggles to explain instances of perception, such as synaesthesia, that appear to lack perceptual presence and for which relevant sensorimotor contingencies are difficult to identify. On alternative “predictive processing” theories, perceptual content emerges from probabilistic inference on the external causes of sensory signals, however this view has addressed neither the problem of perceptual presence nor synaesthesia. Here, I describe a theory of predictive perception of sensorimotor contingencies which (i) accounts for perceptual presence in normal perception, as well as its absence in synaesthesia, and (ii) operationalizes the notion of sensorimotor contingencies and their mastery. The core idea is that generative models underlying perception incorporate explicitly counterfactual elements related to how sensory inputs would change on the basis of a broad repertoire of possible actions, even if those actions are not performed. These “counterfactually-rich” generative models encode sensorimotor contingencies related to repertoires of sensorimotor dependencies, with counterfactual richness determining the degree of perceptual presence associated with a stimulus. While the generative models underlying normal perception are typically counterfactually rich (reflecting a large repertoire of possible sensorimotor dependencies), those underlying synaesthetic concurrents are hypothesized to be counterfactually poor. In addition to accounting for the phenomenology of synaesthesia, the theory naturally accommodates phenomenological differences between a range of experiential states including dreaming, hallucination, and the like. It may also lead to a new view of the (in)determinacy of normal perception.
NeuroBanter 