Tag Archives: publication

Time perception without clocks

Posted on by

0

the_persistence_of_memory

Salvador Dali, The Persistence of Memory, 1931

Our new paper, led by Warrick Roseboom, is out now (open access) in Nature Communications. It’s about time.

More than two thousand years ago, though who knows how long exactly, Saint Augustine complained “What then is time? If no-one asks me, I know; if I wish to explain to one who asks, I know not.”

The nature of time is endlessly mysterious, in philosophy, in physics, and also in neuroscience. We experience the flow of time, we perceive events as being ordered in time and as having particular durations, yet there are no time sensors in the brain. The eye has rod and cone cells to detect light, the ear has hair cells to detect sound, but there are no dedicated ‘time receptors’ to be found anywhere. How, then, does the brain create the subjective sense of time passing?

Most neuroscientific models of time perception rely on some kind internal timekeeper or pacemaker, a putative ‘clock in the head’ against which the flow of events can be measured. But despite considerable research, clear evidence for these neuronal pacemakers has been rather lacking, especially when it comes to psychologically relevant timescales of a few seconds to minutes.

An alternative view, and one with substantial psychological pedigree, is that time perception is driven by changes in other perceptual modalities. These modalities include vision and hearing, and possibly also internal modalities like interoception (the sense of the body ‘from within’). This is the view we set out to test in this new study, initiated by Warrick Roseboom here at the Sackler Centre, and Dave Bhowmik at Imperial College London, as part of the recently finished EU H2020 project TIMESTORM.

*

Their idea was that one specific aspect of time perception – duration estimation – is based on the rate of accumulation of salient events in other perceptual modalities. More salient changes, longer estimated durations. Fewer salient changes, shorter durations. He set out to test this idea using a neural network model of visual object classification modified to generate estimates of salient changes when exposed to natural videos of varying lengths (Figure 1).

timefig1

Figure 1. Experiment design. Both human volunteers (a, with eye tracking) and a pretrained object classification neural network (b) view a series of natural videos of different lengths (c), recorded in different environments (d). Activity in the classification networks is analysed for frame-to-frame ‘salient changes’ and records of salient changes are used to train estimates of duration – based on the physical duration of the video. These estimates are then compared with human reports. We also compare networks trained on gaze-constrained video input versus ‘full frame’ video input.

We first collected several hundred videos of five different environments and chopped them into varying lengths from 1 sec to ~1 min. The environments were quiet office scenes, café scenes, busy city scenes, outdoor countryside scenes, and scenes from the campus of Sussex University.  We then showed the videos to some human participants, who rated their apparent durations. We also collected eye tracking data while they viewed the videos. All in all we obtained over 4,000 duration ratings.

The behavioural data showed that people could do the task, and that – as expected – they underestimated long durations and overestimated short durations (Figure 2a). This ‘regression to the mean’ effect is known as Vierodt’s law in the time perception literature and is very well known. Our human volunteers also showed biases according to the video content, rating busy (e.g., city) scenes as lasting longer than non-busy (e.g., office) scenes of the same physical duration. This is just as expected, if duration estimation is based on accumulation of salient perceptual changes.

For the computational part, we used AlexNet, a pretrained deep convolutional neural network (DCNN) which has excellent object classification performance across 1,000 classes of object. We exposed AlexNet to each video, frame by frame. For each frame we examined activity in four separate layers of the network and compared it to the activity elicited by the previous frame. If the difference exceeded an adaptive threshold, we counted a ‘salient event’ and accumulated a unit of subjective time at that level. Finally, we used a simple machine learning tool (a support vector machine) to convert the record of salient events into an estimate of duration in seconds, in order to compare the model with human reports.  There are two important things to note here. The first is that the system was trained on the physical duration of the videos, not on the human estimates (apparent durations). The second is that there is no reliance on any internal clock or pacemaker at all (the frame rate is arbitrary – changing it doesn’t make any difference).

timefig2

Fig 2. Main results. Human volunteers can do the task and show characteristic biases (a).  When the model is trained on ‘full-frame’ data it can also do the task, but the biases are even more severe (b). There is a much closer match to human data when the model input is constrained by human gaze data (c), but not when the gaze locations are drawn from different trials (d).

There were two key tests of the model.  Was it able to perform the task?  More importantly, did it reveal the same pattern of biases as shown by humans?

Figure 2(b) shows that the model indeed performed the task, classifying longer videos as longer than shorter videos.  It also showed the same pattern of biases, though these were more exaggerated than for the human data (a).  But – critically – when we constrained the video input to the model by where humans were looking, the match to human performance was incredibly close (c). (Importantly, this match went away if we used gaze locations from a different video, d). We also found that the model displayed a similar pattern of biases by content, rating busy scenes as lasting longer than non-busy scenes – just as our human volunteers did. Additional control experiments, described in the paper, rule out that these close matches could be achieved just by changes within the video image itself, or by other trivial dependencies (e.g., on frame rate, or on the support vector regression step).

Altogether, these data show that our clock-free model of time-perception, based on the dynamics of perceptual classification, provides a sufficient basis for capturing subjective duration estimation of visual scenes – scenes that vary in their content as well as in their duration. Our model works on a fully end-to-end basis, going all the way from natural video stimuli to duration estimation in seconds.

*

We think this work is important because it comprehensively illustrates an empirically adequate alternative to ‘pacemaker’ models of time perception.

Pacemaker models are undoubtedly intuitive and influential, but they raise the spectre of what Daniel Dennett has called the ‘fallacy of double transduction’. This is false idea that perceptual systems somehow need to re-instantiate a perceived property inside the head, in order for perception to work. Thus perceived redness might require something red-in-the-head, and perceived music might need a little band-in-the-head, together with a complicated system of intracranial microphones. Naturally no-one would explicitly sign up to this kind of theory, but it sometimes creeps in unannounced to theories that rely too heavily on representations of one kind or another. And it seems that proposing a ‘clock in the head’ for time perception provides a prime example of an implicit double transduction. Our model neatly avoids the fallacy, and as we say in our Conclusion:

“That our system produces human-like time estimates based on only natural video inputs, without any appeal to a pacemaker or clock-like mechanism, represents a substantial advance in building artificial systems with human-like temporal cognition, and presents a fresh opportunity to understand human perception and experience of time.” (p.7).

We’re now extending this line of work by obtaining neuroimaging (fMRI) data during the same task, so that we can compare the computational model activity against brain activity in human observers (with Maxine Sherman). We’ve also recorded a whole array of physiological signatures – such as heart-rate and eye-blink data – to see whether we can find any reliable physiological influences on duration estimation in this task.  We can’t – and the preprint, with Marta Suarez-Pinilla – is here.

*

Major credit for this study to Warrick Roseboom who led the whole thing, with the able assistance of Zaferious Fountas and Kyriacos Nikiforou with the modelling. Major credit also to David Bhowmik who was heavily involved in the conception and early stages of the project, and also to Murray Shanahan who provided very helpful oversight. Thanks also to the EU H2020 TIMESTORM project which supported this project from start to finish. As always, I’d also like to thank the Dr. Mortimer and Theresa Sackler Foundation, and the Canadian Institute for Advanced Research, Azrieli Programme in Brain, Mind, and Consciousness, for their support.

*

Roseboom, W., Fountas, Z., Nikiforou, K., Bhowmik, D., Shanahan, M.P., and Seth, A.K. (2019). Activity in perceptual classification networks as a basis for human subjective time perception. Nature Communications. 10:269.

 

How does the brain fit into the skull?

Posted on by

0

Image

Announcing a new paper co-authored with David Samu and Thomas Nowotny, published yesterday in the open-access journal PLoS Computational Biology.

Influence of Wiring Cost on the Large-Scale Architecture of Human Cortical Connectivity

Macroscopic regions in the grey matter of the human brain are intricately connected by white-matter pathways, forming the extremely complex network of the brain. Analysing this brain network may provide us insights on how anatomy enables brain function and, ultimately, cognition and consciousness. Various important principles of organization have indeed been consistently identified in the brain’s structural connectivity, such as a small-world and modular architecture. However, it is currently unclear which of these principles are functionally relevant, and which are merely the consequence of more basic constraints of the brain, such as its three-dimensional spatial embedding into the limited volume of the skull or the high metabolic cost of long-range connections. In this paper, we model what aspects of the structural organization of the brain are affected by its wiring constraints by assessing how far these aspects are preserved in brain-like networks with varying spatial wiring constraints. We find that all investigated features of brain organization also appear in spatially constrained networks, but we also discover that several of the features are more pronounced in the brain than its wiring constraints alone would necessitate. These findings suggest the functional relevance of the ‘over-expressed’ properties of brain architecture.

New: Image from this paper featured as MRC biomedical image of the day on April 29th 2014!

brainteaser

 

The 30 Second Brain

Posted on by

0

Image

This week I’d like to highlight my new book, 30 Second Brain,  published by Icon Books on March 6th.  It is widely available in both the UK and the USA.  To whet your appetite here is a slightly amended version of the Introduction.

[New Scientist have just reviewed the book]

Understanding how the brain works is one of our greatest scientific quests.  The challenge is quite different from other frontiers in science.  Unlike the bizarre world of the very small in which quantum-mechanical particles can exist and not-exist at the same time, or the mind-boggling expanses of time and space conjured up in astronomy, the human brain is in one sense an everyday object: it is about the size and shape of a cauliflower, weighs about 1.5 kilograms, and has a texture like tofu.  It is the complexity of the brain that makes it so remarkable and difficult to fathom.  There are so many connections in the average adult human brain, that if you counted one each second, it would take you over 3 million years to finish.

Faced with such a daunting prospect it might seem as well to give up and do some gardening instead.  But the brain cannot be ignored.  As we live longer, more and more of us are suffering  – or will suffer – from neurodegenerative conditions like Alzheimer’s disease and dementia, and the incidence of psychiatric illnesses like depression and schizophrenia is also on the rise. Better treatments for these conditions depend on a better understanding of the brain’s intricate networks.

More fundamentally, the brain draws us in because the brain defines who we are.  It is much more than just a machine to think with. Hippocrates, the father of western medicine, recognized this long ago:  “Men ought to know that from nothing else but the brain come joys, delights, laughter and jests, and sorrows, griefs, despondency, and lamentations.” Much more recently Francis Crick – one of the major biologists of our time  – echoed the same idea: “You, your joys and your sorrows, your memories and your ambitions, your sense of personal identity and free will, are in fact no more than the behaviour of a vast assembly of nerve cells and their associated molecules”.  And, perhaps less controversially but just as important, the brain is also responsible for the way we perceive the world and how we behave within it. So to understand the operation of the brain is to understand our own selves and our place in society and in nature, and by doing so to follow in the hallowed footsteps of giants like Copernicus and Darwin.

But how to begin?  From humble beginnings, neuroscience is now a vast enterprise involving scientists from many different disciplines and almost every country in the world.  The annual meeting of the ‘Society for Neuroscience’ attracts more than twenty thousand (and sometime more than thirty thousand!) brain scientists each year, all intent on talking about their own specific discoveries and finding out what’s new.  No single person – however capacious their brain – could possible keep track of such an enormous and fast-moving field.  Fortunately, as in any area of science, underlying all this complexity are some key ideas to help us get by.  Here’s where this book can help.

Within the pages of this book, leading neuroscientists will take you on a tour of fifty of the most exciting ideas in modern brain science, using simple plain English.  To start with, in ‘Building the brain’ we will learn about the basic components and design of the brain, and trace its history from birth (and before!), and over evolution.  ‘Brainy theories’ will introduce some of the most promising ideas about how the brain’s many billions of nerve cells (neurons) might work together.  The next chapter will show how new technologies are providing astonishing advances in our ability to map the brain and decipher its activity in time and space.  Then in ‘Consciousness’ we tackle the big question raised by Hippocrates and Crick, namely the still-mysterious relation between the brain and conscious experience – how does the buzzing of neurons transform into the subjective experience of being you, here, now, reading these words? Although the brain basis of consciousness happens to be my own particular research interest, much of the brain’s work is done below its radar – think of the delicate orchestration of muscles involved in picking up a cup, or in walking across the room.  So in the next chapter we will explore how the brain enables perception, action, cognition, and emotion, both with and without consciousness.  Finally, nothing – of course – ever stays the same. In the last chapter – ‘the changing brain –we will explore some very recent ideas about how the brain changes its structure and function throughout life, in both health and in disease.

Each of the 50 ideas is condensed into a concise, accessible and engaging ’30 second neuroscience’.  To get the main message across there is also a ‘3 second brainwave’, and a ‘3 minute brainstorm’ provides some extra food for thought on each topic. There are helpful glossaries summarizing the most important terms used in each chapter, as well as biographies of key scientists who helped make neuroscience what it is today.  Above all, I hope to convey that the science of the brain is just getting into its stride. These are exciting times and it’s time to put the old grey matter through its paces.

Update 29.04.14.  Foreign editions now arriving!

30SecBrainMontage

Interoceptive inference, emotion, and the embodied self

Posted on by

0

ImageSince this is a new blog, forgive a bit of a catch up.  This is about a recent Trends Cognitive Sciences opinion article I wrote, applying the framework of predictive processing/coding to interoception, emotion, and the experience of body ownership.  There’s a lot of interest at the moment in understanding how interoception (the sense of the internal state of the body) and exteroception (everything else) interact.  Hopefully this will contribute in some way.  The full paper is here.

Interoceptive inference, emotion, and the embodied self

ABSTRACT:  The concept of the brain as a prediction machine has enjoyed a resurgence in the context of the Bayesian brain and predictive coding approaches within cognitive science. To date, this perspective has been applied primarily to exteroceptive perception (e.g., vision, audition), and action. Here, I describe a predictive, inferential perspective on interoception: ‘interoceptive inference’ conceives of subjective feeling states (emotions) as arising from actively-inferred generative (predictive) models of the causes of interoceptive afferents. The model generalizes ‘appraisal’ theories that view emotions as emerging from cognitive evaluations of physiological changes, and it sheds new light on the neurocognitive mechanisms that underlie the experience of body ownership and conscious selfhood in health and in neuropsychiatric illness.

As always, a pre-copy-edited version is here.

Predictive processing, sensorimotor theory, and perceptual presence

Posted on by

0

Seth_Fig3

I have a new ‘Discussion’ paper just out in the journal Cognitive Neuroscience. Right now there is just the target paper – eventually it will appear with published commentaries and my response.  The basic idea is to bring together, in a formal theoretical framework, ideas from Bayesian predictive processing and ‘enactive’ sensorimotor theory.  The new theory explains ‘perceptual presence’ in terms of the counterfactual richness of predictive representations, and it can also explain the absence of such presence in important cases like synaesthesia.

A predictive processing theory of sensorimotor contingencies: Explaining the puzzle of perceptual presence and its absence in synaesthesia

(A pre-copy-edit version can be obtained here)

ABSTRACT: Normal perception involves experiencing objects within perceptual scenes as real, as existing in the world. This property of “perceptual presence” has motivated “sensorimotor theories” which understand perception to involve the mastery of sensorimotor contingencies. However, the mechanistic basis of sensorimotor contingencies and their mastery has remained unclear. Sensorimotor theory also struggles to explain instances of perception, such as synaesthesia, that appear to lack perceptual presence and for which relevant sensorimotor contingencies are difficult to identify. On alternative “predictive processing” theories, perceptual content emerges from probabilistic inference on the external causes of sensory signals, however this view has addressed neither the problem of perceptual presence nor synaesthesia. Here, I describe a theory of predictive perception of sensorimotor contingencies which (i) accounts for perceptual presence in normal perception, as well as its absence in synaesthesia, and (ii) operationalizes the notion of sensorimotor contingencies and their mastery. The core idea is that generative models underlying perception incorporate explicitly counterfactual elements related to how sensory inputs would change on the basis of a broad repertoire of possible actions, even if those actions are not performed. These “counterfactually-rich” generative models encode sensorimotor contingencies related to repertoires of sensorimotor dependencies, with counterfactual richness determining the degree of perceptual presence associated with a stimulus. While the generative models underlying normal perception are typically counterfactually rich (reflecting a large repertoire of possible sensorimotor dependencies), those underlying synaesthetic concurrents are hypothesized to be counterfactually poor. In addition to accounting for the phenomenology of synaesthesia, the theory naturally accommodates phenomenological differences between a range of experiential states including dreaming, hallucination, and the like. It may also lead to a new view of the (in)determinacy of normal perception.