TRF1 Organizer Q&A – Anne Giersch

Anne Giersch studied medicine and specialized in psychiatry before doing a PhD in Cognitive Neuroscience, with a training in Psychophysics and Experimental Psychology in the field of visual grouping. After a post-doctoral stay in Germany, she was hired by the French Medical Research Institute (INSERM) as a full time researcher. She directs a team in Strasbourg in France (INSERM U1114) recognized for its expertise in the exploration of cognitive disorders in schizophrenia. Anne Giersch has worked on cognition, psychopharmacology and schizophrenia for several years, with 70 papers in international journals. She has developed a specific focus on time issues, to uncover the mechanisms of cognitive deficits in schizophrenia and their relationship with neurobiological disorders and clinical symptoms. She claims that the thought fragmentation described in patients may reveal critical mechanisms of disorders affecting the sense of self in patients, but also critical temporal dynamics of our unconscious and conscious mental activity.  

 

How can we determine the brain’s code for time?

And how do we go from the brain code to the experience of time? Duration? Order? Asynchrony? Thing?

 

What aspect of timing does your lab investigate, and what do you consider to be the most pressing and fundamental questions in timing research?

My lab is investigating the pathophysiology of schizophrenia. Those patients have been described as suffering from a disruption of the sense of time continuity, which we can only imagine to be a frightful, unspeakable, experience. The question of the sense of time continuity is so old it might not be considered as a pressing question. However, if its disruption explains some of the terror experienced by the patients; if it leads them to stop from feeling as one unique continuous being over time, then it becomes an emergency. But still one question among other pressing questions.

 

As the Organizers, what are your hopes and expectations for the 1st Timing Research Forum Conference?

The conference brings together researchers coming to timing from different perspectives. This has always been fruitful in research, and my hope for this conference and the following ones is that this timing research will remain open, or even open up more to different approaches and backgrounds, attracting researchers from different fields in a flexible way.

 

What current topics/techniques or new advances in timing research are you most excited about?

I came to timing only after studying psychiatry, and then visual organization under the effect of drugs or pathology. I am now totally absorbed in timing research. I see the whole topic as an ideal way to understand what consciousness is and where our conscious experience comes from, both its content and its container, or structure.

 

What advice do you have for students and postdoctoral researchers interesting in investigating the brain’s code for time?

I would say come! Work and read. As much as you can, in your field and outside your field. Philosophy, neuroimaging, psychology, molecular biology, beyond if you can. And don’t forget to stop, think, and let your mind wander.

Jenny Coull: TRF1 Organizer Q&A

 

Jenny Coull is a CNRS Senior Research Fellow and has been at Aix-Marseille University in France for 15 years. Prior to that she spent 7 years in London at the Functional Imaging Laboratory of UCL.  She conducts functional imaging, psychopharmacological and developmental investigations of timing and temporal attention. Her lab website is – https://sites.google.com/site/jennifercoull/

 

How can we determine the brain’s code for time?

Slowly!

 

What aspect of timing does your lab investigate, and what do you consider to be the most pressing and fundamental questions in timing research?

My research is focused on duration – how we measure current time, and how we can use it to predict future time. I primarily use fMRI but have recently been collaborating on some developmental research, which I’m very excited about.  I like how such different methodologies can be mutually informative.

I think the most fundamental question for timing research is how we represent such a slippery concept in our brains. Time is relative so it can’t possibly exist in a single location of the brain and it must depend, to a certain extent, on memory. Time is intangible so it must need some kind of scaffolding upon which to support itself – a motor representation of time, a spatial representation of time…?

 

As the Organizers, what are your hopes and expectations for 1st Timing Research Forum Conference?

Although the psychological study of time has been around for decades, the neuroscientific investigation of time is relatively recent. Of course, this is largely because of amazing technical progress in the past 25 years or so. Because there is no clinical disorder whose symptoms are characterised by temporal dysfunction, the neuroscientific study of time wasn’t really possible until such technological advances had been made. So my big hope for the 1st TRF conference is that psychologists and neuroscientists get together to learn from, and inspire, one another.

 

What current topics/techniques or new advances in timing research are you most excited about?

The therapeutic possibilities of temporally structuring a patient’s experience to help them overcome the clinical symptoms of their disorder.  For example, the incredible effects of rhythmic auditory stimulation on the motor symptoms of Parkinson’s Disease. Or  finding a way to help schizophrenic patients untangle the temporal order of their experience, which might go some way to temper some of the positive symptoms of the disorder (hallucinations and delusions).

 

What advice do you have for students and postdoctoral researchers interesting in investigating the brain’s code for time?

The neuroscientific investigation of time is a young field with enormous scope for new lines of inquiry. So it’s critical to keep up to date with the overwhelming number of new papers coming out every month (and to keep their supervisors up to date at the same time!!). I would also encourage young neuroscientists to learn from the past and explore the classic psychology literature from the ’50s and ’60s (and before).

Warren Meck: TRF1 Speaker Q&A

 

Warren Meck obtained a B.A. degree in psychology from the University of California, San Diego, a Ph.D. in experimental psychology from Brown University, and has been a professor at Brown University, Columbia University, and now Duke University.

His publications are accessible at Google Scholar and can be downloaded at ResearchGate, which also hosts preprints and descriptions of current research projects.

 

How can we determine the brain’s code for time?

It will take well-designed psychophysical studies in combination with neuroimaging, optogenetic stimulation, and electrophysiological recording techniques (triangulation) to break the code. Evaluating subjects with selective lesions and/or genetic backgrounds will continue to be important as well.

 

What will your talk at the 1st Timing Research Forum Conference focus on?

My talk will focus on the pervasiveness of timing abilities across animal species and the idea that a common timing mechanism is used that co-evolved with motor systems, i.e., to move is to time.

 

What according to you are the most pressing and fundamental questions in timing research?

Goals:

a) To map out the “temporal connectome” for time, whereby central timing mechanisms can monitor and synchronize satellite timing mechanisms.

b) To better understand the relationship between intelligence/working memory capacity and timing accuracy/precision.

 

What current topics/techniques or new advances in timing research are you most excited about?

Optogenetics, i.e., selective stimulation of specific types of neurons and/or pathways thought to be involved in controlling the speed of the “internal clock” as well as its mode of operation (e.g., run, pause, and reset).

 

What advice do you have for students and postdoctoral researchers interested in investigating the brain’s code for time?

I would first recommend that students keep in mind the inspiration provided by Robert Rousseau (Laval University) in his forward to the book Functional and Neural Mechanisms of Interval Timing (CRC Press, 2003).

“For more than a century, time has been an object of study in experimental psychology. In his Experimental Psychology, Titchener (1905) wrote, “A student who knows his time sense … has a good idea of what experimental psychology has been and of what it has come to be.” At the dawn of the 21st century, I believe that Titchener’s judgment about the status of timing and time perception in psychology is still appropriate. As was the case a century ago, knowledge of the current research on timing gives a sense of what cognition, cognitive psychology, and cognitive neuroscience have come to be and will become.”

I would also advise students to learn as much as they can about the different levels of analysis that can be applied in the study of timing and time perception in humans and other animals. For me, this would involve comparative neuroanatomy, electrophysiology, and computational modeling.

Dean Buonomano: TRF1 Speaker Q&A

Dean Buonomano is Professor at the Departments of Neurobiology and Psychology, University of California Los Angeles. At the 1st TRF Conference, he is the Organizer of a symposium on ‘Timing, Neural Dynamics, and Temporal Scaling‘. He regularly tweets about time at @deanbuono.

 

How can we determine the brain’s code for time?

I don’t think there will be a single code for time any more than there is a single code for space in the brain. I think there be will be a number of ways the brain represents and tracks time, depending on the time scale and task at hand. Timing is simply to integral to the brain’s fundamental computations to rely on a single strategy. We have increasingly compelling evidence that in some cases temporal information is encoded in dynamically changing neural activity patterns (population clocks) or ramping of firing rates. The challenge will be to understand the mechanisms by which these codes are generated, and the domain in which different coding and timing strategies are relevant (a problem related to the Taxonomy of Time, see #3 below).

 

What will your talk at the 1st Timing Research Forum Conference focus on?

A striking ability we have at the both the sensory and motor level is to recognize and generate temporal patterns at different speeds—such as the tempo of music or the speed of speech. Along with Hugo Merchant and Mehrdad Jazayeri my talk will focus on the problem of temporal scaling: the ability to produce simple or complex temporal motor patterns at different speeds.

 

What according to you are the most pressing and fundamental questions in timing research?

I think the most pressing question in the timing field may be defining what exactly we mean by the timing field. Specifically, there is an increasing recognition that we need a Taxonomy of Time. A taxonomy of memory (e.g., Procedural x Declarative) was in many ways one of the most important advances in the study of learning and memory in the 20th century. The timing field is severely hampered by our inability to define and pinpoint the different forms, and time scales, of timing and temporal processing.

 

What current topics/techniques or new advances in timing research are you most excited about?

To date most studies have primarily focused on the activity or contribution of a given brain area in a timing task. But the brain is one big “which came first the chicken or the egg” problem when it comes to cause and effect. So I’m excited about improvements in our ability to record from hundreds of neurons in multiple different brain areas simultaneously. I think focusing on the transformations that happen between areas and the differences in representations will provide a powerful tool to understand timing and temporal processing.

 

What advice do you have for students and postdoctoral researchers interesting in investigating the brain’s code for time?

Read, and try to seek out opportunities to write reviews and perspectives.

Dopamine encodes retrospective temporal information

A new study published in Cell Reports shows that midbrain dopamine neurons are sensitive to previously experienced time intervals, and that this is likely to be important in terms of reward processing. Midbrain dopamine neurons are frequently discussed in terms of their roles in reward, motivation, and certain forms of learning. However, within the time perception literature, we commonly associate dopamine as modulating the rate of the internal pacemaker. Naturally, these functions of dopamine are not exclusive, and this study makes important progress in integrating them.

Dopamine in reinforcement learning

While early research implicated dopamine as the principle neurotransmitter responsible for the hedonic nature of “liking” something, the contemporary view conceptualises dopaminergic activity as a reinforcement signal that facilitates learning, rather than directly causing pleasure. This is in part due to the classic finding that phasic dopamine activity in the mesolimbic pathway constitutes a reward prediction error (the difference between expected and received reward), commensurate with prescriptive models of reinforcement learning.

During learning, dopamine responses gradually transfer to the earliest predictors of a reward, and after this associative pairing is established, response to the reward itself is reduced or absent. Importantly, this means that these response dynamics are fundamentally sensitive to the expected time of reward delivery.

Further to this, if rewards are delivered at different delays, the phasic responses of dopamine neurons to cues signalling these rewards depend on the duration of the delay (as well as reward probability, magnitude and type). This decreased response to longer reward delays typifies the economic principle of temporal discounting: rewards are devalued as a function of delay until their receipt. In reflecting the reduced value of delayed rewards, these neural responses demonstrate sensitivity to timing and appear to encode the intervals between cues and prospective (i.e. future) rewards.

Dopamine and time perception

In addition to its associations with motivation and reward, as a pharmacological agent, dopamine has been routinely acknowledged to play a significant role in time perception, in what some refer to as the ‘dopamine clock hypothesis1. Two sets of evidence in particular highlight this.

Firstly, non-human animal studies have pharmacologically manipulated dopamine during time perception tasks. When given dopamine agonists (e.g. methamphetamine) during a peak interval procedure, rats’ response rates peak earlier, as if their internal pacemaker was accelerated. When given dopamine antagonists (e.g. haloperidol), peak responses are later, commensurate with a slowing of the pacemaker2.

Secondly, electrophysiological and optogenetic studies of neurons in the substantia nigra (which produces dopamine and has inputs to the striatum) have shown that optogenetic activation or suppression of these neurons result in later and earlier timed responses, respectively. These results respectively reflect a slower or faster internal pacemaker, which is the opposite pattern of results seen in the pharmacological studies.

The present study

From the background above, we can see that dopamine appears to be involved in both time perception and reward processing. However, dopamine neurons have previously only been shown to encode elapsing and future delays. The study from Fonzi et al. questioned whether dopamine signals could also convey information related to retrospective, past delays. For example, do dopamine responses to a reward cue encode how much time has already been invested in the pursuit of the reward?

The researchers developed a Pavlovian conditioning paradigm with two reward cues that provided identical information about an upcoming reward, but differed in terms of how much time had elapsed since the previous reward. One cue was only presented after a 15–25 s wait time (“short cue”), while the other was only presented after a 65–75 s wait time (“long cue”). The researchers trained rats with this design while simultaneously using fast-scan cyclic voltammetry to record dopamine concentration in the nucleus accumbens core. If the dopamine responses to the short and long cues did not differ, then it would seem that dopamine activity only encodes prospective information. On the other hand, if the dopamine response to the long cue was larger than that of the short cue, this could be said to reflect the sunk cost of time. Conversely, if the signal to the long cue was decreased relative to that of the short cue, this could be said to reflect the rate of reward3.

The results showed that within this simple experimental design, dopamine responses to the long cue were decreased relative to short cue, suggesting that dopamine in the nucleus accumbens encodes reward rate. An alternative possibility was that this differing dopamine response reflected differing expectations about the time of delivery – the response to the long cue could be decreased because as time elapses, it is increasingly likely that the cue will be shown (i.e. a change in hazard rate). However, there was no relationship between the dopamine response and the time elapsed within each cue type. Furthermore, when another cohort of rats was trained with only a single cue for both short and long wait conditions, no differences were seen in the cue-evoked dopamine response for different wait times. Both of these results speak against the possibility that the dopamine response reflected the changing likelihood of reward delivery over time.

Notably, the principle finding above relied on a single analysis, and the relative difference between the short and long cues. The authors of the study thus performed a follow up analysis to determine whether this retrospective temporal information could be encoded when the animals were not able to directly compare cues. To do this, they trained an independent cohort of rats with short trials and long trials in separate sessions. Even in these scenarios, the short cue evoked a larger dopamine response than the long cue, which suggested that the encoding of retrospective delays was context-independent.

However, once these rats were exposed to both cues in a mixed session, the response to the short cue was increased. While for most of the above experiments there were no differences in behaviour between the two conditions, this increase in dopamine response to the short cue in this intermixed session was also accompanied by an increase in behavioural responding. This implies that (while elapsed wait times can be learnt independently) the dopaminergic encoding of retrospective delays is not entirely context-independent. It also shows that while there are not generally behavioural differences between the short and long cues, there appear to be changes in behaviour when there are also changes in dopamine response.

In a final analysis, the researchers also investigated the effect of the previous trial type, and the tonic dopamine signals over the waiting time. Firstly, for rats recently switch from the separate sessions to an intermixed session, they found that dopamine responses to short cues were significantly increased when the preceding trial was a long cue trial, compared when the preceding trial was a short cue trial. Similarly, dopamine levels were increased during the waiting period after long cue trials, relative to short cue trials (but only up to 25 s, before the identity of the current trial was known). From around the point that the identity was known (25 s), conditioned responding decreased when the preceding trial was a long cue trial, relative to when it was a short cue trial. One possible implication here is that a decrease in wait time dopamine could promote increased anticipatory responding. This would be consistent with the electrophysiological and optogenetic evidence that reducing dopamine increases pacemaker rate (see above).

It is important to reiterate that the results in the former two paragraphs only applied to the experiments where rats where moved from separate training on the short and long cues to an intermixed schedule. These results therefore represent peculiarities in how these animals learnt and adapted to their new context. Overall, the results of the first experiment are the most important here: phasic dopamine responses encode previous durations and appear to constitute a signal of previous reward rate.

This study compellingly demonstrates how even simple experimental designs can lead to novel and valuable findings. The fact that nucleus accumbens dopamine responses encode reward rate suggests a potential mechanism that could normalise value signals for future rewards, and provide contextual information such as the sunk cost of time.

If cue-evoked dopamine responses have to encode durations over a large range of timescales (potentially over 15 orders of magnitude) one interesting future avenue for research would be to describe the mapping between these dopamine responses and the duration of the delays preceding them, in order to precisely understand how durations are represented. More work needs to be done to comprehensively understand the functions of tonic and phasic dopamine and how they relate to perceived and experienced durations, but this study makes substantial progress toward this goal.


Source paper:

Fonzi, K. M., Lefner, M. J., Phillips, P. E. M., & Wanat, M. J. (2017). Dopamine Encodes Retrospective Temporal Information in a Context. Cell Reports 20(8), p. 1774. doi: 10.1016/j.celrep.2017.07.076


  1. It should be noted that much research into the neurobiology of reward and motivation typically focuses on the mesolimbic dopamine pathway. This is in contrast to time perception research, which is more often related to the nigrostriatal pathway (this is also commonly associated with movement). However, these pathways are not independent and the nigrostriatal pathway has also been shown to be critical for reward processing. ↩︎
  2. When both drugs were delivered simultaneously, rats’ peak responses are similar to that of a control condition. ↩︎
  3. Previous research has suggested that longer timescale tonic dopamine activity encodes reward rate. ↩︎

Intended outcome appears longer in time

We live in a complex and dynamic world where sometimes our action yields the intended (desired) outcomes and sometimes the unintended outcomes, but does our subjective time changes as a function of outcome being intended or unintended. To find the answer, read the recent article by Mukesh Makwana and Prof. Narayanan Srinivasan, published in Scientific Reports.

In a series of five experiments involving temporal bisection task (Exp1-4) and magnitude estimation task (Exp5), they investigated whether participants perceive the duration of intended outcome differently compared to unintended outcome, and if yes then what are its underlying mechanisms.

They reasoned that when a participant intends an outcome, its representation gets activated and this prior self-activated representation would lead to earlier awareness of the intended outcome compared to unintended outcome  extending the temporal experience. Recently, pre-activation account has been used to explain temporal expansion (Press et al., 2014).

To manipulate intentional nature of the outcome they used a simple color choice question. In each trial, amongst the choice of two colors, they asked participants to indicate what color circle they want to see, by pressing the allocated key for that color. After 250ms (Exp1) of the intentional key press, they were randomly presented with circle of either intended color (50% times) or unintended color (50% times) whose duration was randomly manipulated amongst nine levels (300ms to 700ms in steps of 50ms). This was done to reduce or eliminate the sensory-motor prediction between the key press and the color of the outcome circle, so that the effect of intention on the perceived duration of the outcome is not confounded with probability-based prediction. Irrespective of the intentional nature of the outcome, participants were supposed to report whether they perceived the duration of the outcome as closer to short (300ms) or long (700ms) anchor duration that they learnt in training phase before the main experiment. Each individual data was sorted into two conditions i.e. when they get the intended outcome (i.e. Intended condition) and when they did not get the intended outcome ( i.e. unintended condition). Psychometric (Weibull) functions were fitted for this two conditions and bisection points were calculated. Bisection point or point of subjective equality is the measure of shift in temporal perception, where lower values of bisection point in a condition indicate temporal expansion relative to condition with higher bisection point. Results of Exp1 showed that participants perceived the duration of intended outcome as longer compared to unintended outcome.

They also studied whether increase in delay between the intentional action and its outcome affects the intention induced temporal expansion observed in Exp1. So further two experiments were performed with increased delay between action and outcome i.e. 500ms in Exp2 and 1000ms in Exp3. Rest stimuli, apparatus and procedure were identical to Exp1 except that in Exp2 instead red and green, yellow and blue color circles were used. Results showed that the intention induced temporal expansion was observed till 500ms delay but as the delay increased to 1000ms the temporal expansion effect vanished, suggesting that the self-activated representation fades away around 1000ms of the intentional action.

To establish that for the above-observed temporal expansion effect, intentional activation of the representation is necessary and not just priming or instruction-based action is not sufficient Exp4 was performed. In Exp4, instead of intending and selecting what color circle they wanted to see, in each trial participants  were shown color word i.e. RED or GREEN on the screen and they just pressed the corresponding key. Rest procedure, stimuli and analysis was similar to Exp1. Results showed no difference in duration perception between word congruent condition and word incongruent condition, suggesting the importance of intention in the above effect.

Lastly, Exp5 was performed using magnitude estimation paradigm to investigate whether intention affects the time perception by increasing the pacemaker speed or affecting the switch or gating component of the “internal clock model”.  The internal clock model is the most influential classical model used to explain human timing behaviour. If any factor influences the pacemaker speed then as the magnitude of the actual duration increases the difference between two conditions should also increase given a typical “slope effect”. On the other hand, if the switch or gating component is affected then no slope effect is observed. Results showed no slope effect, indicating that intention might influence the switch or gating mechanism.

In conclusion, a series of experiments in this study provides convincing evidence that intention affects temporal perception and participants perceives the intended outcome to be longer in duration compared to unintended outcome. Moreover, this intention induced temporal expansion effect depends on the temporal contiguity between the action and the outcome and it vanishes at 1000ms action-outcome delay. Furthermore, in terms of internal clock, this effect is most probably not due to increase in pacemaker speed, rather opening or closing of the switch seems more probable mechanism. As humans are intentional agents and intentions forms a critical part of daily life, more studies investigating the effects of intention on perception in general should be pursued.

 

Reference:

 

  1. Press, C., Berlot, E., Bird, G., Ivry, R., & Cook, R. (2014). Moving time: The influence of action on duration perception. Journal of Experimental Psychology: General, 143(5), 1787.

 

Source article:  Makwana, M., & Srinivasan, N. (2017). Intended outcome expands in time. Scientific Reports, 7(6305) doi:  10.1038/s41598-017-05803-1

 

—Mukesh Makwana (mukesh@cbcs.ac.in),

Doctoral student,

Centre of Behavioural and Cognitive Sciences (CBCS), India.

 

Summer 2017 Conference Season

Finally back in London, ON, after a slightly extreme summer conference tour: The Neurosciences and Music IV in Boston; preceded by our homegrown satellite, Neural Entrainment and Rhythm Dynamics (NERD, credit Ed Large for name/acronym combo); cuttingEEG in Glasgow; and the Rhythm Perception and Production Workshop (RPPW) in Birmingham [had a little break in between those last two to drive around Scotland with my dad and brother]. All of it was extremely inspiring, but of course too much info for any one human to retain, so I’ll try to summarize what I felt were some of the highlights.

I’m of course extremely biased, but I loved every minute of NERD. It was full-on day of fantastic 7-min talks on rhythm and entrainment punctuated by thought-provoking discussion periods. There were two major things I took away from NERD (I’ll only talk about one in any detail). First, we’re not all speaking the same language a lot of the time. That’s of course a problem that has been and will be around forever to some extent, and it’s also OK. For example, when I talk about “rhythm” or “beat”, that’s not quite the same thing someone else is talking about when they use the same words. A particularly sticky word at the moment is “entrainment”, which seems to mean a lot of things to a lot of different people, despite being very well defined in the math/physics domains. Even things like “beat salience” or “beat strength” are contested terms, making them hard things to study and talk about. The important thing, I think, is that we make sure we’re operationally defining terms in the papers we’re publishing and the talks we’re giving, so that even if we’re using terms differently, we can talk about the same phenomena. It sounds obvious, but this is done surprisingly infrequently, including by myself I’m sure. The second thing I took away, which I won’t discuss here and which very well may be the subject of a future blog, is that rhythmic/temporal expectation effects on behavior are harder to observe than one might think. More on that later.

There were a million interesting talks and posters at both NeuroMusic and RPPW, but here I’ll focus on timing-related issues. Even though I’ve heard various bits of the data before, I was struck (again) by the idea and accumulating evidence that synchrony is social. We need to be synchronized with each other to successfully navigate conversational turn-taking. Toddlers are more likely to exhibit pro-social helping behaviors towards adults that they have moved in synchrony with compared to someone they have moved out-of-sync with. Babies synchronize eye contact with a singer to the beat of the song the singer is producing.

A relatively new focus was on synchronization between brains. New ways to analyze electrophysiological data (using “intersubject synchronization” or “intersubject correlation”) allow us to assess interpersonal neural synchrony. Traditionally, measures of intersubject synchronization don’t necessarily focus on social situations, but nonetheless show that individual brains are more synchronized with each other (i.e., respond more similarly to the same stimulus) when individuals are more engaged with whatever they are watching or listening to. We presented EEG data that we collected from 60 EEG participants, 20 at a time, in slightly different social situations while viewing/listening to a concert. But I’m not here to self-promote. One really interesting twist on this idea was to use noninvasive brain stimulation to force pairs of brains to be either in sync or out of sync with each other. Despite the situation not actual being social (the individuals making up each pair were not able to see or interact with each other, but did have auditory information about the other person’s behavior), pairs of participants synchronized tapping better with each other when their brains were in sync compared to when their brains were out of sync. The moral of the story is that better neural synchronization leads to better behavioral synchronization, which could in turn lead to stronger affiliation in the social domain.

In general, rhythm and timing were very present topics at the conferences I attended this summer. And it seems like the more we know about how brains are actually involved in behavioral synchrony, the better we stand to understand how synchrony is involved in social situations. I look very much forward to seeing how this research evolved over the next years, and in hopefully being a part of it myself.

Olfactory-Visual Sensory Integration Twists Time Perception

During everyday interactions, our senses are bombarded with different kinds of sensory information, which are processed by dedicated sensory systems operating at different temporal sampling scales to form a coherent percept. The question is whether information from one modality (say olfactory) influences the temporal perception of stimulus from other modality (say vision). Although, previous studies have investigated the effect of auditory stimulus on temporal perception of visual stimuli [1, 2], the evidence for the effect of olfactory stimulus on temporal perception of visual stimuli was lacking. A recent study published in Cerebral Cortex, by Prof. Wen Zhou and her lab members (Dr. Bin Zhou, Guo Feng, and Wei Chen), fills this gap and addresses whether odor influences visual temporal sampling and duration perception.

To study the effect of odor on visual temporal sampling, they used a two alternative forced choice chromatic critical flicker fusion (CFF) task with two isoluminant complimentary color images of banana or apple alternating at different frequencies (15, 20, 22.5 & 25 Hz in different blocks) for duration of 400ms (see figure 1, in original paper). In each trial, there were two 400ms flickering interval each flanked by 200ms mask, and separated by 600ms blank between the two intervals. Out of two, only one interval contained the flickering fruit image (either banana or apple) and participants reported the interval that contained fruit image. Along with visual stimuli, in Exp1 (N=16), participants were also exposed to two different odor stimuli (banana-like, amyl acetate 0.02%v/v in propylene glycol; and apple-like, apple flavor Givaudan, in separate blocks). The idea was to check whether the odor congruency influence the temporal sampling (CFF threshold) for the flickering banana or apple images. Results revealed that participants object detection increased significantly when the odor and the object image content matched, even when the task did not demanded any explicit object discrimination or identification, suggesting that sensory congruency between olfactory and visual inputs boosted the corresponding object visibility around CFF. Another analysis by fitting the psychometric function for the two odor conditions, with frequencies on x-axis and accuracy on y-axis, suggested that olfactory-visual congruency also facilitated the visual temporal sampling.

To establish that the above congruency effect is specific to odor and not just semantic information (or context) provided by the odor, they performed two control experiments. In the first control experiment (Exp2A, N=16), participants performed exactly the same task as Exp1 but instead of actual odors, odorless purified water was used and was suggested to participants as diluted banana or apple odor. In the second control experiment (Exp2B, N=16), semantic textual labels, “banana odor” or “apple odor”, were displayed at the center of the screen. In both the control experiments, they did not observe the odor-visual congruency effect, suggesting that presence of odor is important for such sensory integration.

The next question was to find the neural correlates of the odor-visual congruency effect, emphasizing at what level of visual processing the odor starts modulating it. For this, they performed an EEG experiment (Exp3, N=18) using the same stimuli as in Exp1, but modifying the task a bit. In the modified task, only one flickering interval of 400ms was presented flanked by red-green noise mask of 100ms, and participants reported whether the object was present or absent in that trial. All objects (apple or banana image) were presented to participants’ at subliminal frequency. For nine participants flicker frequency of 22.5Hz was used whereas for other nine participants 25Hz was used. Results from time-frequency analysis, revealed that maximum congruency-induced enhancement (i.e. greater normalized power difference) was observed in electrodes over right temporal regions around 150-300ms post stimulus onset. The difference around this time window suggests that during odor-visual congruency, odor starts influencing vision at the stage of object-level processing. Even source-localization analysis indicated the activation of right temporal region which is again known to be involved in object level representations. Thus, these evidences strongly suggest that odor influences the corresponding visual object at the stage of object-level processing.

From the above experiments, it was evident that the odor-visual congruency modulates visual temporal sampling, so the next logical question was whether it also influences the perceived duration of the visual stimuli. To answer this question, in Exp5 (N=24), they used a 2-Alternative Force Choice (AFC) comparison task, in which one image (either apple or banana) was a standard image (500ms) and the other image (either banana or apple) was test image (of varying durations 300, 350, 400, 450, 500, 550, 600, 650, 700ms). Participants reported which of the two images appeared longer in duration. For half participants (N=12) apple image was standard and banana image was comparison, whereas for other half (N=12) banana image served as standard and apple image served as comparison. Participants in both these groups were exposed to banana-like or apple-like odor in separate blocks. Point of subjective equality (PSE) and difference limen (DL) were measured for both the odor conditions. PSE is the measure of perceived duration whereas DL is the measure of temporal sensitivity. A two way mixed ANOVA on PSE values, with odor (banana-like, apple-like) as within-subjects factor and comparison image (banana image, apple image) as between-subjects factor, showed significant interaction. Further post hoc analysis after Bonferroni correction revealed that participants perceived the duration of the image to be longer when the image content and the odor were congruent compared to when they were incongruent. Similar analysis with DL, did not show any significant difference neither for main effects nor for interaction, suggesting that odor modulates only the perceived duration but not the temporal sensitivity.

Again to confirm that the above congruency effect on perceived duration is due to odor, not just because of semantic information (or context) provided by the odor, they performed two control experiments (Exp5A and Exp5B) similar to Exp2A and Exp2B. In Exp5A (N=24) instead of odor, odorless purified water suggested as diluted banana-like or apple-like odor were presented, whereas in Exp5B (N=24) instead of odor, textual labels (“banana odor” or “apple odor”) were presented on the screen. Neither the purified water nor the textual labels, showed the odor-visual congruency effect of perceived duration as seen in Exp4, suggesting the importance of odor in odor-visual sensory integration to modulate visual temporal perception.

In conclusion, this study provides a convincing evidence for the effect of odor on visual time perception, including temporal sampling and perceived duration. In future, it would be interesting to investigate this effect with other time perception paradigms such as magnitude estimation and measure the slope effect, which might help to know whether odor influences the pacemaker speed or the switch/ gating mechanisms in context of “internal clock model”. Moreover, it would be further interesting to investigate whether such odor-visual congruency effect influence the neural correlate of time perception such as CNV (contingent negative variation) component.

References:

1. Romei, V., De Haas, B., Mok, R. M., & Driver, J. (2011). Auditory stimulus timing influences perceived duration of co-occurring visual stimuli. Frontiers in psychology, 2.

2. Yuasa, K., & Yotsumoto, Y. (2015). Opposite distortions in interval timing perception for visual and auditory stimuli with temporal modulations. PloS one, 10(8), e0135646.

Source article: Zhou, B., Feng, G., Chen, W., & Zhou, W. (2017). Olfaction Warps Visual Time Perception. Cerebral Cortex, 1-11.

—Mukesh Makwana (mukesh@cbcs.ac.in),
Doctoral student,
Centre of Behavioural and Cognitive Sciences (CBCS), India.

The P3 and the subjective experience of time

The P3 (or P300) is an event related potential (ERP) component that has commonly been associated with attentional mechanisms and the updating of expectations. It can be evoked by stimuli in oddball paradigms, where a low-frequency stimulus (the “oddball”) is shown intermixed with high-frequency stimuli (the “standards”). For example, a subject in such a paradigm might be shown a series of words in white font (the standard stimuli), and much more rarely, be shown a word in red font (the oddball). This oddball stimulus commonly elicits a P3 at centro-occipital electrodes, one which is more positive in amplitude compared with the standard stimuli.

Oddball paradigms are also commonly cited in the time perception literature: canonically, the duration of oddball stimuli is overestimated, relative to the standard stimuli. Given this commonality, could the P3 be a neural correlate of this temporal distortion? A recent study by Ernst et al. investigated this question explicitly.

This study used an oddball paradigm as described above, where the standard and oddball stimuli varied in duration (from 600 – 1120 ms). After the presentation of an oddball stimulus, subjects were asked whether this was shorter or longer in duration relative to the preceding standards1. The durations of the stimuli were chosen such that the P3 occurred well before the duration judgement was required (and indeed before the termination of the oddball). This way, it was possible to test whether the amplitude of the relatively early P3 predicted the subsequent duration judgement.

The behavioural data confirmed that time overestimations were observed for the oddball stimuli; the typical temporal oddball effect. Analysis also confirmed the primary hypothesis, larger P3 amplitude led to overestimation of the oddball stimuli. Specifically, the P3 was larger for overestimated, compared to correct trials. (The P3 was also larger for correctly classified “long” judgements compared to correctly classified “short” judgements.) Thus, in trials where the oddball was overestimated, the P3 closely resembled that in correctly judged trials where the oddball was actually longer. In sum, these findings suggest that the P3 tracked the perceived duration of the oddball stimuli.

Average ERP amplitudes at electrode Pz, and scalp distributions, split by duration judgement.
Average ERP amplitudes at electrode Pz, and scalp distributions, split by duration judgement.

The researchers also used a multivariate pattern recognition technique in a supplementary analysis. Here, a classifier was trained to discriminate between standard and oddball stimuli on the basis of the ERP data. Classification accuracy was assessed across the different time windows, and showed above-chance accuracy from 125 ms after oddball presentation, reflecting general ERP differences consistent with the scalp distribution of the P3. By assessing the output of the classifier on trials pooled into correct short, correct long, and overestimated durations, they were able to recapitulate the main findings of the ERP analysis (but for a generated, essentially synthetic, estimated ERP) over the time window of 375 – 600 ms. In essence, because the classifier selectively reproduced an ERP response to oddball stimuli, this analysis lessens the likelihood that some other experimental feature was responsible for the difference in duration perception. Similarly, given the scalp distribution and time window, it also provides some data-driven support that the P3 was responsible for the effect, without specifically pre-selecting for the P32.

Overall, this study provides clear support for the hypothesis that the P3 is a neural correlate of the temporal oddball effect. Given that the P3 has been implicated a rather wide variety of phenomena, these phenomena can now also be interrogated from the perspective of time perception. There is also an ample opportunity for future research to ascertain whether the P3 is either necessary or sufficient for temporal distortions in a wider range of paradigms. In general, this study has delineated the P3 as a valuable component of interest for ongoing time perception research that uses EEG.


Source paper:

Ernst, B., Reichard, S. M., Riepl, R. F., Steinhauser, R., Zimmermann, S. F., & Steinhauser, M. (2017). The P3 and the subjective experience of time. Neuropsychologia, 103, 12–19. http://doi.org/10.1016/j.neuropsychologia.2017.06.033


  1. Notably, there were more trials in which the oddball durations were actually shorter than the standards, ostensibly to increase the number of overestimations. One potential issue is that participants may have tried to balance their proportion of short and long judgements, resulting in some “overestimations” that were due to a decision bias, rather than a perceptual bias. ↩︎
  2. It should be noted, however, that the classifier was trained on all of the electrodes, and not just those located centro-occipitally. ↩︎

Implicit variations of temporal predictability: Shaping the neural oscillatory and behavioral response

Anyone accessing this blog probably doesn’t need to be convinced that the ability to predict the timing of upcoming, behaviorally relevant stimuli is important for our ability to perceive and interact with the world. Although I’m quite rhythm-centric, it’s obvious that there are multiple ways in which we can estimate when something important might occur. For example, when the occurrence of an event is inevitable within a specific time window, its probability of occurrence usually increases as a function of time according to what is referred to as a “hazard function” (think of the probability that a car will eventually break down as you keep driving it). However, it’s also possible to engineer distributions for which the probability of occurrence is centered on a particular time point with a small or large amount of variability. The question is then about the neural mechanisms on which this type of temporal predictability (where the event usually occurs after about 1 second, for example) is based.

A recent EEG study by Herbst and Obleser examined the behavioral and neural differences between more and less temporally predictable situations, where temporal predictability had to be learned implicitly by the participants. The task was a pitch categorization task, in which a single tone was presented on each trial, and participants indicated whether it was high or low. The trick was that the time interval between a “cue” that the trial had started and the “target” (to-be-categorized tone – which was importantly embedded in noise to make the task more difficult) was varied according to distributions that made the target more or less temporally predictable. I’ll focus here on their Experiment 2, in which short blocks were presented in randomized order where the target timing was strongly predictable, weakly predictable, or not predictable.

Behaviorally, classical foreperiod effects made it clear that the basic experimental design worked as planned – reaction times decreased with increasing foreperiods (the later the target, the faster the RT). However, the condition-specific behavioral effects (or lack thereof) call into question whether the elegant experimental design (that involved completely implicit learning of temporal predictability) worked as well would have been hoped. The size of the foreperiod effect was indeed larger for temporally predictable compared to unpredictable conditions. However, the critical interaction was actually decidedly nonsignificant. Given that I might have rather expected some benefit at the expected time for the predictable conditions, rather than just a steeper foreperiod effect, I leave it up to the reader to judge whether they are sufficiently convinced by the behavioral results.

However, the some of the neural effects do seem to solidly indicate that implicitly learned temporal predictability was doing something. For example, P2-ish ERP magnitudes decreased with temporal predictability, and a later negative deflection increased in magnitude with temporal predictability. Maybe most interesting, dynamic changes in alpha power seemed to anticipate the expected target onset – alpha power increased briefly after cue onset, then decreased below baseline, and seemed to rebound back to baseline levels in anticipation of target onset. This effect was more obvious for temporally predictable compared to unpredictable conditions. [Of course, this begs the question why getting alpha back to baseline (to a zero-value) would be good for performance.]

For what should have been the most interesting neural dependent measure though, the results are confusing. The authors hypothesized (as I would have), that phase consistency across trials in low-frequency bands (esp. delta, ~0.5–4 Hz) would be higher prior to predictable than undpredictable targets. The reason is that temporal predictability allows low-frequency neural oscillations to get into the right phase at the right time for upcoming stimuli, which might be exactly why we perceive predictable things better than unpredictable things. This goes for paradigms using rhythmic stimuli to entrain low-frequency oscillations as well as more classical foreperiod-style paradigms that vary temporal predictability of a target in a more interval-based fashion. Turns out, Herbst and Obleser observed exactly the opposite – delta phase consistency was reduced for predictable compared to unpredictable schemes (though this difference did occur just after the cue and wasn’t necessarily present leading up to the target when it would have been expected).

With respect to delta phase, there are several possible explanations for the surprising results (that delta phase was less concentrated for predictable situations). First, the authors took great care not to contaminate the pre-target time window with target-evoked responses. By removing the target-evoked ERPS before time–frequency transformation, they may have removed an artifact that has been present in previous studies. Second, the authors took greater care than any paper of which I’m personally aware to not just manipulate foreperiod, but to randomize inter-trial intervals in a way that wouldn’t allow for entrainment to the low-frequency pace of the task itself. To my knowledge, all studies of the neural underpinnings of temporal preparation (except for the one being discussed here) using a fixed or jittered inter-trial interval have never taken such care to abolish an overall experimental pacing. Nonetheless, I still would not have expected opposite phase-consistency results.

In any case, I think the paradigm – where temporal predictability had to be learned entirely implicitly – is remarkably clever and can be used in future work to truly understand the neural mechanisms underlying temporal predictability that is not entrainment-based (i.e., based on rhythm). Given recent work moving in this direction, this work carefully removing rhythmicity (here, of the task itself) and eliminating evoked responses that could contaminate phase-concentration measures, should be used as an example of thoughtful experimental design.