Time perception and the heart

The heart has held a central place in theories of human function for centuries. It was proclaimed as the seat of life in Ancient Greek and Byzantine medical literature, and the Ancient Egyptians embalmed their dead by carefully preserving the heart while unceremoniously discarding the brain1. This stands as a stark inversion of today’s physiological priorities. However, researchers have recently again been investigating human cognition and behaviour from the perspective of cardiac physiology.

My colleagues and I recently published a paper showing that temporal reproduction is associated with heart rate. In this post I will briefly discuss this paper, and review the literature on why heart rate — and autonomic nervous system function in general — is deeply connected to cognition.

Cardiac physiology

The idea that periodic physiological signals might constitute a biological clock is not new2. The most popular, recent account of this idea, endorsed by Marc Wittmann and Bud Craig, suggests that time perception is a function of integrated interoceptive signals34, which explains why physiological factors such as arousal5 and body temperature6 can affect time perception.

However, a simple linear relationship between heart rate and time perception is almost definitely an oversimplification. For instance, it’s clear that increasing heart rate (by exercise for instance) doesn’t affect time perception significantly7. But there are other elements of cardiac signals that provide a wider overview of the autonomic nervous system. This arises because the heart is innervated by both the sympathetic (SNS) and parasympathetic nervous system (PNS), and each of these arms have slightly different latencies (the PNS acts more rapidly on the heart). Thus, different components of the variability of heart rate can be attributed to the SNS and PNS. This means, for instance, that if analysed in the time-frequency domain, power in the high-frequency band of heart rate variability reflects the functioning of the PNS8.

Spectral components of heart rate variability. Power in different frequency bands corresponds to the function of the autonomic nervous system. VLF, very low frequency; LF, low frequency; HF, high frequency.
Spectral components of heart rate variability. Power in different frequency bands corresponds to the function of the autonomic nervous system. VLF, very low frequency; LF, low frequency; HF, high frequency.

Over the last decade or so, researchers have used these types of analysis techniques to link cardiac signals with different types of cognitive processes. This includes emotion regulation9, self-control10, and working memory11. Further to this, directly stimulating the ANS via the vagus nerve can also enhance memory processes12.

Temporal reproduction and heart rate variability

In our study, we assessed whether performance in a temporal reproduction task was associated with these different measures of cardiac activity. Healthy participants completed a standard temporal reproduction task with durations spanning from 2 – 15 seconds, while we recorded electrocardiogram (ECG). We characterised participants temporal reproductions with a two parameter psychophysical function (Steven’s power law), where the exponent parameter indicates the concavity of the function and thus whether participants tend to under-reproduce (exponent > 1), or over-reproduce (exponent < 1) intervals.

We found that that different frequency components of heart rate variability were associated with differences in participants’ reproductions. Specifically, we found that the high-frequency component was negatively associated with the exponent parameter of the psychophysical function. This suggests that individuals with higher PNS cardiac influence under-reproduced longer intervals compared with those with lower PNS cardiac influence. We also found a similar negative association between the low-frequency component and the exponent. The interpretation of the low-frequency component of heart rate variability is not as clear as that of the high-frequency component, as it reflects physiological processes other than purely SNS function13. However, increases in the low-frequency component have been previously been associated with decreases in attention and fatigue due to time-on-task14, thus it is possible that individuals who experienced more task-related fatigue performed more poorly on the reproduction task. Overall, these results show that time perception is associated with autonomic nervous system function.

Time reproduction function split by low-frequency heart rate variability (LF-HRV). Participants with high LF-HRV under-reproduced longer intervals.
Time reproduction function split by low-frequency heart rate variability (LF-HRV). Participants with high LF-HRV under-reproduced longer intervals.

Existing literature

This is not the first time that researchers have found links between time perception and heart rate. For instance, one study found that individuals have better absolute accuracy at reproducing durations when their heart rate slows down during the encoding of the sample interval15. Individuals with higher resting heart rate variability are also more accurate at reproducing durations16. Similarly, individuals with overall higher heart rate variability are also more accurate in a temporal bisection task17. As the PNS is responsible for slowing down heart rate, and is also the principle determinant of heart rate variability, this would suggest that individuals with higher PNS function are more accurate at duration reproduction in general.

However, other studies that have directly stimulated the vagus nerve (the main nerve of the PNS), have found that this can cause overestimations (under-reproductions) of time in the ranges of 34 – 230 seconds18. This is approximately consistent with our results showing that individuals with higher PNS function under-reproduced durations, however it conflicts somewhat with the results of the above studies. One possible discrepancy is that the above studies used absolute measures of accuracy, whereas we observed a directional effect. Whatever the reason for these differences, the relationship between autonomic nervous system function and temporal reproduction clearly merits further investigation.

In sum, this research seems to suggest that while the beats of the heart themselves are not analogous to a pacemaker, changes in the autonomic nervous system are either a corollary of a timing mechanism, or the autonomic nervous system directly plays role in how humans perceive time. Given the non-invasive nature of heart rate recordings, future research may be able to clarify exactly how signals from the heart inform our perception of time.


Source paper:

Fung, B. J., Crone, D. L., Bode, S., & Murawski, C. (2017). Cardiac Signals Are Independently Associated with Temporal Discounting and Time Perception. Frontiers in Behavioral Neuroscience, 11, 369. http://doi.org/10.3389/fnbeh.2017.00001

More reading on how cardiac signals relate to cognition:

Thayer, J. F., & Lane, R. D. (2000). A model of neurovisceral integration in emotion regulation and dysregulation. Journal of Affective Disorders, 61(3), 201–216. http://doi.org/10.1016/S0165-0327(00)00338-4

Thayer, J. F., & Lane, R. D. (2009). Claude Bernard and the heart–brain connection: Further elaboration of a model of neurovisceral integration. Neuroscience and Biobehavioral Reviews, 33(2), 81–88. http://doi.org/10.1016/j.neubiorev.2008.08.004


  1. Lykouras, E., Poulakou-Rebelakou, E., & Ploumpidis, D. N. (2010). Searching the seat of the soul in Ancient Greek and Byzantine medical literature. Acta Cardiologica, 65(6), 619–626. http://doi.org/10.2143/AC.65.6.2059857 ↩︎
  2. Goudriaan, J.C., 1921. Le rhythm psychique dans ses rapports avec les frequences cardiaques et respiratoires. Arch. Neerl. Physiol. 6, 77–110. ↩︎
  3. Craig, A. D. (2009). Emotional moments across time: a possible neural basis for time perception in the anterior insula. Philosophical Transactions of the Royal Society B-Biological Sciences, 364(1525), 1933–1942. http://doi.org/10.1098/rstb.2009.0008 ↩︎
  4. Wittmann, M. (2009). The inner experience of time. Philosophical Transactions of the Royal Society of London. Series B, Biological Sciences, 364(1525), 1955–1967. http://doi.org/10.1098/rstb.2009.0003 ↩︎
  5. Droit-Volet, S., & Meck, W. H. (2007). How emotions colour our perception of time. Trends in Cognitive Sciences, 11(12), 504–513. http://doi.org/10.1016/j.tics.2007.09.008 ↩︎
  6. Wearden, J. H., & Penton-Voak, I. S. (1995). Feeling the heat: Body temperature and the rate of subjective time, revisited. The Quarterly Journal of Experimental Psychology, 48(2), 129–141. http://doi.org/10.1080/14640749508401443 ↩︎
  7. Schwarz, M. A., Winkler, I., & Sedlmeier, P. (2012). The heart beat does not make us tick: The impacts of heart rate and arousal on time perception. Attention, Perception & Psychophysics, 75(1), 182–193. http://doi.org/10.3758/s13414-012-0387-8 ↩︎
  8. Berntson, G. G., Bigger, J. T., Eckberg, D. L., Grossman, P., Kaufmann, P. G., Malik, M., et al. (1997). Heart rate variability: origins, methods, and interpretive caveats. Psychophysiology, 34(6), 623–648. ↩︎
  9. Park, G., & Thayer, J. F. (2014). From the heart to the mind: cardiac vagal tone modulates top-down and bottom-up visual perception and attention to emotional stimuli. Frontiers in Psychology, 5, 278. http://doi.org/10.3389/fpsyg.2014.00278 ↩︎
  10. Segerstrom, S. C., & Nes, L. S. (2007). Heart Rate Variability Reflects Self-Regulatory Strength, Effort, and Fatigue. Psychological Science, 18(3), 275–281. http://doi.org/10.1111/j.1467-9280.2007.01888.x ↩︎
  11. Hansen, A. L., Johnsen, B. H., & Thayer, J. F. (2003). Vagal influence on working memory and attention. International Journal of Psychophysiology, 48(3), 263–274. http://doi.org/10.1016/S0167-8760(03)00073-4 ↩︎
  12. Clark, K. B., Naritoku, D. K., Smith, D. C., Browning, R. A., & Jensen, R. A. (1999). Enhanced recognition memory following vagus nerve stimulation in human subjects. Nature Neuroscience, 2(1), 94–98. http://doi.org/10.1038/4600 ↩︎
  13. Reyes Del Paso, G. A., Langewitz, W., Mulder, L. J. M., van Roon, A., & Duschek, S. (2013). The utility of low frequency heart rate variability as an index of sympathetic cardiac tone: A review with emphasis on a reanalysis of previous studies. Psychophysiology, 50(5), 477–487. http://doi.org/10.1111/psyp.12027 ↩︎
  14. Fairclough, S. H., & Houston, K. (2004). A metabolic measure of mental effort. Biological Psychology, 66(2), 177–190. http://doi.org/10.1016/j.biopsycho.2003.10.001 ↩︎
  15. Meissner, K., & Wittmann, M. (2011). Body signals, cardiac awareness, and the perception of time. Biological Psychology, 86(3), 289–297. http://doi.org/10.1016/j.biopsycho.2011.01.001 ↩︎
  16. Pollatos, O., Laubrock, J., & Wittmann, M. (2014). Interoceptive Focus Shapes the Experience of Time. PloS One, 9(1), e86934. http://doi.org/10.1371/journal.pone.0086934 ↩︎
  17. Cellini, N., Mioni, G., Levorato, I., Grondin, S., Stablum, F., & Sarlo, M. (2015). Heart rate variability helps tracking time more accurately. Brain and Cognition, 101, 57–63. http://doi.org/10.1016/j.bandc.2015.10.003 ↩︎
  18. Biermann, T., Kreil, S., Groemer, T. W., Maihöfner, C., Richter-Schmiedinger, T., Kornhuber, J., & Sperling, W. (2011). Time Perception in Patients with Major Depressive Disorder during Vagus Nerve Stimulation. Pharmacopsychiatry, 44(05), 179–182. http://doi.org/10.1055/s-0031-1280815 ↩︎

The benefits and costs of temporal attention

Attending to specific moments in time improves the quality of sensory information presented at these moments. Behavioural and neural benefits of temporal orienting have been shown in several contexts, including rhythmic regularities of the presented stimuli, cues informing about the relevance of specific time windows, and evolving probabilities of events occurring over time. However, it is not clear to what extent the effects and mechanisms of temporal attention are shared with other domains of attentional selection. For example, spatial attention not only improves the processing of stimuli presented at the attended location, but also has detrimental effects on processing stimuli presented elsewhere, as compared to neutral baseline conditions. This is exactly the focus of a recent paper by Denison, Heeger and Carrasco, who investigate whether similar perceptual trade-offs characterise temporal attention.

In a series of behavioural experiments, the authors present cues that inform participants of the latency of targets about which they will be most likely prompted for a response. Specifically, participants are asked to discriminate the visual orientation of gratings presented at various latencies after the auditory cue. In the simplest scenario (Experiment 1), the cue informs (with 75% validity) whether the orientation of the first (1000 ms after the cue) or the second target (250 ms later) will need to be reported. Crucially, 20% of the trials were left neutral and contained an uninformative cue, allowing for a baseline comparison. An analysis of accuracy and reaction times showed that when participants believe they will be asked about the first target, they are better and faster at discriminating its orientation than when they can’t predict which target will be relevant, and their performance is weakest if they are invalidly cued to the second target. This result suggests that participants attending a later time window will perform worse when asked about stimuli presented before this time window. But is this effect symmetric – i.e., is task performance worse for targets presented late if participants attend an earlier time window? While the behavioural effects show a pattern consistent with such an effect, the pairwise comparisons of the three experimental conditions (valid, neutral, and invalid cues) are not significant. This is not entirely surprising, given that at the moment of report the late targets might be accessed more easily since they are more recent and have not been interrupted by other irrelevant targets.

Thus, in a second experiment, the same task was administered but this time with three consecutively presented targets. However, the results were only partly consistent with the simpler version of the experiment. For example, attentional costs for targets presented before an attended time window were marginally significant or not observed. Interestingly, while attentional benefits of cueing were not observed for the intermediate latency, targets presented at this latency showed robust costs when participants attended an earlier time window, suggesting that in some contexts temporal attention might disrupt processing of targets presented after the attended time window, perhaps similar to an attentional blink.

To address the question whether temporal attention actually improves the quality of visual representations (as previously shown for rhythmic orienting) or can be explained by other factors such as missing the unattended targets (as in attentional blink) or mistakes in which targets are reported, another version of the task was run. This time, again using two targets, participants had to reproduce the orientation of targets instead of only reporting whether they were tilted clockwise or counter-clockwise. The results of these experiment suggested that attention primarily affected the precision of sensory representation but not the rates of guesses (expected if unattended targets were completely missed) or target swaps. However, as in the first experiment, these effects were largely confined to targets presented early, with no evidence of benefits or costs at longer latencies.

Taken together, while these studies provide further evidence for behavioural benefits of temporal attention and for the first time directly address attentional costs in the unattended time windows, the results aren’t always symmetric or consistent across experiments. Some of these differences can be explained by task specifics; however, it would be interesting to see whether similar effects can be identified in a more continuous version of the task, where the influence of each consecutively presented stimulus – attended or not – on the final orientation report could be quantified using established modelling techniques. Finally, the paper by Denison et al., does provoke further questions about the possible neural implementation of the observed perceptual trade-offs – where any differences that might be observed between the neural mechanisms of temporal and spatial attention will likely transform our understanding on attentional selection.

Ryszard Auksztulewicz, Oxford Centre for Human Brain Activity

Source article: Denison RN, Heeger DJ, Carrasco M (2017) Attention flexibly trades off across points in time. Psychon Bull Rev, Jan 4. doi: 10.3758/s13423-016-1216-1.

Frequency tagging indexes cortical entrainment related to temporal prediction

Listening to music typically induces a strong sense of an underlying beat. Although clearly related to periodicities contained in the stimulus, beat perception is internally generated since beat induction can occur in instances where no stimulation is present (i.e., in syncopated rhythms). Thus the capacity to extract a beat from periodic input provides an interesting phenomenon for examining the neural processes that temporally organise and ulitmately structure our perception of the world.

The neural mechanisms associated with beat-induction can be measured with a frequency tagging technique applied to EEG data recorded while participants listen to music. This approach involves examining the peaks in the frequency spectrum of EEG data that correspond to periodicities contained in the stimulus. By examining the activity evoked by syncopated rhythms (where the beat percept does not correspond to the sensory input), previous studies have shown that this technique indexes both periodic activity associated with the sensory input, and endogenous processes involved in extracting temporal structures from periodic input.

To further demonstrate the functional significance of this approach, Nozaradan, Peretz and Keller examined how neural entrainment measured with frequency tagging is correlated with individual differences in rhythmic motor control. The authors presented both unsyncopated and syncopated auditory rhythms whilst brain activity was recorded with EEG. Individual differences in ability to detect the beat in these rhythms were assessed offline with finger tapping tasks. In addition participants also completed another finger tapping task designed to assess temporal prediction. This stimulus comprised an aperiodic, predictable sequence of tones whereby the tempo continually varied between 400-600 ms (with a sinusoidal contour). This sequence was used to assess participants’ ability to anticipate the upcoming stimulus interval by quantifying the lag-1 and lag-0 correlation between the inter-stimulus interval and the inter-tap interval. Accurate prediction of the stimulus sequence would result in a larger lag-0 correlation than the lag-1 correlation, whereas tracking behaviour would be observed as a larger lag-1 correlation than lag-0 correlation.

The results showed that periodic stimulation produced peaks in the frequency spectrum of the recorded EEG data that corresponded to beat and non-beat frequencies. Importantly however, behavioural performance correlated selectively with the strength of entrainment in the beat frequencies. Tapping accuracy (mean asynchrony in the beat perception tasks) and the temporal prediction index (from the tempo change task) correlated positively with the height of the peaks for the beat induced frequencies, whereas the degree of entrainment in non-beat frequencies was negatively correlated with periodic tapping accuracy and was uncorrelated with prediction in tempo changing sequences. Together, these results highlight the functional significance of processes indexed by the frequency tagging approach, and show that beat perception is related to selective entrainment of neural activity to beat related frequencies.

The authors argue that the relationship between neural entrainment and temporal prediction is consistent with predictive coding models, whereby the brain optimises behaviour by forming internal models of the causes of sensory events. These internal models act as templates based on past experience that optimise sensory processing by providing predictions about the timing of upcoming sensory input. This argument is supported by another study published in 2016, which showed that temporal predictions associated with both periodic and aperiodic sequences lower sensory thresholds in a pitch discrimination task. Intriguingly, this study showed that isochronous stimulation also produced faster response times, whereas aperiodic stimulation did not. The authors of this paper argued that this dissociation may reflect the lowering of motor thresholds caused by simple sensory-motor coupling in the isochronous context, whereas changes in sensory processing may have been due to controlled processes that update internal models (i.e., linked to period correction). It remains to be seen whether the frequency tagging approach can be used to further dissociate the exogenous and endogenous processes involved in temporal prediction.

 

Bronson Harry

The MARCS Institute, Western Sydney University

Controlling Time Perception using Optogenetics

Imagine you have a device which can help you control your perception of time, so that you can speed up or slow down the subjective time at your will. Sounds like a science fiction but not in the near future. Yes a step has been taken to control time perception in mice using optogenetics. A recent study by Sofia Soares, Bassam Atallah, and Joseph Paton published in Science, not only measured the activity of the Dopaminergic (DA) neurons in substantia nigra pars compacta (SNc) in mice but also manipulated the activity of these neurons using optogentics resulting in altered timing behaviour.

They first trained mice to categorize variable interval between two tones (0.6, 1.05, 1.26, 1.38, 1.62, 1.74, 1.95 and 2.4 sec) as shorter or longer than 1.5 sec by touching the right or left port with their nose. The correct response was rewarded. After around 2 months of training mice were very accurate in performing this task.

Next, to confirm that the activity in SNc-DA neurons is essential for accurate timing, they used pharmacogenetic suppression approach. In this, they injected the SNc of a set of mice with the viral vector carrying the genes (hM4D(Gi)-mCherry). When hM4Di is expressed it does not affect the normal functioning of the neurons, but when an inert molecule like clozapine-N-oxide (CNO) is introduced it activates the hM4Di which then results in silencing of those neurons. They found that the timing behavior become very poor in the set of mice treated with CNO compared to the control set which were treated with saline. They also checked for timing accuracy before and after the CNO treatment showing that mice regained its timing accuracy.

Further, to establish a strong molecular basis for trial-by-trial variation in timing, they used the fiber photometry approach. In this, they measured the activity in the DA neurons while mice were performing the timing task. To accomplish this they injected SNc of a set of mice with the viral vector carrying the genes (GCaMP6f and tdTomato). When neurons are active there is lot of intake of Ca2+, which then binds with GCaMP6f and tdTomato protein leading to conformation changes and fluorescence. The amount of fluorescence is the indicator of activity of the cells. To assess whether trial-by-trial change in the activity within these neurons correlates with the timing behavior of mice, they distributed all the trials based on the measured activity of DA neurons into three categories i.e. low activity, medium activity, and high activity. They found that when the activity in the dopamine neurons was high, mice more often reported shorter response, whereas when the activity was low, mice more often reported longer response. Thus, the activity in the DA neurons of SNc could predict the mice behavioural response.

Finally, to establish a causal link between the activity of the DA neurons in SNc and timing behavior, they used optogenetic approach. They injected neurons in SNc with viral vectors carrying genes (ChR2-EYFP, and ArchT-GFP, or eNpHR3.0.EYFP). ChR2 codes for photo sensitive channel rhodopsin which when stimulated with 473nm (blue light) leads to influx of positive ions (cations) inside the cells thus activating the cell. On the other hand, eNpHR3 codes for halorhodopsin which when stimulated with 596nm (yellow light) leads to influx of chloride ions thus silencing the neurons by hyperpolarization. They found that when these neurons were optogentically activated using blue light, mice responded shorter response more often. Whereas when these neurons were optogentically silenced using yellow light, mice responded longer response more often. These changes in timing behavior were transient as mice timing accuracy was again restored in the absence of light. Thus, the authors were able to manipulate the timing behaviour of the mice using optogenetics.

The study explains the underestimation of time during fun or pleasurable activity where dopamine is supposedly high and overestimation of time during stress and fear where dopamine is low. Although, this study suggests that increased activity in the DA neurons leads to underestimation of time, however there are studies which suggest that increase in dopamine leads to overestimation of time (1,2,3).

Overall, this study happens to be the first to manipulate timing behaviour in mice using optogenetics, giving hope for the future where individuals could one day manipulate their own subjective time.

Source article: Soares, S., Atallah, B. V., & Paton, J. J. (2016). Midbrain dopamine neurons control judgment of time. Science, 354(6317), 1273-1277.

References:

  1. Buhusi, C. V., & Meck, W. H. (2002). Differential effects of methamphetamine and haloperidol on the control of an internal clock. Behavioral neuroscience, 116(2), 291.
  2. Failing, M., & Theeuwes, J. (2016). Reward alters the perception of time. Cognition, 148, 19-26.
  3. Terhune, D. B., Sullivan, J. G., & Simola, J. M. (2016). Time dilates after spontaneous blinking. Current Biology, 26(11), R459-R460.

—-Mukesh Makwana (Doctoral student), mukesh@cbcs.ac.in
Centre of Behavioural and Cognitive Sciences (CBCS), University of Allahabad, India.

Temporal predictability modulates putative midbrain activity: evidence from human EEG

Predictable timing has been shown to modulate the neural processing of auditory stimuli at multiple stages and time scales, e.g. reducing the amplitude of the P50 and N1 potentials in the EEG. The modulatory effects of predictable timing include an enhancement of repetition suppression (see these examples) and omission responses to tones whose identity can also be predicted. However, most of the previously reported modulations of evoked responses occur relatively late, and have primarily been attributed to cortical processing. Can similar modulatory effects of predictable timing be observed at earlier, putatively subcortical stages?

A recent paper by Gorina-Careta et al. addresses this question by focusing on the human auditory frequency-following response (FFR) – a sustained EEG component serving as a proxy for the auditory brainstem response. The FFR signal is phase-locked to the periodic characteristics of the eliciting stimulus with a short delay (~15 ms), and has previously been shown to be sensitive to contextual factors. The authors recorded the FFR at the central electrode (Cz) in response to an auditory sequence consisting of a rapidly repeated syllable /wa/. The F0 formant of the /wa/ syllable – describing the most prominent frequency component of the auditory stimulus – was set to 100 Hz.

Accordingly, the 100 Hz component of the FFR signal was significantly reduced by temporal predictability (although the reported effect sizes were rather modest), suggesting that even very early auditory processing stages can be modulated by predictable timing. However, the modulatory effect of predictability appeared only after several hundred repetitions, indicating that the putative subcortical responses are shaped over the course of learning. While a closer inspection of Figure 1B might suggest that perhaps the most prominent modulation of the 100 Hz FFR component occurs not in the analysed time window (65-180 ms, corresponding to the steady-state part of the FFR) but in a pre-stimulus time window (shown from -40 ms), this is likely due to a contamination of the baseline in the unpredictable condition by stimulus presentation.

The authors also analysed the extent to which timing predictability influences the neural pitch strength, using a metric that quantified the magnitude of EEG phase-locking to the syllable pitch. While this metric also showed a modulation by timing predictability and its sensitivity to stimulus repetition, the pattern of results was opposite to the FFR. Thus, neural pitch strength was stronger under predictable timing, an effect which was especially prominent during the first 200 repetitions and disappeared after 500 repetitions. This interaction was due to a gradual reduction of the neural phase-locking to the stimulus pitch over the course of learning in the predictable condition, and no learning-related differences in pitch strength in the unpredictable condition.

On a methodological note, it would be interesting to see if the effects would be similar – or perhaps more robust and/or consistent over time – had the authors chosen a different F0 of the acoustic stimuli. In this paper, the F0 at 100 Hz falls exactly at the first harmonic of line noise (50 Hz), one of the most prominent artefacts in EEG signals. Thus, especially in the predictable condition (in which the stimulus onset asynchrony was fixed at 366 ms), approximately every third stimulus might be presented at a phase interfering with line noise.

Nevertheless, these results suggest two complementary mechanisms of temporal predictability: an initially increased neural phase-locking to the physical stimulus which disappears over the course of learning, and a gradual suppression of the neural response to the primary stimulus frequency (F0) occurring at later stages of learning. The authors interpret the first result as a reflection of more reliable processing of complex acoustic inputs. Thus, by increasing the signal-to-noise ratio, temporal predictability might facilitate the extraction of characteristic input features and the forming of neural predictions, which in turn suppress the responses to the most predictable (i.e. highly repetitive) aspects of the stimuli. The latter finding might therefore reflect a gradual deployment of neural predictions formed under temporal predictability to lower (subcortical) stages of auditory processing.

Invasive work in the rodent auditory system shows that the modulation of neural responses to predictable (e.g. repetitive) stimuli occurs already at subcortical stages, including the midbrain. This paper suggests that also the temporal predictability of stimuli might influence the short-latency neural responses associated with activity in the auditory brainstem. While invasive recordings might be necessary to establish an unequivocal link between the modulation of neural activity by temporal predictability and specific subcortical structures, these results offer further support for proposals that even the very early stages of sensory processing might be shaped by statistical regularities in the environment.

Ryszard Auksztulewicz, Oxford Centre for Human Brain Activity

Source article: Gorina-Careta N, Zarnowiec K, Costa-Faidella J, Escera C (2016) Timing predictability enhances regularity encoding in the human subcortical auditory pathway. Scientific Reports 6:37405. doi: 10.1038/srep37405.

Time-dependency in perceptual decision-making

Sequential sampling models are a class of models that are widely supported by empirical and modelling studies in perceptual decision-making. These models propose that noisy sensory information for each choice alternative is accumulated over time until a particular decision threshold is reached, which in turn leads to a response associated with that threshold (see Forstmann et al., (2016) for a nice review). Standard sequential sampling models like the drift diffusion model (DDM) assume that this decision process is context-dependent but time-invariant, meaning that both the rate at which the evidence is accumulated and the decision threshold can vary across different contexts but remain fixed over the course of a single decision. One drawback of this assumption is that the time required to make a choice increases with ambiguity in sensory evidence. This can lead to suboptimal behaviour in contexts that require subjects to strike a balance between response speed and accuracy (the speed-accuracy tradeoff), especially when the potential cost of continued deliberation increases with time. Now, however, a paper from Murphy and colleagues (2016) has provided convergent behavioural, electrophysiological and model-based evidence for the presence of a dynamic ‘urgency signal’ during perceptual decision-making which strongly refutes the assumption of a time-invariant decision policy and suggests that human decision-makers may be considerably more flexible than previously thought.

Perceptual decision-making tasks that solely prioritise accuracy rather than the speed of choices do not in principle invoke time-dependency. Even in tasks which should promote a dynamic speed-accuracy tradeoff, human decision-makers have been found to display an accuracy bias whereby choices are slower and more cautious than required, which leads to lower task payoffs on average. Under such conditions, standard sequential sampling models provide a good fit for the data without the need to incorporate a time-dependent component in the decision policy. In a new twist on common experimental designs in the field of perceptual decision-making, Murphy and colleagues (2016) applied an incentive scheme during performance of a standard two-alternative motion discrimination task that laid an especially heavy monetary penalty (10x that of an incorrect decision) on failure to make a decision within a stipulated time (1.4 seconds). In contrast, the magnitude of reward and penalty was the same for correct and incorrect trials, respectively. Thus, failure to make a decision within the temporal deadline cost participants on this task ten correct trials whereas an incorrect choice cost just one correct trial. Such an incentive scheme reduces the accuracy bias and should lead to strong time-dependency, if human decision-makers are capable of it.

Murphy et al. first examined the empirical conditional accuracy functions relating accuracy to reaction time (RT), which provide a window onto variation in the amount of accumulated evidence that subjects required for decision commitment. These functions suggested two phenomena when subjects performed with versus without the deadline on choices: a ‘static’, time-invariant lowering of the required evidence coupled with a gradual decrease in required evidence as time progressed within a single trial. Moreover, approximately zero evidence was required to make a decision around the time of the deadline, which resulted in chance performance when decisions took that long to be made but in very few missed deadlines. The latter findings in particular are hallmarks of time-dependency in the decision process. Mechanistically, these empirical observations may arise from two distinct sources in the framework of a sequential sampling model: a decision threshold might collapse over time within a trial; or, the threshold could remain fixed and some form of additional input (an urgency signal) might instead be added to the evidence accumulation process itself  as the trial progresses. To distinguish between these possibilities, Murphy et al. examined brain activity (in the form of EEG) recorded during task performance. They found that electrophysiological signals in the µ frequency range (8-14Hz), which are thought to reflect building decision-related motor preparation, exhibited both increased  pre-trial baseline activity under speed pressure (corresponding to a static urgency effect) and a dynamic increase in activity over the course of a trial for both the choices (reflecting a time-dependent urgency effect). These observations were further supported by computational modelling showing that a version of the DDM that included an urgency signal with both static and time-dependent components, coupled with a fixed decision threshold, explained the behavioural data far better than the standard DDM without an urgency signal but with a condition-dependent, time-invariant decision threshold.

Equipped with these findings, Murphy et al., (2016) also explored whether time-dependent urgency was present in trials under mild speed-pressure (without any explicit penalty for missed deadlines) by reanalysing data from a different set of experiments. They found that a time-dependent decision policy seemed to be deployed, albeit less severely, even in contexts where speed pressure is mild. This suggests that the assumption of time-invariance may not even hold in standard perceptual decision-making tasks and that time-dependency is an important factor that cannot be ignored in studies of decision-making.

How might the flexible urgency signal described above be generated in the brain? One appealing candidate mechanism that has already received some attention from computational neuroscientists is  modulation of the ‘gain’ or responsivity of the brain circuits thought to carry out neural evidence accumulation. Moreover, several studies have identified that pupil diameter seems to provide a reliable non-invasive index of the activity of low-level neuromodulatory systems that boast diffuse cortical projections and are hypothesised to control global neural gain (see Aston-Jones & Cohen (2005) for a review). Using pupillometry, Murphy et al., (2016) found in a final study that tonic pupil diameter prior to trial onset was higher when subjects performed under the temporal deadline, reflecting the static urgency effect. In addition, phasic, trial-evoked pupil fluctuations revealed a time-dependent increase in pupil size as the deadline approached, suggesting that the time-dependent urgency effect might be achieved through global gain modulation. Formal modelling of the pupil time-series showed that the input to the pupil system during decision formation is a ramping signal that increased monotonically with elapsed decision time under deadline. Lastly, simulations using a simple neural network model provided strength to the hypothesis that global gain modulation is a plausible biophysical mechanism for generating static and time-dependent urgency in the brain.

The above results, though important for decision-making researchers in general, hold equal relevance in timing research. Performing a task such as the one used by Murphy et al. requires subjects to sample and accumulate sensory evidence while also continually updating estimates of the elapsed time since trial onset, thus concurrently recruiting brain regions involved in both decision-making and time perception. The input to the neural system responsible for generating the urgency signal may thus originate from a network of brain regions involved in the estimation of elapsed time (for e.g., the dorso-medial prefrontal cortex).Perceptual decision-making experiments usually assume temporal invariance of the decision policy in a single trial level. The paper by Murphy et al., (2016) shows that this can no longer be the case. As it is well established that distributed and varied brain regions contribute to human cognition in general, it is time that more studies incorporate established theories from various domains (for e.g., time perception and decision-making as in the experiments above) to obtain better insights into the working of human brain.

Source article:

Murphy, P. R., Boonstra, E. & Nieuwenhuis, S. (2016). Global gain modulation generates time-dependent urgency during perceptual choice in humans. Nat. Commun. 7, 13526. doi: 10.1038/ncomms13526.

Articles cited:

Forstmann, B. U., Ratcliff, R. & Wagenmakers, E. J. (2016). Sequential sampling models in cognitive neuroscience: advantages, applications, and extensions. Annu. Rev. Psychol. 67, 641-666.

Aston-Jones, G. & Cohen, J. D. (2005). An integrative theory of locus coeruleus- norepinephrine function: adaptive gain and optimal performance. Annu. Rev. Neurosci. 28, 403-450.

Visual cortex responses reflect temporal structure of continuous quasi-rhythmic sensory stimulation

As interest in the mechanistic roles of neural oscillations and neural entrainment in perception and cognition increases, so does interest in the bounding conditions for entrainment. The degree of temporal regularity is an intuitive feature to consider – entrainment to a completely periodic stimulus is clear, but entrainment to a completely structureless stimulus is impossible by definition. Somewhere in between are behaviorally relevant stimuli, such as music and speech in the auditory modality or lip movements and gestures in the visual modality. A number of papers have used time-domain measures that allow for more dynamic measures of entrainment, such cerebro-acoustic phase lag or mutual information. However, approaches making use of “steady-state evoked potentials” or “frequency-tagging” methods to measure entrainment typically transform long epochs of time-domain neural data to the frequency domain and use the height of peaks in the frequency spectrum to index the strength of entrainment at the corresponding rate. This approach effectively discards dynamics of entrainment and, as demonstrated in a new paper by Keitel, Thut, and Gross in NeuroImage, may lead to underestimations of entrainment strength when the stimulus rate varies over time.

The authors constructed well-controlled visual stimuli for which contrast fluctuated independently in the two hemifields within theta- (4–7 Hz), alpha- (8–13 Hz), or beta-band (14–20 Hz) ranges. At the same time, the frequency of modulation in each hemifield was itself modulated according to random, continuous modulation functions that were uncorrelated across hemifields, leading to “quasi-rhythmic” visual stimulation (and perhaps the first time “quasi-rhythmic” has been concretely, operationally defined!). An attention manipulation (“attend left” vs. “attend right”) allowed the authors to compare (using EEG) entrainment strength and strength of attentional modulation for quasi-rhythmic stimuli with modulations in each frequency band and to compare these data to fixed-frequency sinusoidal modulations in the alpha range (10 Hz on the left and 12 Hz on the right).

Although there are very many interesting findings reported in the paper (and I encourage anyone reading this blog to check out the paper!), I’d like to focus on an important methodological issue that the paper confronts as well as its implications. A critical feature of quasi-rhythmic stimuli in any modality is that the instantaneous frequency wanders around over time. For that reason, converting a whole time series of electrophysiological data to the frequency domain reduces the signal-to-noise ratio for any single frequency compared to fixed-frequency stimulation (and violates the stationarity assumption of the Fourier transform). The authors elegantly demonstrate this by analyzing EEG responses to quasi-rhythmic stimulation in two ways. First, they use an approach based on calculating the cross-coherence between short segments of narrow-band EEG (multi-taper method) and the corresponding segments of the stimulus. This technique leaves temporal dynamics intact, and demonstrates entrainment of frequency-band-specific neural activity to quasi-rhythmic stimuli. Analyzing the same EEG data using an approach that considers only power of short data segments and then averages those frequency-domain representations (if I’ve read that correctly; ignoring the fact that frequency may change over the time course of stimulation) failed to reveal entrainment, and instead looked like the power spectrum that might be expected during a resting-state measurement, regardless of the frequency range of the visual stimulation.

This demonstration potentially reconciles conflicting results in the literature regarding strength of entrainment to perfectly regular versus quasi-rhythmic stimuli. Moreover, this finding highlights the importance of not ignoring the dynamic, nonstationary nature of behaviorally relevant stimuli and the neural activity that synchronizes to such stimuli. Approaches focusing on steady-state evoked potentials and frequency-tagging often convert long stretches of time-domain data to the frequency domain without considering dynamics – which may be a close enough approximation when the stimulus has a single frequency, but certainly doesn’t represent the way that brains work generally or how entrainment to quasi-rhythmic, behaviorally relevant stimuli works more specifically. In order to really understand neural “dynamics” and how they are related to perception and cognition, making use of analysis techniques that don’t obscure dynamics will be critical. I’m optimistic that demonstrations like the current one – that not all analysis approaches preserve the dynamic nature of entrainment to quasi-rhythmic stimuli and that this matters for interpretation – will allow us to better understand the roles of neural entrainment in perception and cognition in naturalistic situations.

–Molly Henry, University of Western Ontario

Temporal statistical regularity results in a bias of perceived timing

Statistical regularity in the stimulus leads to learned expectations that give rise to intrinsic biases affecting the processing of subsequent stimulus information. One unresolved question in timing research is how learned temporal regularity affects the perceived timing of subsequent stimuli. The prevailing hypothesis is that expectations should bias any deviant intervals to be more similar to the regular intervals on which the expectations are learned. Precisely, expectations will have a symmetric performance effect on early and late stimuli.

In a series of experiments, Di Luca and Rhodes (2016) presented subjects with a sequence of isochronous stimuli followed by a test stimulus of varying asynchrony, and subjects are required to report if they perceive the test stimulus as isochronous or anisochronous compared to the initial sequence. They find that when subjects are presented with isochronous stimulus sequence, the expectations learned from the sequence give rise to a bias, termed Bias by expected timing (BET). Under such conditions, the minimum detectable anisochrony in the test stimulus should be greater than the actual BET. This is because BET counteracts the improved detectability of stimuli presented later than expected i.e., stimuli following a long sequence that are presented later than expected are perceptually accelerated against the detectability of asynchrony. However, behavioural results show that perceptual delay is only present at large anisochronies for stimuli presented earlier than expected. Thus, the effect of BET for early stimuli is insufficient to counteract the effect of improved detectability, leading to an asymmetric distribution of responses. To summarise, BET leads to acceleration of stimuli presented at the expected time point or later and a perceptual delay for stimuli presented earlier than expected (figure 1 below). Di Luca and Rhodes (2016) used novel experimental paradigm to obtain a less biased measure of BET in different conditions.

 

figure-1

Figure 1: A: Example trial sequences where participants judged the temporal order of the audiovisual pair presented at the end of a sequence. Top: An audio sequence with the final stimulus presented earlier than expected (negative anisochrony) and light presented before the final audio stimulus (positive stimulus-onset asynchrony). Bottom: A visual sequence with the final stimulus presented later than expected (positive anisochrony) and audio presented before the final visual stimulus (negative stimulus-onset asynchrony). B: Average perception of subjective simultaneity (PSS) corresponding to the stimulus-onset asynchrony at which the audio and visual stimulus are perceived to be simultaneous. The difference between the PSS values on the two curves indicates the BET. If there was no change in perceived timing across presented anisochronies, the PSS curves should be horizontal. Instead, the BET changes as a function of anisochrony as stimuli presented at -80ms are perceptually delayed while stimuli presented at 0 ms and +40ms are perceptually accelerated (from Di Luca and Rhodes (2016)).

Different classes of models that explain how the brain deals with temporal regularities (interval-based models and entrainment models) predict a symmetric performance of BET and cannot explain the asymmetry in performance seen above. Di Luca and Rhodes (2016) explain this counter-intuitive effect using a Bayesian model of perceived timing. A Bayesian framework usually requires two distributions: apriori probability and likelihood function that combine together to estimate the posterior probability distribution. In experimental paradigms described in the paper, subjects could perform the task by comparing the perceived timing of the test stimuli to the expected timing (the apriori probability). The probability of sensing a stimulus after it has occurred is given by the likelihood function and is modelled as a monophasic impulse response function resulting from an exponential low pass filter. As the isochronous sequence progresses in the trial, the initially flat prior updates dynamically after the posterior distribution and becomes increasingly similar to the asymmetric likelihood function (figure 2 below). The asymmetric prior represents the learned expectation and when combined with the asymmetric likelihood pushes the posterior away from the likelihood and towards the prior distribution. The perceived timing is given by the mean of the posterior probability distribution. As shown in figure 2, stimulus presented after or before the expected time leads to an acceleration or delay of perceived time respectively.

 

figure-2

Figure 2: Bayesian model of perceived timing. A: Likelihood probability distributions for audio and visual stimuli presented at t = 0. B: The apriori distribution curves for the next stimulus as the trial progresses. C: Posterior probability distributions obtained by integrating the prior and likelihood distributions for stimuli presented before, at and later than the expected time-point respectively.Perceived timing is the mean of the posterior distribution. Due to the asymmetric shape of prior and likelihood distributions, the posterior is pushed towards the prior distribution resulting in a bias of perceived timing (from Di Luca and Rhodes, 2016).

Now that we have a model that explains the data, the next step is to identify the neural correlates for the Bayesian model proposed above. It has been shown that temporal expectations lead to a desynchronization of alpha-band activity, where the neural response to stimuli is amplified at the expected time point (Rohenkohl and Nobre, 2011). As a result, stimulus presented before the expected time point is thus not amplified (leading to a perceptual delay) whereas stimuli presented at the expected time point or later are accelerated. Other studies have shown that alpha-beta band activity mediates feedback projections in human visual processing (Michalareas et al., 2016) and in a predictive coding framework, feedback connections subserve the signalling of predictions from higher cortical areas to lower cortical areas in the decision-making hierarchy. Taken together, these studies suggest alpha-band activity as a strong candidate for the neurophysiological correlate of BET.

Source article: Di Luca, M. & Rhodes, D. (2016). Optimal perceived timing: Integrating sensory information with dynamically updated expectations. Scientific Reports 6: 28563.

 Articles cited:

Rohenkohl, G. & Nobre, A. C. (2011). Alpha oscillations related to anticipatory attention follow temporal expectations. J. Neurosci. 31, 14076 – 14084.

Michalareas, G., Vezoli, J., van Pelt, S., Schoffelen, J.-M., Kennedy, H. & Fries, P. (2016). Alpha-beta and gamma rhythms subserve feedback and feedforward influences among human visual cortical areas. Neuron, 89, 384 – 397.

Mental context biases retrospective temporal judgements

Our sense of time spans multiple scales; from seconds to minutes and days to years. Judgements for different temporal intervals however do not rely on a single unitary timing system, but instead rely on separate neural networks. For example, judgements within the sub- and supra-second range rely on networks involved in motor control (basal ganglia and cerebellum), whereas brain regions involved in long term memory and spatial navigation (prefrontal cortex and hippocampus) are involved in judgements spanning weeks and years.

A new functional magnetic resonance imaging study, led by researchers from Princeton, fills the gap in our understanding of how we judge time across intermediate timescales. The study examined time perception in the range of minutes by testing how accurately participants could estimate the amount of time that had elapsed between short excerpts taken from a 25-minute, science-fiction radio story.

The study investigated the idea that temporal estimation for intermediate intervals is related to the degree to which events are associated with similar contextual cues. This idea is based on theories of memory which posit that the recency of events can be ascertained by retrieving slowly varying contextual representations associated with global mental states. These representations include external environmental features (i.e., spatial location) and internal states such as goals and emotions. According to this theory, contextual cues bias temporal judgements such that the interval between events containing similar contextual features should be underestimated, whereas the the interval between events containing few contextual features should be overestimated.

To test this theory, participants listened to the radio story while brain activity was measured with functional magnetic resonance imaging. After the scanning session, participants completed a surprise temporal judgement test. Participants were presented two short clips that were either 2 or 6 minutes apart and were asked to estimate the time that elapsed between each excerpt. The degree to which these target clips were associated with similar mental context was estimated by examining brain activity recorded while participants initially listened to each clip. The authors used multi-voxel pattern analysis, a method that exploits distributed patterns of brain activity within a region of interest to measure the neural representations formed by different perceptual and cognitive states. MVPA was carried out by correlating the pattern of neural responses (across voxels) evoked by each clip with the idea that clips that share similar content, should also evoke highly correlated patterns of brain activity.

A region of interest analysis showed that pattern similarity in the right entorhinal cortex was correlated with temporal estimates. That is, clips that evoked similar patterns of brain activity within this region were associated with shorter duration estimates in the temporal judgement test. This result was also found when the correlations between judgements and pattern similarity were calculated for each clip pair across participants, indicating that variations in temporal judgements were not solely due to clips sharing perceptual features (i.e., if clips shared similar music).

Evidence that temporal judgements are based on representations formed in the entorhinal cortex is consistent with this region’s’ role in binding event content (i.e., objects, people, actions) within a broader spatial and temporal context. Indeed, a follow up analysis that examined the auto-correlation of evoked patterns within the entorhinal cortex showed that pattern similarity in this region fluctuated more slowly throughout the story than in neighbouring lateral temporal lobes. Together these results confirm the major predictions of the mental context theory of temporal estimation: temporal judgements are based on slowly varying representations that bind event content within broader contextual cues.

It remains to be seen whether the entorhinal cortex plays a general role in retrospective duration estimates for different tasks, contexts and timeframes. One possibility is that this region is particularly attuned to the temporal relations between events in spoken narratives. Depending on the nature of the story, the temporal relationships between events in a spoken narrative may be somewhat compressed compared to everyday experiences. In this case, it might be expected that the entorhinal cortex usually supports temporal judgements over hours and days in more naturalistic contexts.

Research published by the same group has shown that brain regions appear to be attuned to different temporal frequencies, a finding that most likely reflects the kind of representations formed within a region. It might be possible that temporal judgements for other timeframes (i.e., tens of seconds or hours) or different event content (daily activities, details of a conversation) may rely on other brain regions that are better suited for retrieving key information about different experiences.

 

Bronson Harry

The MARCS Institute, Western Sydney University

Society for Neuroscience 2016

Wow! As always, SfN 2016 was a completely full-on experience. This year was in San Diego, which is obviously the only appropriate location for a conference of that size that is consistently held in October/November. Saturday afternoon featured a ‘Temporal Processing’ poster session, which looked excellent. The session included work from labs around the world presenting on a great range of timing-related topics. Unfortunately (but also fortunately), I presented my own poster during this session and so saw zero other posters. But it’s good to be busy during your own poster session. The highlight for me was that many from the session joined for a post-poster drink, so there was still an opportunity to interact with many presenters and their co-authors, some more scientifically and some more socially. There was also a session on ‘Neural Circuits for Timing, Temporal Processing, and Sequences’ on Tuesday afternoon and another ‘Temporal Processing’ session Wednesday morning. And I’m sure there were uncountable things I missed.

The most inspiring thing I personally saw at SfN was a talk by Flavio Fröhlich in a symposium on “Advances in Noninvasive Brain Stimulation Along the Space-Time Continuum”. Although neither the symposium nor the talk were on timing per se, I think the sky is the limit in terms of applying noninvasive brain stimulation techniques to understand the neural bases of timing, most obviously in the domain of rhythm and beat perception, where neural oscillations and neural entrainment might be manipulated. The talk (“From biophysics to treatment: rational design of non-invasive brain stimulation to modulate thalamo-cortical oscillations”) provided a rapid-fire introduction to work in the Fröhlich lab spanning levels from mathematical/computational modeling of thalamocortical circuits to clinical trials testing the efficacy of noninvasive brain stimulation for post-traumatic stress disorder and depression (and covering everything in between).

All of the work builds from the idea that ongoing neural oscillations can be “picked up”, or entrained, by noninvasive brain stimulation. The novel insight that it provided though, which is simultaneously completely obvious and not obvious at all (at least for me), is that it’s not necessarily sufficient to blast a brain with stimulation in order to cause entrainment or to cause the presence of a neural oscillation of a particular frequency. If the target neural signal isn’t present at the time stimulation is applied, we can’t expect it to be entrained or enhanced. For example, applying noninvasive brain stimulation that mimics the shape of sleep spindles to an awake person doesn’t do much. But, when applied to a sleeping person for whom sleep spindles are naturally present, transcranial alternating current stimulation (tACS) that enhances those sleep spindles actually enhances motor memory, in particular when the stimulation is specially designed to match the sleep spindles of that particular person! Despite being completely mind-blowing, this actually makes complete sense from a dynamical systems perspective.

Given the current hype surrounding the involvement of neural oscillations and neural entrainment in rhythm and beat perception (into which I definitely buy), it seems that a natural next step is in the direction of noninvasive brain stimulation techniques (and in particular what one might think of as “time-domain” techniques, such as tACS). Can we shift the metrical interpretation or phase alignment of the beat in an ambiguous rhythm?; can we disrupt the perception of a beat altogether?; can we improve beat perception for syncopated rhythms or for individuals that are weak beat perceivers to begin with? All of these would contribute to our understanding of how neural oscillations and entrainment are related to beat perception. But I think the key take-away message from the talk is that null results cannot and should not be interpreted as evidence that brain stimulation won’t deliver answers to these questions (a problem very much not specific to brain stimulation or beat perception). Instead, whether the neural “conditions” are right should be at the center of any forays we make into the world of noninvasive brain stimulation. I suggest that this might require paying careful attention to individual differences, which are very apparent in the context of rhythm and beat perception – this is in line with growing interest in personalized medicine. Tuning our approaches by knowing about the neural preferences an individual person in a particular context may allow us to develop more effective treatments to improve, for example, movement and gait in Parkinson’s disease or memory in Alzheimer’s disease.

– Molly Henry, University of Western Ontario