The effects of color on time perception – Blue stimuli are temporally overestimated

In a paper recently published in Scientific Reports, Sven Thönes, Christoph von Castell, Julia Iflinger, and Daniel Oberfeld investigated whether duration judgments depend on the color (hue) of stimuli to be judged.

As color represents a basic feature of visual stimuli in lab experiments as well as in every-day environments, potential effects of hue on our perception of time are important to be considered. In particular, the well-known effects of arousal on time perception suggest that arousing hues, such as red, induce an overestimation of duration.

In a two-interval duration-discrimination task, the authors investigated whether participants indeed overestimate the duration of red stimuli in comparison to blue stimuli, while controlling for differences in brightness (individual adjustments by means of flicker photometry) and saturation (colorimetric adjustment in terms of the CIELAB color space). The mean duration of the stimuli was 500 ms. Moreover, the participants’ affective reaction (arousal, valence, dominance) towards the color stimuli were measured by means of the Self Assessment Manikin Scales.

Interestingly, the results showed a significant overestimation of the duration of blue compared to red stimuli, even though the red stimuli were rated as being more arousing. The estimated point of subjective equality showed that blue and red stimuli were perceived to be of equal duration when the blue stimulus was in fact 60 ms (12%) shorter than the red stimulus.

These surprising results (high arousal related to temporal underestimation) question arousal to be the main driving factor in the context of color and time perception. Moreover, the precision (variability) of duration judgments, i.e., the duration difference limen, did not differ between red and blue stimuli, questioning also an explanation in terms of attentional processes. The authors propose that specific neurophysiological mechanisms of color processing might be the basis of the effect, which need to be investigated in more detail in future studies.

Importantly, in timing-related visual experiments, it needs to be considered that the hue of the stimuli can affect time perception.

Source article:

Thönes, S., von Castell, C., Iflinger, J., & Oberfeld, D. (2018). Color and time perception: Evidence for temporal overestimation of blue stimuli. Scientific Reports, 8(1688) doi: 10.1038/s41598-018-19892-z

–Dr. Sven Thones (thoenes@ifado.de)

Linking sense of agency to perceived duration

Sense of agency (SoA) is an important feeling associated with voluntary actions, enabling one to experience that he/she is controlling the actions and through them the events in the external environment. Until now, only the distortion of time interval between the action and its consequence (i.e. intentional binding effect) was associated with SoA, but a recent study by Shu Imaizumi and Tomohisa Asai, published in Consciousness and Cognition showed that even perceived duration of consequence is linked with SoA.

To investigate this association, they measured the perceived duration (measure of subjective time) of visual display and the rating for amount of control (explicit measure of agency) as a function of temporal contiguity (between action and visual display) and identity of visual display (being participants own hand vs. someone else’s hand). In each trial, participant performed a complex hand gesture as depicted by the image on the screen. This hand gesture was recorded by the overhead camera and projected on the screen after variable delay. While participants performed this task, their hands were covered so the only visual feedback of their action was the one that they saw on the screen. Participants reported whether they perceived the duration of the displayed video feedback (3000ms) as “short” or “long”. They also reported whether they felt that they controlled the displayed hand, by providing a binary response as “totally agree” or “totally disagree”.

The agency was manipulated in two ways, one by changing the visual display (self vs. others) and second, by manipulating the action consequence delay (50ms, 250ms, 500ms, 1000ms or 1500ms). In half trials, participants saw recording of their own hand (self-condition) and in other half trials, they saw the prerecorded clips of other person hand movements performing similar action (other-condition). Orthogonally, the temporal contiguity between action and visual feedback (50ms, 250ms, 500ms, 1000ms or 1500ms) was also manipulated. Based on prior studies on SoA, it was expected that seeing visual feedback of one’s own hand should elicit stronger SoA compared to seeing someone else hand. Similarly, one should experience a stronger SoA for visual feedback displayed with short delay (50ms, 250ms, or 500ms) compared to longer delay (1000ms or 1500ms). They hypothesized that if SoA influences perceived duration then participants should report “long” judgment more often for conditions that are known to boost SoA.

Results revealed that when the visual feedback display consisted of participant’s own hand, they reported stronger SoA and perceived the duration as longer, for short action outcome delay (50ms, 250ms, or 500ms) and this effect become weaker as the delay become longer (1000ms and 1500ms). Furthermore, the above effect was not observed when display consisted of someone else hand, suggesting the possibility that SoA and perceived outcome duration might be linked. Another similar experiment, investigated the effect of participants own hand projected from first person perspective (upright) vs. second person perspective (inverted). Authors expected that inverted perspective would be treated as non-self and will not influence perceived duration, but surprisingly both inverted as well as upright perspective showed similar effect on perceived duration and agency, suggesting that independent of orientation the visual information regarding one’s own hand is processed in a similar manner.

In conclusion, this study provides evidence that SoA also affects perceived duration and participants perceives the outcome duration to be longer when they feel stronger SoA. However, this study is unclear about the exact mechanism that would explain the observed temporal expansion associated with SoA. Moreover, only single duration was used to evaluate changes in temporal perception. Another recent study published in Scientific Reports by Makwana and Srinivasan, also demonstrated similar temporal expansion associated with intentional action, which was sensitive to temporal contiguity and source of action (intention-based vs. stimulus-based). They demonstrated the intention induced temporal expansion, using multiple durations and paradigms (temporal bisection and magnitude estimation), In addition, they also investigated its underlying mechanism in terms of internal clock (most influential model of time perception), suggesting the role of switch dynamics and not the pacemaker speed, to be involved in such temporal expansion. Thus, these studies overall suggest that intention and intentional action, not only influence the time between the action and the outcome but may also influence other aspects of the outcome events such as its duration, and more studies are required to fully understand in what all ways our perception is distorted  by intentional action.

 

Reference:

  1. Moore, J. W., & Obhi, S. S. (2012). Intentional binding and the sense of agency: a review. Consciousness and cognition21(1), 546-561.
  2. Makwana, M., & Srinivasan, N. (2017). Intended outcome expands in time. Scientific Reports, 7(6305) doi: 1038/s41598-017-05803-1

 

Source article:  Imaizumi, S., & Asai, T. (2017). My action lasts longer: Potential link between subjective time and agency during voluntary action. Consciousness and cognition, 51, 243-257.

 

—Mukesh Makwana (mukesh@cbcs.ac.in),

Doctoral student,

Centre of Behavioural and Cognitive Sciences (CBCS), India.

Expectation, information processing, and subjective duration

A paper recently published in Attention, Perception, and Psychophysics tested an implementation of the temporal oddball illusion (according to which standard stimuli seem shorter than oddball stimuli of the same duration) in a novel context using a novel methodology (musical imagery reproduction). This paper is, to the authors knowledge, the first to test whether the temporal oddball illusion translates from single events to multiple-event sequences, and whether information processing influences this potential translation.

In two experiments, musical chord sequences of varying durations (3.5 s; 7 s; 11.9 s) did or did not contain auditory oddballs (sliding tones), and people listened to the sequences while engaged in either direct temporal or indirect temporal processing. We manipulated information processing by independently varying the task (Experiment 1), the sequence event structure (Experiments 1 and 2), and the sequence familiarity (Experiment 2). The task was either to complete a verbal estimation (“What is the duration of this excerpt?”) or a musical imagery reproduction (“Imagine that excerpt playing back in your head. Re-play it through your head the exact way you heard it play through the headphones, from start to finish. Press the green button to mark the start of the excerpt you’re imagining. Press the red button to mark the finish of the excerpt you’re imagining.”). The sequence event structure was either repeated (the mere repetition of a single chord), coherent (chords progressions that follow the rules of Western tonal harmony), or incoherent (the coherent sequences scrambled such that the chords progressions violated the rules of Western tonal harmony). The sequence familiarity was either familiar (presented during an exposure phase) or unfamiliar (not presented during the exposure phase). Completing a verbal estimation task, and listening to coherent, repeated, and familiar sequences induces direct temporal processing. Completing a musical imagery reproduction task, and listening to incoherent and unfamiliar sequences induces indirect temporal processing.

The main findings were that the sequences containing oddballs seemed shorter and longer than those not containing oddballs when people were engaged in direct and indirect temporal processing, respectively. These results support the dual-process contingency model of short interval time estimation, and can be explained using the notion of an information processing continuum (Zakay, 1993): as attention shifted from counting seconds (direct temporal processing) to listening to music (indirect temporal processing), for example, the effect of oddballs shifted from decreasing the number of seconds counted to increasing the amount of music remembered.

References:

Zakay, D. (1993). Relative and absolute duration judgments under prospective and retrospective paradigms. Attention, Perception, & Psychophysics, 54, 656–664. https://doi.org/10.3758/BF03211789

Source paper:

Simchy-Gross, R., & Margulis, E. H. (2017). Expectation, information processing, and subjective duration. Attention, Perception, & Psychophysics, 1-17. https://doi.org/10.3758/s13414-017-1432-4

 

Reprints are available at https://www.researchgate.net/profile/Rhimmon_Simchy-Gross

– Rhimmon Simchy-Gross

PhD student

Music Cognition Lab @ University of Arkansas

Dopamine encodes retrospective temporal information

A new study published in Cell Reports shows that midbrain dopamine neurons are sensitive to previously experienced time intervals, and that this is likely to be important in terms of reward processing. Midbrain dopamine neurons are frequently discussed in terms of their roles in reward, motivation, and certain forms of learning. However, within the time perception literature, we commonly associate dopamine as modulating the rate of the internal pacemaker. Naturally, these functions of dopamine are not exclusive, and this study makes important progress in integrating them.

Dopamine in reinforcement learning

While early research implicated dopamine as the principle neurotransmitter responsible for the hedonic nature of “liking” something, the contemporary view conceptualises dopaminergic activity as a reinforcement signal that facilitates learning, rather than directly causing pleasure. This is in part due to the classic finding that phasic dopamine activity in the mesolimbic pathway constitutes a reward prediction error (the difference between expected and received reward), commensurate with prescriptive models of reinforcement learning.

During learning, dopamine responses gradually transfer to the earliest predictors of a reward, and after this associative pairing is established, response to the reward itself is reduced or absent. Importantly, this means that these response dynamics are fundamentally sensitive to the expected time of reward delivery.

Further to this, if rewards are delivered at different delays, the phasic responses of dopamine neurons to cues signalling these rewards depend on the duration of the delay (as well as reward probability, magnitude and type). This decreased response to longer reward delays typifies the economic principle of temporal discounting: rewards are devalued as a function of delay until their receipt. In reflecting the reduced value of delayed rewards, these neural responses demonstrate sensitivity to timing and appear to encode the intervals between cues and prospective (i.e. future) rewards.

Dopamine and time perception

In addition to its associations with motivation and reward, as a pharmacological agent, dopamine has been routinely acknowledged to play a significant role in time perception, in what some refer to as the ‘dopamine clock hypothesis1. Two sets of evidence in particular highlight this.

Firstly, non-human animal studies have pharmacologically manipulated dopamine during time perception tasks. When given dopamine agonists (e.g. methamphetamine) during a peak interval procedure, rats’ response rates peak earlier, as if their internal pacemaker was accelerated. When given dopamine antagonists (e.g. haloperidol), peak responses are later, commensurate with a slowing of the pacemaker2.

Secondly, electrophysiological and optogenetic studies of neurons in the substantia nigra (which produces dopamine and has inputs to the striatum) have shown that optogenetic activation or suppression of these neurons result in later and earlier timed responses, respectively. These results respectively reflect a slower or faster internal pacemaker, which is the opposite pattern of results seen in the pharmacological studies.

The present study

From the background above, we can see that dopamine appears to be involved in both time perception and reward processing. However, dopamine neurons have previously only been shown to encode elapsing and future delays. The study from Fonzi et al. questioned whether dopamine signals could also convey information related to retrospective, past delays. For example, do dopamine responses to a reward cue encode how much time has already been invested in the pursuit of the reward?

The researchers developed a Pavlovian conditioning paradigm with two reward cues that provided identical information about an upcoming reward, but differed in terms of how much time had elapsed since the previous reward. One cue was only presented after a 15–25 s wait time (“short cue”), while the other was only presented after a 65–75 s wait time (“long cue”). The researchers trained rats with this design while simultaneously using fast-scan cyclic voltammetry to record dopamine concentration in the nucleus accumbens core. If the dopamine responses to the short and long cues did not differ, then it would seem that dopamine activity only encodes prospective information. On the other hand, if the dopamine response to the long cue was larger than that of the short cue, this could be said to reflect the sunk cost of time. Conversely, if the signal to the long cue was decreased relative to that of the short cue, this could be said to reflect the rate of reward3.

The results showed that within this simple experimental design, dopamine responses to the long cue were decreased relative to short cue, suggesting that dopamine in the nucleus accumbens encodes reward rate. An alternative possibility was that this differing dopamine response reflected differing expectations about the time of delivery – the response to the long cue could be decreased because as time elapses, it is increasingly likely that the cue will be shown (i.e. a change in hazard rate). However, there was no relationship between the dopamine response and the time elapsed within each cue type. Furthermore, when another cohort of rats was trained with only a single cue for both short and long wait conditions, no differences were seen in the cue-evoked dopamine response for different wait times. Both of these results speak against the possibility that the dopamine response reflected the changing likelihood of reward delivery over time.

Notably, the principle finding above relied on a single analysis, and the relative difference between the short and long cues. The authors of the study thus performed a follow up analysis to determine whether this retrospective temporal information could be encoded when the animals were not able to directly compare cues. To do this, they trained an independent cohort of rats with short trials and long trials in separate sessions. Even in these scenarios, the short cue evoked a larger dopamine response than the long cue, which suggested that the encoding of retrospective delays was context-independent.

However, once these rats were exposed to both cues in a mixed session, the response to the short cue was increased. While for most of the above experiments there were no differences in behaviour between the two conditions, this increase in dopamine response to the short cue in this intermixed session was also accompanied by an increase in behavioural responding. This implies that (while elapsed wait times can be learnt independently) the dopaminergic encoding of retrospective delays is not entirely context-independent. It also shows that while there are not generally behavioural differences between the short and long cues, there appear to be changes in behaviour when there are also changes in dopamine response.

In a final analysis, the researchers also investigated the effect of the previous trial type, and the tonic dopamine signals over the waiting time. Firstly, for rats recently switch from the separate sessions to an intermixed session, they found that dopamine responses to short cues were significantly increased when the preceding trial was a long cue trial, compared when the preceding trial was a short cue trial. Similarly, dopamine levels were increased during the waiting period after long cue trials, relative to short cue trials (but only up to 25 s, before the identity of the current trial was known). From around the point that the identity was known (25 s), conditioned responding decreased when the preceding trial was a long cue trial, relative to when it was a short cue trial. One possible implication here is that a decrease in wait time dopamine could promote increased anticipatory responding. This would be consistent with the electrophysiological and optogenetic evidence that reducing dopamine increases pacemaker rate (see above).

It is important to reiterate that the results in the former two paragraphs only applied to the experiments where rats where moved from separate training on the short and long cues to an intermixed schedule. These results therefore represent peculiarities in how these animals learnt and adapted to their new context. Overall, the results of the first experiment are the most important here: phasic dopamine responses encode previous durations and appear to constitute a signal of previous reward rate.

This study compellingly demonstrates how even simple experimental designs can lead to novel and valuable findings. The fact that nucleus accumbens dopamine responses encode reward rate suggests a potential mechanism that could normalise value signals for future rewards, and provide contextual information such as the sunk cost of time.

If cue-evoked dopamine responses have to encode durations over a large range of timescales (potentially over 15 orders of magnitude) one interesting future avenue for research would be to describe the mapping between these dopamine responses and the duration of the delays preceding them, in order to precisely understand how durations are represented. More work needs to be done to comprehensively understand the functions of tonic and phasic dopamine and how they relate to perceived and experienced durations, but this study makes substantial progress toward this goal.


Source paper:

Fonzi, K. M., Lefner, M. J., Phillips, P. E. M., & Wanat, M. J. (2017). Dopamine Encodes Retrospective Temporal Information in a Context. Cell Reports 20(8), p. 1774. doi: 10.1016/j.celrep.2017.07.076


  1. It should be noted that much research into the neurobiology of reward and motivation typically focuses on the mesolimbic dopamine pathway. This is in contrast to time perception research, which is more often related to the nigrostriatal pathway (this is also commonly associated with movement). However, these pathways are not independent and the nigrostriatal pathway has also been shown to be critical for reward processing. ↩︎
  2. When both drugs were delivered simultaneously, rats’ peak responses are similar to that of a control condition. ↩︎
  3. Previous research has suggested that longer timescale tonic dopamine activity encodes reward rate. ↩︎

Intended outcome appears longer in time

We live in a complex and dynamic world where sometimes our action yields the intended (desired) outcomes and sometimes the unintended outcomes, but does our subjective time changes as a function of outcome being intended or unintended. To find the answer, read the recent article by Mukesh Makwana and Prof. Narayanan Srinivasan, published in Scientific Reports.

In a series of five experiments involving temporal bisection task (Exp1-4) and magnitude estimation task (Exp5), they investigated whether participants perceive the duration of intended outcome differently compared to unintended outcome, and if yes then what are its underlying mechanisms.

They reasoned that when a participant intends an outcome, its representation gets activated and this prior self-activated representation would lead to earlier awareness of the intended outcome compared to unintended outcome  extending the temporal experience. Recently, pre-activation account has been used to explain temporal expansion (Press et al., 2014).

To manipulate intentional nature of the outcome they used a simple color choice question. In each trial, amongst the choice of two colors, they asked participants to indicate what color circle they want to see, by pressing the allocated key for that color. After 250ms (Exp1) of the intentional key press, they were randomly presented with circle of either intended color (50% times) or unintended color (50% times) whose duration was randomly manipulated amongst nine levels (300ms to 700ms in steps of 50ms). This was done to reduce or eliminate the sensory-motor prediction between the key press and the color of the outcome circle, so that the effect of intention on the perceived duration of the outcome is not confounded with probability-based prediction. Irrespective of the intentional nature of the outcome, participants were supposed to report whether they perceived the duration of the outcome as closer to short (300ms) or long (700ms) anchor duration that they learnt in training phase before the main experiment. Each individual data was sorted into two conditions i.e. when they get the intended outcome (i.e. Intended condition) and when they did not get the intended outcome ( i.e. unintended condition). Psychometric (Weibull) functions were fitted for this two conditions and bisection points were calculated. Bisection point or point of subjective equality is the measure of shift in temporal perception, where lower values of bisection point in a condition indicate temporal expansion relative to condition with higher bisection point. Results of Exp1 showed that participants perceived the duration of intended outcome as longer compared to unintended outcome.

They also studied whether increase in delay between the intentional action and its outcome affects the intention induced temporal expansion observed in Exp1. So further two experiments were performed with increased delay between action and outcome i.e. 500ms in Exp2 and 1000ms in Exp3. Rest stimuli, apparatus and procedure were identical to Exp1 except that in Exp2 instead red and green, yellow and blue color circles were used. Results showed that the intention induced temporal expansion was observed till 500ms delay but as the delay increased to 1000ms the temporal expansion effect vanished, suggesting that the self-activated representation fades away around 1000ms of the intentional action.

To establish that for the above-observed temporal expansion effect, intentional activation of the representation is necessary and not just priming or instruction-based action is not sufficient Exp4 was performed. In Exp4, instead of intending and selecting what color circle they wanted to see, in each trial participants  were shown color word i.e. RED or GREEN on the screen and they just pressed the corresponding key. Rest procedure, stimuli and analysis was similar to Exp1. Results showed no difference in duration perception between word congruent condition and word incongruent condition, suggesting the importance of intention in the above effect.

Lastly, Exp5 was performed using magnitude estimation paradigm to investigate whether intention affects the time perception by increasing the pacemaker speed or affecting the switch or gating component of the “internal clock model”.  The internal clock model is the most influential classical model used to explain human timing behaviour. If any factor influences the pacemaker speed then as the magnitude of the actual duration increases the difference between two conditions should also increase given a typical “slope effect”. On the other hand, if the switch or gating component is affected then no slope effect is observed. Results showed no slope effect, indicating that intention might influence the switch or gating mechanism.

In conclusion, a series of experiments in this study provides convincing evidence that intention affects temporal perception and participants perceives the intended outcome to be longer in duration compared to unintended outcome. Moreover, this intention induced temporal expansion effect depends on the temporal contiguity between the action and the outcome and it vanishes at 1000ms action-outcome delay. Furthermore, in terms of internal clock, this effect is most probably not due to increase in pacemaker speed, rather opening or closing of the switch seems more probable mechanism. As humans are intentional agents and intentions forms a critical part of daily life, more studies investigating the effects of intention on perception in general should be pursued.

 

Reference:

 

  1. Press, C., Berlot, E., Bird, G., Ivry, R., & Cook, R. (2014). Moving time: The influence of action on duration perception. Journal of Experimental Psychology: General, 143(5), 1787.

 

Source article:  Makwana, M., & Srinivasan, N. (2017). Intended outcome expands in time. Scientific Reports, 7(6305) doi:  10.1038/s41598-017-05803-1

 

—Mukesh Makwana (mukesh@cbcs.ac.in),

Doctoral student,

Centre of Behavioural and Cognitive Sciences (CBCS), India.

 

Summer 2017 Conference Season

Finally back in London, ON, after a slightly extreme summer conference tour: The Neurosciences and Music IV in Boston; preceded by our homegrown satellite, Neural Entrainment and Rhythm Dynamics (NERD, credit Ed Large for name/acronym combo); cuttingEEG in Glasgow; and the Rhythm Perception and Production Workshop (RPPW) in Birmingham [had a little break in between those last two to drive around Scotland with my dad and brother]. All of it was extremely inspiring, but of course too much info for any one human to retain, so I’ll try to summarize what I felt were some of the highlights.

I’m of course extremely biased, but I loved every minute of NERD. It was full-on day of fantastic 7-min talks on rhythm and entrainment punctuated by thought-provoking discussion periods. There were two major things I took away from NERD (I’ll only talk about one in any detail). First, we’re not all speaking the same language a lot of the time. That’s of course a problem that has been and will be around forever to some extent, and it’s also OK. For example, when I talk about “rhythm” or “beat”, that’s not quite the same thing someone else is talking about when they use the same words. A particularly sticky word at the moment is “entrainment”, which seems to mean a lot of things to a lot of different people, despite being very well defined in the math/physics domains. Even things like “beat salience” or “beat strength” are contested terms, making them hard things to study and talk about. The important thing, I think, is that we make sure we’re operationally defining terms in the papers we’re publishing and the talks we’re giving, so that even if we’re using terms differently, we can talk about the same phenomena. It sounds obvious, but this is done surprisingly infrequently, including by myself I’m sure. The second thing I took away, which I won’t discuss here and which very well may be the subject of a future blog, is that rhythmic/temporal expectation effects on behavior are harder to observe than one might think. More on that later.

There were a million interesting talks and posters at both NeuroMusic and RPPW, but here I’ll focus on timing-related issues. Even though I’ve heard various bits of the data before, I was struck (again) by the idea and accumulating evidence that synchrony is social. We need to be synchronized with each other to successfully navigate conversational turn-taking. Toddlers are more likely to exhibit pro-social helping behaviors towards adults that they have moved in synchrony with compared to someone they have moved out-of-sync with. Babies synchronize eye contact with a singer to the beat of the song the singer is producing.

A relatively new focus was on synchronization between brains. New ways to analyze electrophysiological data (using “intersubject synchronization” or “intersubject correlation”) allow us to assess interpersonal neural synchrony. Traditionally, measures of intersubject synchronization don’t necessarily focus on social situations, but nonetheless show that individual brains are more synchronized with each other (i.e., respond more similarly to the same stimulus) when individuals are more engaged with whatever they are watching or listening to. We presented EEG data that we collected from 60 EEG participants, 20 at a time, in slightly different social situations while viewing/listening to a concert. But I’m not here to self-promote. One really interesting twist on this idea was to use noninvasive brain stimulation to force pairs of brains to be either in sync or out of sync with each other. Despite the situation not actual being social (the individuals making up each pair were not able to see or interact with each other, but did have auditory information about the other person’s behavior), pairs of participants synchronized tapping better with each other when their brains were in sync compared to when their brains were out of sync. The moral of the story is that better neural synchronization leads to better behavioral synchronization, which could in turn lead to stronger affiliation in the social domain.

In general, rhythm and timing were very present topics at the conferences I attended this summer. And it seems like the more we know about how brains are actually involved in behavioral synchrony, the better we stand to understand how synchrony is involved in social situations. I look very much forward to seeing how this research evolved over the next years, and in hopefully being a part of it myself.

Olfactory-Visual Sensory Integration Twists Time Perception

During everyday interactions, our senses are bombarded with different kinds of sensory information, which are processed by dedicated sensory systems operating at different temporal sampling scales to form a coherent percept. The question is whether information from one modality (say olfactory) influences the temporal perception of stimulus from other modality (say vision). Although, previous studies have investigated the effect of auditory stimulus on temporal perception of visual stimuli [1, 2], the evidence for the effect of olfactory stimulus on temporal perception of visual stimuli was lacking. A recent study published in Cerebral Cortex, by Prof. Wen Zhou and her lab members (Dr. Bin Zhou, Guo Feng, and Wei Chen), fills this gap and addresses whether odor influences visual temporal sampling and duration perception.

To study the effect of odor on visual temporal sampling, they used a two alternative forced choice chromatic critical flicker fusion (CFF) task with two isoluminant complimentary color images of banana or apple alternating at different frequencies (15, 20, 22.5 & 25 Hz in different blocks) for duration of 400ms (see figure 1, in original paper). In each trial, there were two 400ms flickering interval each flanked by 200ms mask, and separated by 600ms blank between the two intervals. Out of two, only one interval contained the flickering fruit image (either banana or apple) and participants reported the interval that contained fruit image. Along with visual stimuli, in Exp1 (N=16), participants were also exposed to two different odor stimuli (banana-like, amyl acetate 0.02%v/v in propylene glycol; and apple-like, apple flavor Givaudan, in separate blocks). The idea was to check whether the odor congruency influence the temporal sampling (CFF threshold) for the flickering banana or apple images. Results revealed that participants object detection increased significantly when the odor and the object image content matched, even when the task did not demanded any explicit object discrimination or identification, suggesting that sensory congruency between olfactory and visual inputs boosted the corresponding object visibility around CFF. Another analysis by fitting the psychometric function for the two odor conditions, with frequencies on x-axis and accuracy on y-axis, suggested that olfactory-visual congruency also facilitated the visual temporal sampling.

To establish that the above congruency effect is specific to odor and not just semantic information (or context) provided by the odor, they performed two control experiments. In the first control experiment (Exp2A, N=16), participants performed exactly the same task as Exp1 but instead of actual odors, odorless purified water was used and was suggested to participants as diluted banana or apple odor. In the second control experiment (Exp2B, N=16), semantic textual labels, “banana odor” or “apple odor”, were displayed at the center of the screen. In both the control experiments, they did not observe the odor-visual congruency effect, suggesting that presence of odor is important for such sensory integration.

The next question was to find the neural correlates of the odor-visual congruency effect, emphasizing at what level of visual processing the odor starts modulating it. For this, they performed an EEG experiment (Exp3, N=18) using the same stimuli as in Exp1, but modifying the task a bit. In the modified task, only one flickering interval of 400ms was presented flanked by red-green noise mask of 100ms, and participants reported whether the object was present or absent in that trial. All objects (apple or banana image) were presented to participants’ at subliminal frequency. For nine participants flicker frequency of 22.5Hz was used whereas for other nine participants 25Hz was used. Results from time-frequency analysis, revealed that maximum congruency-induced enhancement (i.e. greater normalized power difference) was observed in electrodes over right temporal regions around 150-300ms post stimulus onset. The difference around this time window suggests that during odor-visual congruency, odor starts influencing vision at the stage of object-level processing. Even source-localization analysis indicated the activation of right temporal region which is again known to be involved in object level representations. Thus, these evidences strongly suggest that odor influences the corresponding visual object at the stage of object-level processing.

From the above experiments, it was evident that the odor-visual congruency modulates visual temporal sampling, so the next logical question was whether it also influences the perceived duration of the visual stimuli. To answer this question, in Exp5 (N=24), they used a 2-Alternative Force Choice (AFC) comparison task, in which one image (either apple or banana) was a standard image (500ms) and the other image (either banana or apple) was test image (of varying durations 300, 350, 400, 450, 500, 550, 600, 650, 700ms). Participants reported which of the two images appeared longer in duration. For half participants (N=12) apple image was standard and banana image was comparison, whereas for other half (N=12) banana image served as standard and apple image served as comparison. Participants in both these groups were exposed to banana-like or apple-like odor in separate blocks. Point of subjective equality (PSE) and difference limen (DL) were measured for both the odor conditions. PSE is the measure of perceived duration whereas DL is the measure of temporal sensitivity. A two way mixed ANOVA on PSE values, with odor (banana-like, apple-like) as within-subjects factor and comparison image (banana image, apple image) as between-subjects factor, showed significant interaction. Further post hoc analysis after Bonferroni correction revealed that participants perceived the duration of the image to be longer when the image content and the odor were congruent compared to when they were incongruent. Similar analysis with DL, did not show any significant difference neither for main effects nor for interaction, suggesting that odor modulates only the perceived duration but not the temporal sensitivity.

Again to confirm that the above congruency effect on perceived duration is due to odor, not just because of semantic information (or context) provided by the odor, they performed two control experiments (Exp5A and Exp5B) similar to Exp2A and Exp2B. In Exp5A (N=24) instead of odor, odorless purified water suggested as diluted banana-like or apple-like odor were presented, whereas in Exp5B (N=24) instead of odor, textual labels (“banana odor” or “apple odor”) were presented on the screen. Neither the purified water nor the textual labels, showed the odor-visual congruency effect of perceived duration as seen in Exp4, suggesting the importance of odor in odor-visual sensory integration to modulate visual temporal perception.

In conclusion, this study provides a convincing evidence for the effect of odor on visual time perception, including temporal sampling and perceived duration. In future, it would be interesting to investigate this effect with other time perception paradigms such as magnitude estimation and measure the slope effect, which might help to know whether odor influences the pacemaker speed or the switch/ gating mechanisms in context of “internal clock model”. Moreover, it would be further interesting to investigate whether such odor-visual congruency effect influence the neural correlate of time perception such as CNV (contingent negative variation) component.

References:

1. Romei, V., De Haas, B., Mok, R. M., & Driver, J. (2011). Auditory stimulus timing influences perceived duration of co-occurring visual stimuli. Frontiers in psychology, 2.

2. Yuasa, K., & Yotsumoto, Y. (2015). Opposite distortions in interval timing perception for visual and auditory stimuli with temporal modulations. PloS one, 10(8), e0135646.

Source article: Zhou, B., Feng, G., Chen, W., & Zhou, W. (2017). Olfaction Warps Visual Time Perception. Cerebral Cortex, 1-11.

—Mukesh Makwana (mukesh@cbcs.ac.in),
Doctoral student,
Centre of Behavioural and Cognitive Sciences (CBCS), India.

The P3 and the subjective experience of time

The P3 (or P300) is an event related potential (ERP) component that has commonly been associated with attentional mechanisms and the updating of expectations. It can be evoked by stimuli in oddball paradigms, where a low-frequency stimulus (the “oddball”) is shown intermixed with high-frequency stimuli (the “standards”). For example, a subject in such a paradigm might be shown a series of words in white font (the standard stimuli), and much more rarely, be shown a word in red font (the oddball). This oddball stimulus commonly elicits a P3 at centro-occipital electrodes, one which is more positive in amplitude compared with the standard stimuli.

Oddball paradigms are also commonly cited in the time perception literature: canonically, the duration of oddball stimuli is overestimated, relative to the standard stimuli. Given this commonality, could the P3 be a neural correlate of this temporal distortion? A recent study by Ernst et al. investigated this question explicitly.

This study used an oddball paradigm as described above, where the standard and oddball stimuli varied in duration (from 600 – 1120 ms). After the presentation of an oddball stimulus, subjects were asked whether this was shorter or longer in duration relative to the preceding standards1. The durations of the stimuli were chosen such that the P3 occurred well before the duration judgement was required (and indeed before the termination of the oddball). This way, it was possible to test whether the amplitude of the relatively early P3 predicted the subsequent duration judgement.

The behavioural data confirmed that time overestimations were observed for the oddball stimuli; the typical temporal oddball effect. Analysis also confirmed the primary hypothesis, larger P3 amplitude led to overestimation of the oddball stimuli. Specifically, the P3 was larger for overestimated, compared to correct trials. (The P3 was also larger for correctly classified “long” judgements compared to correctly classified “short” judgements.) Thus, in trials where the oddball was overestimated, the P3 closely resembled that in correctly judged trials where the oddball was actually longer. In sum, these findings suggest that the P3 tracked the perceived duration of the oddball stimuli.

Average ERP amplitudes at electrode Pz, and scalp distributions, split by duration judgement.
Average ERP amplitudes at electrode Pz, and scalp distributions, split by duration judgement.

The researchers also used a multivariate pattern recognition technique in a supplementary analysis. Here, a classifier was trained to discriminate between standard and oddball stimuli on the basis of the ERP data. Classification accuracy was assessed across the different time windows, and showed above-chance accuracy from 125 ms after oddball presentation, reflecting general ERP differences consistent with the scalp distribution of the P3. By assessing the output of the classifier on trials pooled into correct short, correct long, and overestimated durations, they were able to recapitulate the main findings of the ERP analysis (but for a generated, essentially synthetic, estimated ERP) over the time window of 375 – 600 ms. In essence, because the classifier selectively reproduced an ERP response to oddball stimuli, this analysis lessens the likelihood that some other experimental feature was responsible for the difference in duration perception. Similarly, given the scalp distribution and time window, it also provides some data-driven support that the P3 was responsible for the effect, without specifically pre-selecting for the P32.

Overall, this study provides clear support for the hypothesis that the P3 is a neural correlate of the temporal oddball effect. Given that the P3 has been implicated a rather wide variety of phenomena, these phenomena can now also be interrogated from the perspective of time perception. There is also an ample opportunity for future research to ascertain whether the P3 is either necessary or sufficient for temporal distortions in a wider range of paradigms. In general, this study has delineated the P3 as a valuable component of interest for ongoing time perception research that uses EEG.


Source paper:

Ernst, B., Reichard, S. M., Riepl, R. F., Steinhauser, R., Zimmermann, S. F., & Steinhauser, M. (2017). The P3 and the subjective experience of time. Neuropsychologia, 103, 12–19. http://doi.org/10.1016/j.neuropsychologia.2017.06.033


  1. Notably, there were more trials in which the oddball durations were actually shorter than the standards, ostensibly to increase the number of overestimations. One potential issue is that participants may have tried to balance their proportion of short and long judgements, resulting in some “overestimations” that were due to a decision bias, rather than a perceptual bias. ↩︎
  2. It should be noted, however, that the classifier was trained on all of the electrodes, and not just those located centro-occipitally. ↩︎

Implicit variations of temporal predictability: Shaping the neural oscillatory and behavioral response

Anyone accessing this blog probably doesn’t need to be convinced that the ability to predict the timing of upcoming, behaviorally relevant stimuli is important for our ability to perceive and interact with the world. Although I’m quite rhythm-centric, it’s obvious that there are multiple ways in which we can estimate when something important might occur. For example, when the occurrence of an event is inevitable within a specific time window, its probability of occurrence usually increases as a function of time according to what is referred to as a “hazard function” (think of the probability that a car will eventually break down as you keep driving it). However, it’s also possible to engineer distributions for which the probability of occurrence is centered on a particular time point with a small or large amount of variability. The question is then about the neural mechanisms on which this type of temporal predictability (where the event usually occurs after about 1 second, for example) is based.

A recent EEG study by Herbst and Obleser examined the behavioral and neural differences between more and less temporally predictable situations, where temporal predictability had to be learned implicitly by the participants. The task was a pitch categorization task, in which a single tone was presented on each trial, and participants indicated whether it was high or low. The trick was that the time interval between a “cue” that the trial had started and the “target” (to-be-categorized tone – which was importantly embedded in noise to make the task more difficult) was varied according to distributions that made the target more or less temporally predictable. I’ll focus here on their Experiment 2, in which short blocks were presented in randomized order where the target timing was strongly predictable, weakly predictable, or not predictable.

Behaviorally, classical foreperiod effects made it clear that the basic experimental design worked as planned – reaction times decreased with increasing foreperiods (the later the target, the faster the RT). However, the condition-specific behavioral effects (or lack thereof) call into question whether the elegant experimental design (that involved completely implicit learning of temporal predictability) worked as well would have been hoped. The size of the foreperiod effect was indeed larger for temporally predictable compared to unpredictable conditions. However, the critical interaction was actually decidedly nonsignificant. Given that I might have rather expected some benefit at the expected time for the predictable conditions, rather than just a steeper foreperiod effect, I leave it up to the reader to judge whether they are sufficiently convinced by the behavioral results.

However, the some of the neural effects do seem to solidly indicate that implicitly learned temporal predictability was doing something. For example, P2-ish ERP magnitudes decreased with temporal predictability, and a later negative deflection increased in magnitude with temporal predictability. Maybe most interesting, dynamic changes in alpha power seemed to anticipate the expected target onset – alpha power increased briefly after cue onset, then decreased below baseline, and seemed to rebound back to baseline levels in anticipation of target onset. This effect was more obvious for temporally predictable compared to unpredictable conditions. [Of course, this begs the question why getting alpha back to baseline (to a zero-value) would be good for performance.]

For what should have been the most interesting neural dependent measure though, the results are confusing. The authors hypothesized (as I would have), that phase consistency across trials in low-frequency bands (esp. delta, ~0.5–4 Hz) would be higher prior to predictable than undpredictable targets. The reason is that temporal predictability allows low-frequency neural oscillations to get into the right phase at the right time for upcoming stimuli, which might be exactly why we perceive predictable things better than unpredictable things. This goes for paradigms using rhythmic stimuli to entrain low-frequency oscillations as well as more classical foreperiod-style paradigms that vary temporal predictability of a target in a more interval-based fashion. Turns out, Herbst and Obleser observed exactly the opposite – delta phase consistency was reduced for predictable compared to unpredictable schemes (though this difference did occur just after the cue and wasn’t necessarily present leading up to the target when it would have been expected).

With respect to delta phase, there are several possible explanations for the surprising results (that delta phase was less concentrated for predictable situations). First, the authors took great care not to contaminate the pre-target time window with target-evoked responses. By removing the target-evoked ERPS before time–frequency transformation, they may have removed an artifact that has been present in previous studies. Second, the authors took greater care than any paper of which I’m personally aware to not just manipulate foreperiod, but to randomize inter-trial intervals in a way that wouldn’t allow for entrainment to the low-frequency pace of the task itself. To my knowledge, all studies of the neural underpinnings of temporal preparation (except for the one being discussed here) using a fixed or jittered inter-trial interval have never taken such care to abolish an overall experimental pacing. Nonetheless, I still would not have expected opposite phase-consistency results.

In any case, I think the paradigm – where temporal predictability had to be learned entirely implicitly – is remarkably clever and can be used in future work to truly understand the neural mechanisms underlying temporal predictability that is not entrainment-based (i.e., based on rhythm). Given recent work moving in this direction, this work carefully removing rhythmicity (here, of the task itself) and eliminating evoked responses that could contaminate phase-concentration measures, should be used as an example of thoughtful experimental design.

Time perception in schizophrenia

The following is a guest post from Sven Thönes at Johannes Gutenberg-Universität Mainz, Germany.

Over the last decades, numerous studies have reported that the perception of time and the basic processing of temporal information is distorted in patients suffering from schizophrenia. The investigation of timing in schizophrenia is of particular interest because the notion of mistimed information transfer by Andreasen, et al. (1999), which is one of the most popular theories on the cognitive impairments and clinical outcomes in schizophrenia, assumes that distorted temporal processing may underlie the patients’ symptoms.

In a recent meta-analytical review published in Clinical Psychology Review, Sven Thoenes (Leibniz Research Centre for Working Environment and Human Factors) and Daniel Oberfeld (Johannes Gutenberg-University Mainz) reevaluated the data from 957 patients with schizophrenia and 1060 healthy control participants, provided by 68 studies on timing in schizophrenia from the past 65 years. The original studies applied a large variety of different temporal tasks, such as time estimation, production, reproduction, and duration discrimination, as well as the detection of temporal gaps and judgments of temporal order or simultaneity. Importantly, the reported behavioral measures represented different aspects of temporal performance, indexing between-group differences either in accuracy (i.e., signed deviation of the duration judgments from the veridical value) or precision (i.e., variability of the judgments or sensitivity in a discrimination task).

In their analyses, Thoenes and Oberfeld differentiated between different duration ranges and different temporal tasks representing either time perception (judgments of time intervals) or basic temporal processing (e.g., judgments of temporal order) as well as between effects of schizophrenia on accuracy and precision.

Interestingly, independent of the specific temporal tasks and interval durations used, the results clearly demonstrate that both time perception and basic temporal processing are less precise (more variable) in patients with schizophrenia (Hedges’ g > 1.00), whereas effects of schizophrenia on the accuracy of time perception are rather small and task-dependent. These results are in accordance with the theoretical assumption of mistimed information transfer and the notion of a more variable internal clock in patients with schizophrenia, but not with a strong effect of schizophrenia on clock speed.

The review suggests that future research needs to investigate to what degree the impairment of temporal precision may be due to clock-unspecific processes, such as general cognitive deficiencies in schizophrenia. Moreover, research should aim at combining established experimental and phenomenological approaches in order to gain a broader understanding of the specific temporal distortions in schizophrenia.

While research on time perception and temporal processing may be interesting as such, the recent meta-analysis shows that it may also be supportive for the generation and testing of theories in adjacent (cognitive) domains, such as clinical psychology and neuropsychology.


Source article:

Thönes, S. & Oberfeld, D. (2017). Meta-analysis of time perception and temporal processing in schizophrenia: Differential effects on precision and accuracy. Clinical Psychology Review, 54, 44-64. http://dx.doi.org/10.1016/j.cpr.2017.03.007

References

Andreasen, N. C., Nopoulos, P., O’Leary, D. S., Miller, D. D., Wassink, T., & Flaum, L. (1999). Defining the phenotype of schizophrenia: Cognitive dysmetria and its neural mechanisms. Biological Psychiatry, 46, 908-20.