<--Reviewer's response: Thanks for the clarifications!
<--This is a nice short and concise summary of the anatomical, physiological and functional characteristics of the basal ganglia. I had a few minor questions. First, is it worth mentioning the nucleus accumbens, or at least, that the current summary does not apply to this nucleus? Second, is it appropriate to refer to the loops in Figure 4 as 'sensorimotor' loops, given that various of these 'loops' have cognitive functions too? Third, should there be a reference to the potentially opposite effects of dopamine on neural activity at D1 and D2 receptors (e.g. Hernandez-Lopez and colleagues)? Fourth, could the author clarify, in the 'action selection' secion, to what extent the role of the basal ganglia in selection is mediated by dopamine, and how this can be reconciled with views that dopamine disinhibits the direct pathway but inhibits the indirect pathway? The latter (albeit oversimplified) view seems to suggest that dopamine does not 'sharpen' selection, but rather lowers the threshold for all (good and bad) responses. Is the view that dopamine has contrasting effects on the direct and indirect pathways perhaps outdated? Fifth, I would be interested to know whether the author thinks that the basal ganglia is important for selecting representations that have no direct consequences for motor adaptation. Finally, as I understand it, the data suggest that the phasic dopamine signal cannot carry information regarding the identity of sensory events (or about whether they are rewarding or aversive), but rather only regarding their saliency or unpredictability. Dopamine may promote the reselection of responses that immediately precede salient or unpredicted sensory events. Does this mean that dopamine also promotes the reselection of inappropriate responses when these are preceded by salient, unpredicted but aversive sensory events?-->
<--Author's response: Thank you ! I will try (as it turns out unsuccessfully) to keep my comments on each of your “minor” questions brief:
1. Nucleus accumbens. My understanding of relevant literature is that most consider the ventral striatum or nucleus accumbens should be included as part of the general term ‘striatum’. In support of this view Voorn et al (2004) state in their review that….“Anatomy and neurophysiology show that the two striatal areas (caudate/putamen and nucleus accumbens) have the same basic structure and that sharp boundaries are absent”. Following this lead, my intention was that the ventral striatum/accumbens should be included in the general conceptual analysis offered in this article. Thus, the ventral striatal region could be expected to perform the same kind of computational operations on its inputs, from prefrontal cortical and subcortical limbic regions, as the dorsal striatum performs on its sensorimotor input.
2. Sensorimotor loops. Here I didn’t want to push my luck ! The purpose of this figure was to draw a direct comparison between well established cortical-basal ganglia loops and less well established sub-cortical-basal ganglia loops. The concept of cognitive processing in subcortical regions would undoubtedly be contentious in a way that sensorimotor processing is not. Consequently, I sought to compare non-contentious examples of cortical and sub-cortical loops. The point being, that whatever the functional nature of inputs to and outputs from both cortical and subcortical regions, there is a side loop than runs to and returns from the basal ganglia. However, the fundamental difference is that the thalamic relay is on the input side of one (subcortical) and output side of the other (cortical). The functional significance of this difference remains mysterious….at least to me…I guess we need to know more about the subcortical loops.
3. Effects of dopamine on D1 and D2 receptors. In preparing this article I was aware it was exceeding the recommended guide-lines for length. My reading of the relevant literature on the effects of dopamine in the striatum suggested there remains much controversy in this area. Therefore, for the sake of brevity, I thought it best to simply to make reference to the excellent and detailed reviews of this area by Salem Nicola. Should anyone feel called upon to expand this part of the article….please feel free.
4. Dopamine and action selection. Again I think this is too big an issue to be considered (necessarily in detail) in a brief overview of the basal ganglia. First, as will be apparent from the section on Direct and Indirect pathways, I am increasingly sceptical about the value of this conceptualisation and its ability to predict the dynamics of the output nuclei. Secondly, I agree that general tonic activation of DA receptors (e.g. as a results of systemic amphetamine or apomorphine) does not ‘sharpen’ selection; if anything it does the opposite. Thus, with small to intermediate doses the system seems more vulnerable to interrupt in the sense that animals/humans are more easily distracted by sensory events. At high doses, of course, you get the behavioural stereotypies where the system seems abnormally resistant to interrupt. In our models (Gurney et al 2001; Prescott et al 2006), the effect of increased levels of tonic dopamine causes less exclusive selection among the channels with more than one channel becoming disinhibited.
The issue of precisely timed phasic dopamine inputs is different and heavily dependent on the other signals present in the striatum at the time of the arrival of the pulse of dopamine. To my knowledge there is not yet any relevant electrophysiological data on the effects of DA on the response of medium spiny neurones to simultaneous and precisely timed inputs from the cortex and thalamus. Our hypothesis (Redgrave and Gurney 2006 – also see below) is that phasic dopamine should increase the probability of immediately prior attended stimuli and behavioural output being re-selected.
5. Non-motor adaptations: Given that there are non-motor territories of the basal ganglia that have more-or-less the same intrinsic microarchitectures (which appear in the sensorimotor domains to select), the answer would be an unequivocal ‘yes’. Remember, not only can you not do two different things with the same set of muscles at the same time, you cannot think (have an idea) about different things at the same time. The non-motor territories may have a hand in this phenomenon (stream of consciousness ?)…i.e. determining from all the possible things you could be thinking about at any point in time, a winner-that-takes all.
6. Dopamine, good and/or bad. The reviewer has slightly misunderstood our current position. To the best of our knowledge, the largely subcortical afferent sensory processing made available to ventral midbrain dopamine neurones has the capacity to signal unpredicted biologically salient events. The latter can be subdivided into bad and not-bad. Let me explain. In 1999, we thought (Redgrave et al 1999 TINS) that phasic dopamine signalled the unpredicted occurrence of all salient stimuli (good, all novel and bad) with a positive burst of activity – salient determined by whether the stimulus has a representation in the deep layers of the superior colliculus (the brain’s sentinel which provides direct input to substantia nigra pars compacta – Comoli et al 2003). The collicular deep layer response to a novel neutral stimulus habituates within a handful of trials (which is probably why dopamine neurones show a similar rapid habituation); this does not happen if the stimulus is rewarding, punishing or predicts either. We therefore thought that both reward- and punishment-related stimuli would elicit a positive dopamine response, since both have the capacity to block response habituation in the collicular deep layers.
The nice thing about a clear prediction like this is that it can be falsified…i.e. we were wrong. The report by Ungless et al (2003, Science, 303, 2040) together with our own experiments (Coizet et al, 2006, Neuroscience, 139, 1479) demonstrated that dopamine neurones are unequivocally inhibited (at short latency <100ms) by frankly noxious stimuli (pinch and footshock). So, DA neurones exhibit a negative phasic response to aversive stimuli. Presumably, this specialised response to aversive stimuli can occur because the body contains specialised nociceptors. For various reasons, no where in the sensory pathways afferent to dopamine neurones are there comparable specialised ‘reward’ detectors.
Which brings me back to the statement made above − dopamine neurones can signal the occurrence of biological salient events, aversive ones with a negative response, and ‘not bad ones’ (novel or reward related) with positive response. Our 2006 hypothesis (Redgrave and Gurney) is that dopamine provides a teaching signal used to discover if an unpredicted event was caused by something the agent did; and if it was, exactly what has to be operated on in which way to produce the event. Thus, through repetition, the system can converge on a novel action (a movement with a predicted outcome). If the event is not-bad (novel or reward-related) it would be safe for a positive dopamine signal to promote the repetition of recent patterns of attention and motor output to determine agency and action discovery. On the other hand, if the event is bad (aversive…or failure of a predicted reward !) current behavioural should be terminated and not repeated….which is what the negative dopamine response may promote. Perhaps this is why most superstitions are associated with the occurrence of aversive events.
Finally, to distinguish this reinforcement-prediction error hypothesis from standard reward-prediction error hypotheses of phasic dopamine function (e.g. Schultz and Dickinson 2000), I think that ‘real reward-prediction errors’ are computed else where in the brain. Given the range and variety of rewarding events, this is most likely to occur in structures receiving input from sophisticated scene analysing and object recognition systems − my guess, supported by increasing amounts of data concerning representations of the ‘economic value’ of stimuli, would be the frontal cortex and/or limbic structures such as the amygdala. Note that representations of ‘economic value’ can best be generated only after the superior colliculus has initiated a gaze shift to bring an unpredicted event onto the fovea, and its identity subsequently determined….i.e. long after the phasic dopamine signal has come and gone. Although both novel-neutral and reward-related stimuli trigger phasic dopamine responses, the consequences of computing ‘real reward prediction errors’ can be distinguished. If the event is neutral, this information is (somehow) fed back to the superior colliculus to initiate, or allow to proceed, the process of habituation. Alternatively, if the unpredicted event is a reward or a predictor of reward, habituation in the colliculus is blocked. I think it more likely that Law of Effect reinforcement learning is driven by post-gaze shift computations of ‘economic value’ while the discovery of novel action-outcomes is promoted by pre-gaze shift phasic dopamine signalling. Note that in the absence of the latter the former would be impossible. (Please see Redgrave and Gurney 2006 Nature Reviews Neuroscience, 7,967, for references on which this analysis is based).
I hope this answers the points and apologies for length.-->
<-- Author's response to Referee A:
1) In answer to your question concerning the comparative numbers of medium spiny and the different classes of interneurones it seems in rodents 96-97% of striatal neurones are medium spiny neurones. In primates there are proportionally more interneurones; in humans the number may be as large as ~20%. I have amended the text referring the reader to the Tepper and Bolam paper for precise numbers
2) With reference to my point concerning the difficulty of establishing "normal" patterns of input activity in slice and anaesthetised preparations…..and your response that “this was compensated for in the Gruber, Solla, Surmeier & Houk (2003) model, which concluded that DA neuromodulation induces both bistability and nonlinear amplification. Something about this should be added”. I would prefer not to do this, first, because the article is already much longer than recommended by Scholarpedia. Consequently, to discuss this issue in a way to make it accessible to the general reader would entail going into inappropriate levels of detail. Secondly, I know of no biological experiments in slice, or indeed in intact anaesthetised preparations, where appropriately timed motor/attentional efference copy signals converge on the medium spiny neurones with appropriately timed phasic glutamtergic signals from the different thalamic channels, and with dopamine from substantia nigra. In the absence of these “normal” conditions, which the anatomy says must occur, it will be very difficult to determine the precise effect of at least phasic dopamine on the activity of medium spiny neurones. Abnormally timed activation of different subsets of these inputs could be responsible for what appears to be conflicting results in the biological literature.
3) All other suggestions have been gratefully accepted.-->
<-- Author's response to Referee B: All suggestions gratefully accepted.-->
<review>I guess it would also be interesting to comment on models of basal ganglia functioning that involve functions of interval timing and working memory. Such models could be dealt with in a note, perhaps relating them to the already reviewed models of action selection and reinforcement learning. IMHO, the four functions of cortico-striatal circuits are instantiations of a higher-level function. Anyway, references follow:
Beiser DG, Houk JC (1998). Model of cortical-basal ganglionic processing: Encoding the serial order of sensory events. Journal of Neurophysiolocy 79: 3168-3188.
Brown GDA, Preece T, Hulme C (2000). Oscillator-based memory for serial order. Psychological Review 107: 127-181.
Frank MJ, Loughry B, O'Reilly RC (2001). Interactions between frontal cortex and basal ganglia in workin memory: A computational model. Cognitive, Affective and Behavioral Neuroscience 1: 137-160.
Matell MS, Meck WH (2004). Cortico-striatal circuits and interval timing: Coincidence detection of oscillatory processes. Cognitive Brain Research 21: 139-170.
Lustig C, Matell MS, Meck WH (2005). Not "just" a coincidence: Frontal-striatal interactions in working memory and interval timing. Memory 13: 441-448.
Thanks for your attention </review>
<-- Author's response to review.-->
We also have quantitative models, in our case of action selection (e.g. Humphries MD, Stewart RD, Gurney KN. 2006. A physiologically plausible model of action selection and oscillatory activity in the basal ganglia. J Neurosci 26(50):12921-12942), and are currently working on reinforcement models of agency and action discovery. However,all the mechanisms and 'models' proposed in this article on the anatomy and physiology of the basal ganglia are strictly biological and of the qualitative box-and-arrow type. I understand there is to be a separate entry to Scholarpedia specifically on quantitative models of the basal ganglia. This would seem the appropriate place for a full discussion of the articles proposed above.