Talk:Models of visual cortex

I like the general scope of the article. It covers a large body of relevant literature and provides a comprehensive overview of the wide variety of proposed models of the visual cortex. Still, I would like to suggest a few changes. Most importantly, I think the article could use a little restructuring or a short overview of what is to come in the introduction. Currently, it starts with general remarks on the anatomy, visual models, the feed-forward and feed-back modes of information flow and then talks about normative approaches. This is all on a more general descriptive level. Following this, a switch of explanatory level is made and we find descriptions of neuronal circuity and work our way up again via models of visual function to hierarchical models of visual processing. Then, the article dives back into the discussion on feed-back processing and finishes off with remarks regarding the development of new experimental (not computational modeling) techniques. There are two ways to approach this. First, one could start with a functional description of the visual hierarchy (ventral vs dorsal stream and hierarchical organization) and then directly concentrate on the large-scale models (such as the one of the authors) as the main part. From there, one could focus on normative approaches of functional organization and then become more and more detailed, leading to single-cell models. Another option would be to stick to the different levels of explanation, starting with single-cell models, then circuity and lower-level functionality and then finally models addressing the interplay of complete functionally defined regions.

Some more detailed comments:

Architecture and function of the visual cortex

Anatomy

• This section begins with a description of the anatomy of the visual cortex. In how far does the description of the two-streams-hypothesis (dorsal and ventral) faithfully describe the anatomy? Without doubt, this theory and the hierarchical ordering of visual areas are both essential for this article, but maybe at a different place or with a different title (e.g. 'theories on the general functional organization')

• "(about thirty or so in monkeys)" --> Please add a reference here and also please provide the number of areas in humans

Visual Modules

• Here, you make the point that several independent modules exist, which implement different visual functions. You state that "This assumption is based in part on the existence of feature maps for the analysis of color, orientation, depth, motion, spatial frequency, etc.". If this statement is meant to dissociate the typical areas of e.g. the ventral stream (V1,V2,V4,LOC,etc), this statement is problematic. For instance in striate cortex (V1) different organizational maps can be found in this one identical region. Similar principles have been suggested for higher-order areas such as IT (see and potentially cite Op de Beeck et al. 2008). Thus, I would suggest to extend this section and to clarify what is meant by the term 'modules'.

The feedforward-feedback dichotomy

• In this section, you emphasize mostly on feed-forward mechanisms and state that backprojections are neither necessary nor sufficient. The question is, however, neither necessary nor sufficient for what visual function exactly? While the modeling community mostly relies on strict feed-forward processing, back-projections were shown experimentally to be at least necessary for visual awareness (see Pascual-Leone & Walsh 2001, and also Kamitani & Tong 2005). Moreover, we know form studies addressing visual memory that back-projections to V1 might be crucial (Harrison & Tong 2009). Finally, we know of the importance of back-projections from studies addressing visual imagery (see Thomas Serre's own paper "Reading the mind's eye: decoding category information during mental imagery."). Please discuss these topics and cite the appropriate literature.

• When it comes to information processing in the visual hierarchy, then it would also be good to mention the reverse hierarchy theory of Ahissar & Hochstein and the earlier work by Ullman (1995) describing the interplay of feedback and feedforward processing via sequence-seeking.

Models, levels of analysis and biological realism

• For the naive reader, it might be slightly confusing that this section uses the terminology of bottom-up and top-down not in the typical sense of information processing, but in the sense of two types of approaches towards vision. I do not think it would harm the section, if you were to drop the terms and just describe that there are two types of approaches according to which models can be classified.

Normative models

• Normative approaches towards understanding visual processing are an important aspect in the modeling of visual function. I would therefore suggest to extend this section and to add more of the relevant literature.

Models of visual functions

Attention and eye movements

• Small typo here: One of the first computational model --> One of the first computational models

• In this section, you present the feature-reweighting approach towards modeling different tasks (i.e. Wolfe, 2007). Maybe it is worth noting that others have demonstrated the limits of this approach (Betz et al. 2010 JoV) by showing that computational models based on reweighted featured maps can only explain a small fraction of the variance observed in the fixation-behavior or humans in different tasks.

Hierarchical models of visual cortex

• As this is clearly one of the main parts of this article, one should add further models and references. For instance, one could cite the Rolls VisNet2 architecture, as it combines local learning rules with a hierarchical representation.

Backprojections, image inference and how visual cortex really works

• In how far is the case of 'hypothesis-verification' different from what has been described in the reverse hierarchy theory? In both accounts the return to lower levels is required (on demand), as they contain more localized and detailed information. From 'vision at a glance', which is generating hypotheses, to 'vision with scrutiny' for verification purposes seems to be a vital part of both approaches. Please clarify.

State of the field and final remarks

• To be honest, it strikes me as odd, that the final paragraph in an article on modeling visual cortex states that in order to develop better models, one 'has to wait for new experimental techniques to manipulate identified populations of neurons in a reversible, transient, deliberate and delicate manner'. While it is absolutely true that improved experimental techniques will certainly boost and improve the development in the domain of modeling, I would nevertheless argue that we have not yet reached a dead end and that there is still a lot of experimental data that needs to be explained. Thus, I would rephrase this part to emphasize the future work that needs to be done in the domain of modeling.

Reviewer C

The article tries to give a very broad overview over the visual cortex, covering many aspects regarding anatomy, physiology, computational and conceptional models. As such I consider the scope of the article too general, and it would certainly benefit from a stronger focus on its central topic, models of visual cortex. The style of presentation for an encyclopedic article should start from the general and then branch into the details, i.e. rather begin with general considerations on models in computational (visual) neuroscience instead of anatomical details. The overall structure of the article could also be improved by choosing a more self-evident organization of the topics, along general ordering principles like granularity and scale of the models.

More detailed comments:

Architecture and function of the visual cortex

This section may be removed. The examples for ventral/dorsal pathways and modular segregation could be parts of a section dealing with conceptual high-level models. The subsection on "feedforward-feedback dichotomy" should be absorbed into the "operations and circuits" section.

Models, levels of analysis and biological realism

A more comprehensive treatment of the ideas of (Marr & Poggio 1977) would considerably improve this section. The "spiking vs. non-spiking" subsection should be absorbed into "biological realism". In general the section could be improved by a better conceptual treatment of the relation of models and their reality counterpart.

Models of visual functions and hierachical models of visual cortex

The organisation of the two sections is not very well-structured. Actually the section on hierachical models only deals with the author's ventral pathway model and should be a subsection of the "visual functions" section, covering object recognition as a function.

Backprojection, image inference ...

I think that a large part of the discussion in this section actually should go into the final section "state of the field". The more detailed discussion on bottom-up and top-down interaction should be concentrated within one section (cp. several earlier sections) and not scattered across the article.