Cortical Kernel

 

 

In this body of research questions were raised about the consequences of the interaction of temporal and spatial nonlinearities as well as about concerns over contemporary linearity metrics. There were enough nonlinearities in both simple and complex cells to raise questions about the utility of the existing linear/nonlinear classification. The effects of a frequency doubling, full-wave rectification temporal nonlinearity in Y/complex cells versus a threshold nonlinearity in X/simple cells needed to be assessed. The responses of simple cells have been shown to be well modeled by a ‘soft threshold’. The effects of known earlier nonlinearities observed to occur in more distal visual cells can be observed being cancelled or compensated for, resulting in very linear processing within the simple cells. It could be the case that the difference in the mechanisms across cell types reflects an attempt by the visual system to compensate for fundamental limitations in neuronal processing in ways best suited to the different properties being extracted by the different cell types. This research demonstrates that there is a need to address the functional consequences different forms of nonlinearities observed in the response of cells in the visual pathway (e.g., thresholding, response compression, and full-wave rectification). For example, full-wave rectification presents with significant even-order terms above second-order, what might this additional distortion mean to the accuracy of energy models? Specifically, total system requirements such as mechanisms to support the acquisition and transport of visual information must be considered in coordination with what must be done to process that information.

 

Additionally, how the presence of nonlinearities affects the different visual modalities (form, motion, depth, color) should also be incorporated in the new vision models. There is a need to extend the examination of the effects of different nonlinearities in each of these modalities by not only analyzing the distortions in the physiological data but by integrating them with the psychophysical results as well. A cell’s response needs to be viewed with respect to the properties that cell is designed to extract and to process. This activity does not have to maintain a veridical representation of all the information in its field of view. To properly assess nonlinearities it must be determined what information is being extracted in the different known pathways (e.g., in simple or in complex cells). The results obtained here demonstrate that it is important not to always hold a rigid, mathematical definition, but to also consider the possibility of methods to extract, cancel, or compensate for nonlinearities (e.g., can full-wave rectification provide a response cushion that mitigates the effects of a threshold nonlinearity?). What might the different nonlinearities mean for feature hierarchies as compared to population responses? Some general nonspecific nonlinearities might be easier to add and then later remove than other nonlinearities more specifically linked to response features or values.

 

In contrast, the results we presented at ARVO (1980) demonstrated that as activity increased, temporal nonlinearity decreased suggesting that a balance is being sought between rectification and response compression in these cells. The potential for neural adaptation as cell activity increases raises issues over what constitutes perceptual linearity in such non-stationary systems. It all depends on what is being extracted – absolute luminance or the contrast of the intrinsic surface reflectance.

 

Some of these quarter century-old data and issues continue to influence debates in the present day. How do the physiological nonlinearities observed in the visual pathways relate to Rust and Movshon’s attachment to artifice? Consider the effects of shifting the operating point of these systems by using natural versus analytic stimuli. The stationarity of the visual system underlies most assumptions related to the ability to generalize results. Where do these nonlinearities observed in the Wiener data fit in with the New Standard Model? Can the argument to use ‘natural’ images to characterize components of the visual system be reduced to defining what is necessary to maintain an appropriate operating point? How might the temporal dynamics of gain control, time dependent adaptation and learning implement advantages equivalent to going au natural? Can the New Standard model take the responses observed with simple, analytic stimuli and extrapolate those data to obtain responses appropriate for the more complex real-world stimuli?

 

Operating point issues aside, wasn’t the greatest accomplishment of the Old Standard Model the degree to which the physiological data could be made consistent and integrated with the corresponding psychophysical data? It would be good to see a comprehensive New Standard Model that demonstrates the magnitude and significance of all the perceptual consequences that follow from the new crop of physiological nonlinearities.