A curated collection of resources and research on vision neuroscience, with a focus on efficient neural representation and active vision following the Sharpening Our Sight workshop at CoSyNe 2024.
This is a collaborative work-in-progress. Please contribute via PRs!
- WEBVISION. The Organization of the Retina and Visual System
Helga Kolb, Ralph Nelson, Eduardo Fernandez and Bryan Jones - The New Visual Neurosciences (2013)
John S. Werner and Leo M. Chalupa - The Computational Neuroscience of Vision (2002)
Edmund T. Rolls and Gustavo Deco - Vision Science: Photons to Phenomenology (1999)
Stephen E. Palmer - Retinal Computation (2021)
Greg Schwartz - Vision (1982)
David Marr - Active Vision: The Psychology of Looking and Seeing (2003)
John M Findlay, Iain D Gilchrist - The Ecological Approach to Visual Perception (1979)
James J. Gibson - Eye movements and vision (1967)
Alfred L. Yarbus
External Lists
- Visual Cortex and Deep Networks: Learning Invariant Representations (2016)
Tomaso Poggio & Fabio Anselmi - Spikes, Exploring the Neural Code (1996)
William Bialek, Rob de Ruyter van Steveninck, Fred Rieke and David Warland - Principles of Neural Design (2017)
Peter Sterling and Simon Laughlin - Fundamentals of Computational Neuroscience, 3rd edition (2022)
Thomas P. Trappenberg - An Introductory Course in Computational Neuroscience (2018)
Paul Miller - The Computational Brain, 25th Anniversary Edition (2016)
Patricia S. Churchland and Terrence J. Sejnowski - Visual Population Codes. Toward a Common Multivariate Framework for Cell Recording and Functional Imaging (2011)
Nikolaus Kriegeskorte and Gabriel Kreiman - Theoretical Neuroscience. Computational and Mathematical Modeling of Neural Systems (2005)
Peter Dayan and Laurence F. Abbott - Bayesian Brain. Probabilistic Approaches to Neural Coding (2011)
Kenji Doya, Shin Ishii, Alexandre Pouget and Rajesh P.N. Rao - Dynamical Systems in Neuroscience. The Geometry of Excitability and Bursting (2010)
Eugene M. Izhikevich
- Neuromatch Academy, Computational Neuroscience (2023)
- Neural Computation @ UC Berkeley (2022)
Bruno Olshausen
- MedARC Neuroimaging and AI Lab (2024)
- NeuroAI @ UCL (2024)
- Swartz Seminar Series @ NYU (2024)
- van Vreeswijk Theoretical Neuroscience Seminar (2024)
- Cambridge Neurotech Seminars (2024)
Abstract
Vision is remarkably efficient, capable of rapidly perceiving rich visual detail despite limited computational resources. To understand this feat, we must consider both how the visual system represents complex natural stimuli efficiently, as well as how active perceptual processes guide selective attention. This workshop brings together leading researchers investigating efficient representation strategies and goal-directed mechanisms of natural vision.
To explore representational frameworks for natural stimuli, this workshop will overview different stimulation and analysis approaches used across visual areas to study complex natural stimuli. In parallel, it will examine how active perceptual processes shape efficient representation. Talks will explore how the visual system optimizes representation based on motivational drivers and behavioral objectives within natural environments. This goal-directed approach helps vision prioritize the most behaviourally relevant stimuli.
By integrating theory, machine learning, and experimental data, this interdisciplinary workshop aims to advance our conceptualization of vision as both an efficient representational and an active, goal-directed process of sense-making. Achieving a deeper understanding of these fundamental aspects of perception has implications for both neuroscience and developing more human-like computer vision.
We selected two relevant papers from each speaker at our Cosyne workshop, with that we would like to kickstart this repository hosting relevant papers in vision neuroscience, leaning towards computation and intersections between active vision and efficient neural representations.
-
Cris Niell: Neural dynamics underlying active vision in the mouse
- Movement-Related Signals in Sensory Areas: Roles in Natural Behavior (2020)
Philip RL Parker, Morgan A Brown, Matthew C Smear, Cristopher M Niell - Natural behavior is the language of the brain (2022)
Cory T Miller, David Gire, Kim Hoke, Alexander C Huk, Darcy Kelley, David A Leopold, Matthew C Smear, Frederic Theunissen, Michael Yartsev, Cristopher M Niell
- Movement-Related Signals in Sensory Areas: Roles in Natural Behavior (2020)
-
Carsen Stringer: Towards a simplified model of primary visual cortex
- Spontaneous behaviors drive multidimensional, brainwide activity (2019)
Carsen Stringer, Marius Pachitariu, Nicholas Steinmetz, Charu Bai Reddy, Matteo Carandini, Kenneth D Harris - High-precision coding in visual cortex (2021)
Carsen Stringer, Michalis Michaelos, Dmitri Tsyboulski, Sarah E Lindo, Marius Pachitariu
- Spontaneous behaviors drive multidimensional, brainwide activity (2019)
-
Christopher Summerfield: How prospection skews the human representation of space during navigation
- Goal-seeking compresses neural codes for space in the human hippocampus and orbitofrontal cortex (2023)
Paul S. Muhle-Karbe, Hannah Sheahan, Giovanni Pezzulo, Hugo J. Spiers, Samson Chien, Nicolas W. Schuck, Christopher Summerfield - Learning to count visual objects by combining" what" and" where" in recurrent memory (2022)
Jessica AF Thompson, Hannah Sheahan, Christopher Summerfield
- Goal-seeking compresses neural codes for space in the human hippocampus and orbitofrontal cortex (2023)
-
Stuart Trenholm: On assigning semantic meaning to the preferred stimulus of a visual neuron
- The feature landscape of visual cortex (2023)
Rudi Tong, Ronan da Silva, Dongyan Lin, Arna Ghosh, James Wilsenach, Erica Cianfarano, Pouya Bashivan, Blake Richards, Stuart Trenholm - Parallel Mechanisms Encode Direction in the Retina (2011)
Stuart Trenholm, Kyle Johnson, Xiao Li, Robert G. Smith, and Gautam B. Awatramani
- The feature landscape of visual cortex (2023)
-
Wiktor Młynarski: Interacting with the environment through flexible sensory codes
-
Andreas Tolias: A Neuro-AI approach to decrypting neural representations
- Towards a foundation model of the mouse visual cortex (2023)
Eric Y Wang, Paul G Fahey, Kayla Ponder, Zhuokun Ding, Andersen Chang, Taliah Muhammad, Saumil Patel, Zhiwei Ding, Dat Tran, Jiakun Fu, Stelios Papadopoulos, Katrin Franke, Alexander S Ecker, Jacob Reimer, Xaq Pitkow, Fabian H Sinz, Andreas S Tolias - State-dependent pupil dilation rapidly shifts visual feature selectivity (2022)
Katrin Franke, Konstantin F Willeke, Kayla Ponder, Mario Galdamez, Na Zhou, Taliah Muhammad, Saumil Patel, Emmanouil Froudarakis, Jacob Reimer, Fabian H Sinz, Andreas S Tolias
- Towards a foundation model of the mouse visual cortex (2023)
-
Sylvia Schröder: The impact of behavior and internal states on subcortical visual processing
-
Jin Hwa Lee: CEBRA Tutorial on Vision Dataset
- Dynamic causal modelling of eye movements during pursuit: confirming precision-encoding in V1 using MEG (2016)
RA Adams, M Bauer, D Pinotsis, KJ Friston - A computational learning theory of active object recognition under uncertainty (2013)
A Andreopoulos, JK Tsotsos - Neural network simulations of the primate oculomotor system III. An one-dimensional, one-directional model of the superior colliculus (1998)
A Bozis, AK Moschovakis - Present concepts of oculomotor organization (1998)
U Büttner, JA Büttner-Ennever - A neural basis for visual search in inferior temporal cortex (1993)
L Chelazzi, EK Miller, J Duncan, R Desimone - Control of goal-directed and stimulus-driven attention in the brain (2002)
M Corbetta, GL Shulman - Saccade target selection and object recognition: Evidence for a common attentional mechanism (1996)
H Deubel, WX Schneider - Brain circuits for the internal monitoring of movements (2008)
Marc A. Sommer and Robert H. Wurtz - Yarbus, eye movements, and vision (2010)
Benjamin W Tatler, Nicholas J Wade, Hoi Kwan, John M Findlay, and Boris M Velichkovsky - Neuronal mechanisms of visual stability (2008)
Robert H. Wurtz
- Representation Learning in Sensory Cortex: a theory (2014)
Fabio Anselmi and Tomaso Poggio - Understanding Human Object Vision: A Picture Is Worth a Thousand Representations (2023)
Stefania Bracci and Hans P. Op de Beeck - Neural Representation and Neural Computation (1990)
Patricia S. Churchland, Terrence J. Sejnowski - Neural Representation and the Cortical Code (2000)
R. Christopher deCharms and Anthony Zador - The neural representation of visual space (1977)
N. Drasdo - Categorical Representation of Visual Stimuli in the Primate Prefrontal Cortex (2001)
DAVID J. FREEDMAN, MAXIMILIAN RIESENHUBER, TOMASO POGGIO, AND EARL K. MILLER - Shape representation in the inferior temporal cortex of monkeys (1995)
Nikos K. Logothetis, Jon Pauls, Tomaso Poggio - Representation of local geometry in the visual system (1987)
J. J. Koenderink & A. J. van Doorn - The singularities of the visual mapping (1976)
J. J. Koenderink & A. J. van Doorn - Neural representations for object perception: structure, category, and adaptive coding (2011)
Zoe Kourtzi, Charles E Connor - Visual information processing: the structure and creation of visual representations (1980)
D. Marr - Natural Image Statistics and Neural Representation (2001)
Eero P Simoncelli and Bruno Olshausen - Invariant Recognition Shapes Neural Representations of Visual Input (2018)
Andrea Tacchetti, Leyla Isik, and Tomaso A. Poggio - Invariant recognition drives neural representations of action sequences (2017)
Andrea Tacchetti, Leyla Isik, Tomaso Poggio
- Neural population control via deep image synthesis (2019)
Pouya Bashivan, Kohitij Kar, James J DiCarlo - Deep convolutional models improve predictions of macaque V1 responses to natural images (2019)
Santiago A Cadena, George H Denfield, Edgar Y Walker, Leon A Gatys, Andreas S Tolias, Matthias Bethge, Alexander S Ecker - Diverse task-driven modeling of macaque V4 reveals functional specialization towards semantic tasks (2022)
Santiago A Cadena, Konstantin F Willeke, Kelli Restivo, George Denfield, Fabian H Sinz, Matthias Bethge, Andreas S Tolias, Alexander S Ecker - How well do deep neural networks trained on object recognition characterize the mouse visual system? (2019)
Santiago A Cadena, Fabian H Sinz, Taliah Muhammad, Emmanouil Froudarakis, Erick Cobos, Edgar Y Walker, Jake Reimer, Matthias Bethge, Andreas Tolias, Alexander S Ecker - Deep neural networks rival the representation of primate IT cortex for core visual object recognition (2014)
Charles F Cadieu, Ha Hong, Daniel LK Yamins, Nicolas Pinto, Diego Ardila, Ethan A Solomon, Najib J Majaj, James J DiCarlo - Simulating a primary visual cortex at the front of cnns improves robustness to image perturbations (2020)
Joel Dapello, Tiago Marques, Martin Schrimpf, Franziska Geiger, David Cox, James J DiCarlo - A rotation-equivariant convolutional neural network model of primary visual cortex (2019)
Alexander S Ecker, Fabian H Sinz, Emmanouil Froudarakis, Paul G Fahey, Santiago A Cadena, Edgar Y Walker, Erick Cobos, Jacob Reimer, Andreas S Tolias, Matthias Bethge - Population code in mouse V1 facilitates read-out of natural scenes through increased sparseness (2014)
E. Froudarakis, P. Berens, A. S. Ecker, R. J. Cotton, F. H. Sinz, D. Yatsenko, P. Saggau, M. Bethge, A. S. Tolias - Controlling perceptual factors in neural style transfer (2017)
Leon A Gatys, Alexander S Ecker, Matthias Bethge, Aaron Hertzmann, Eli Shechtman - Context-dependent selectivity to natural images in the retina (2022)
Matías A Goldin, Baptiste Lefebvre, Samuele Virgili, Mathieu Kim Pham Van Cang, Alexander Ecker, Thierry Mora, Ulisse Ferrari, Olivier Marre - Neural system identification for large populations separating “what” and “where” (2017)
David Klindt, Alexander S Ecker, Thomas Euler, Matthias Bethge - Brain-like object recognition with high-performing shallow recurrent ANNs (2019)
Jonas Kubilius, Martin Schrimpf, Kohitij Kar, Rishi Rajalingham, Ha Hong, Najib Majaj, Elias Issa, Pouya Bashivan, Jonathan Prescott-Roy, Kailyn Schmidt, Aran Nayebi, Daniel Bear, Daniel L Yamins, James J DiCarlo - Cornet: Modeling the neural mechanisms of core object recognition (2018)
Jonas Kubilius, Martin Schrimpf, Aran Nayebi, Daniel Bear, Daniel LK Yamins, James J DiCarlo - Generalization in data-driven models of primary visual cortex (2021)
Konstantin-Klemens Lurz, Mohammad Bashiri, Konstantin Willeke, Akshay K Jagadish, Eric Wang, Edgar Y Walker, Santiago A Cadena, Taliah Muhammad, Erick Cobos, Andreas S Tolias, Alexander S Ecker, Fabian H Sinz - Task-driven convolutional recurrent models of the visual system (2018)
Aran Nayebi, Daniel Bear, Jonas Kubilius, Kohitij Kar, Surya Ganguli, David Sussillo, James J DiCarlo, Daniel L Yamins - A deep learning framework for neuroscience (2019)
Blake A Richards, Timothy P Lillicrap, Philippe Beaudoin, Yoshua Bengio, Rafal Bogacz, Amelia Christensen, Claudia Clopath, Rui Ponte Costa, Archy de Berker, Surya Ganguli, Colleen J Gillon, Danijar Hafner, Adam Kepecs, Nikolaus Kriegeskorte, Peter Latham, Grace W Lindsay, Kenneth D Miller, Richard Naud, Christopher C Pack, Panayiota Poirazi, Pieter Roelfsema, João Sacramento, Andrew Saxe, Benjamin Scellier, Anna C Schapiro, Walter Senn, Greg Wayne, Daniel Yamins, Friedemann Zenke, Joel Zylberberg, Denis Therien, Konrad P Kording - Brain-score: Which artificial neural network for object recognition is most brain-like? (2018)
Martin Schrimpf, Jonas Kubilius, Ha Hong, Najib J Majaj, Rishi Rajalingham, Elias B Issa, Kohitij Kar, Pouya Bashivan, Jonathan Prescott-Roy, Franziska Geiger, Kailyn Schmidt, Daniel LK Yamins, James J DiCarlo - Stimulus domain transfer in recurrent models for large scale cortical population prediction on video (2018)
Fabian Sinz, Alexander S Ecker, Paul Fahey, Edgar Walker, Erick Cobos, Emmanouil Froudarakis, Dimitri Yatsenko, Zachary Pitkow, Jacob Reimer, Andreas Tolias - Inception loops discover what excites neurons most using deep predictive models (2019)
Edgar Y Walker, Fabian H Sinz, Erick Cobos, Taliah Muhammad, Emmanouil Froudarakis, Paul G Fahey, Alexander S Ecker, Jacob Reimer, Xaq Pitkow, Andreas S Tolias - Performance-optimized hierarchical models predict neural responses in higher visual cortex (2014)
Daniel LK Yamins, Ha Hong, Charles F Cadieu, Ethan A Solomon, Darren Seibert, James J DiCarlo - Using goal-driven deep learning models to understand sensory cortex (2016)
Daniel LK Yamins, James J DiCarlo - Unsupervised neural network models of the ventral visual stream (2021)
Chengxu Zhuang, Siming Yan, Aran Nayebi, Martin Schrimpf, Michael C Frank, James J DiCarlo, Daniel LK Yamins
U.S. National Science Foundation (NSF) project focused on data sharing.
Two-Photon Imaging data for video (dynamic) stimuli (2023 version) and images (static, 2021 version).
Electrophysiology and Behavioral data across the ventral visual stream (V1, V2, V4, IT).
Functional Magnetic Resonance Imaging (fMRI) and Magnetoencelography (MEG) data (in the 2019 version).
- Algonauts 2023
mantained by Cichy's, Oliva's, Kay's and Roig's Lab - Algonauts 2021
mantained by Cichy's, Oliva's and Roig's Lab - Algonauts 2019
mantained by Cichy's, Oliva's and Roig's Lab