The visual cortex is located in the occipital lobe of the brain and is primarily responsible for interpreting and processing visual information received from the eyes. The amount of visual information received and processed by the visual cortex is truly massive. Nearly half of the brain is in some way dedicated to vision—either direct communication pathways from the retina of the eyes to the occipital lobe, or to indirect visual processing and visual skills. The visual cortex is divided into six critical areas depending on the structure and function of the area. These are often referred to as V1, V2, V3, V4, V5, and the inferotemporal cortex. The primary visual cortex (V1) is the first stop for visual information in the occipital lobe.
The brain is filled with depressions or grooves (sulci) and elevations (gyri). These help increase the overall surface area of the brain. The primary visual cortex is located in and around the calcarine fissure, which is a characteristic landmark sulcus in the occipital lobe.
Prior to reaching V1, visual information is separated into the right and left visual field shortly after the nerve cells making up the optic nerve leave the eye. This separation occurs in the optic chiasm, and allows visual information from the right and left eye to combine together. The farther back in the brain the visual pathway travels, the closer corresponding points between the right and left become associated. By the time the information is received by V1, the corresponding points in vision are perfectly meshed. A secondary consequence of this cross over at the optic chiasm is that the right side of the visual cortex processes vision from the left visual field, and the left side of the visual cortex processes vision from the right visual field.
V1 can be thought of as a sorting area. Cells from the entire visual field are represented, but the majority of V1 is dedicated to cells associated with the foveal (central) vision. The fovea is the small anatomical area of the retina responsible for fine-detail vision. The main function is to process all the incoming visual information and pass the correct information to the more specialized areas of the cortex. The more specialized areas are termed the extrastriate cortex, and include visual areas V2, V3, V4, V5, and the inferotemporal cortex.
Visual processing can broadly be separated into two pathways. These are the "what" and "where" components of visual processing, and is often referred to as the two-stream model. The occipitotemporal (ventral) processing stream is critical for visual recognition of objects (“what” something in vision is). Input primarily is received from parvocellular cells in the lateral genicular nucleus of the thalamus. These are smaller cells dedicated to fine-tuned spatial resolution (think color and clarity).
The occipitoparietal (dorsal) processing stream is critical for visually-guided action and localization of objects in space (“where” something in vision is). This pathway primarily receives input from the magnocellular cells in the lateral genicular nucleus of the thalamus. These cells are larger and are dedicated to temporal resolution (think low detail and movement).
Although the flow of visual information appears to be in different areas of the brain, it is paramount to understand that visual processing is a cooperative task. These systems work together to benefit the body as a whole. The visual system contains a series of reciprocal pathways or feedback loops to help with information gathering and processing.
V2 receives information directly from V1 and passes information to V3, V4, and V5. There is also a feedback loop to send signals back to V1.
V3 communicated directly with the respective dorsal and ventral subsystems of V2. Dorsal V3 seems to play a role in processing motion, while ventral V3 may play a role in color sensitivity. V3 as a whole is less well-defined compared to other areas of the visual cortex.
V4 receives information from V2 and is part of the ventral processing stream. Cells in V4 are very responsive to color.
V5 is part of the dorsal processing pathway and contains cells highly sensitive to motion.
The inferotemporal cortex is located along the lower (inferior) portion of the temporal lobe. This area of the brain is part of the ventral processing stream and seems to respond best so simple shapes (circle, square, etc.).
Sources: Neuroanatomy, Cerebral Cortex: https://www.ncbi.nlm.nih.gov/books/NBK537247/
Two Visual Pathways in Primates Based on Sampling of Space: Exploitation and Exploration of Visual Information: https://www.frontiersin.org/articles/10.3389/fnint.2016.00037/full