3 Chapter 3: Consciousness of the Environment

The Essential Neuroscience of Human Consciousness

The Essential Neuroscience of Human Consciousness

Amedeo D’Angiulli

Chapter 3: Consciousness of the Environment

3.1. Neural correlates of consciousness and unconsciousness

3.2. Consciousness & Attention

Chapter 3: Consciousness of the Environment

Chapter 3.1. Neural correlates of consciousness and unconsciousness

Visual awareness is one of the most studied domains of environmental consciousness. The main reasons have to do with the evolution of the mammalian brain and the enormous importance, both culturally and biologically, of visual images in humans. There is, however, another (much more obvious) reason: there has been a lot of scientific research about vision, to the point where it is undoubtedly the most studied sense in the history of neuroscience.

A great deal of research has accumulated showing that there are neural and behavioral responses to visual stimuli not entering consciousness, generally known as subliminal stimuli (from “sub limen” or “under threshold”). That is, stimuli are processed by the brain without us ever being conscious of them. Among others, there has been experimental evidence for in-depth processing of:

1. Suppressed unconscious information from one eye during binocular rivalry;
2. Semantic information (semantic priming) coming from invisible words presented few hundred milliseconds before the appearance of related words;
3. Undetected stimuli, such as fearful faces, presented via masking.

These are only a portion of all manipulations used to distinguish two categories of brain processes: neural correlates of the contents of visual awareness or unconscious processes. These all have instances where the changes in consciousness are involuntary or unprompted by the observer, which might have made visual awareness an “easier” aspect of environmental consciousness to model.

All these lines of evidence have derived from experiments that use the minimal contrast approach; that is, the contrast between conditions in which a stimulus can be seen/consciously reported and conditions in which the same stimulus cannot. The important part is that the stimulus be presented almost identically, so that changes in awareness are not confounded with changes that occur due to unconscious processing.

The phenomenon of binocular rivalry, for example, consists in projecting in each eye/retina a different image at the same time. This can be done using variants of the stereoscope (two mirrors in front of each eye reflecting two different images) or more modern equivalents, such as flashing the images on a monitor. In nearly all experiments, the two images (typically unrelated to each other) are flashed very quickly to prevent major eye movements. Several experimenters have used people and houses, in part because of the different visual pathways processing each. The ventral “What” pathway goes from primary visual striate cortex (V1) to Inferior Temporal Cortex (IT). Different types of objects are processed separately in distinct networks: the fusiform face area (FFA) and the parahippocampal place area (PPA). When two related images are flashed, what’s seen is a fusion of both. However, when two incompatible images are flashed simultaneously, like a face and a house, the resulting illusion is that only one image is seen alternatingly: both images switch in and out of consciousness from one moment to another. The two images compete for conscious access, hence the term rivalry. Since the eyes have an order of dominance in most individuals, one image can be preferred a majority of the time (for example, the image in the dominant right eye may win 60% of the time).

Some experiments have asked humans and monkeys to report what image is seen (and when) in this illusion by pressing a key. During this period, fMRI is recorded to see how brain activity correlates with conscious reporting of the images. Looking at the relative difference in the two areas that should be activated (FFA and PPA), the results are striking.

fMRI activity is associated with both images in the early visual pathways, visual thalamus, lateral geniculate nucleus (LGN) and the primary visual areas (V1, V2, V3). However, these areas show increased activation for the image that is concurrently conscious and reported seen, whereas they show an opposite decrease in activation for the suppressed image. Moreover, the increase in activation for what’s seen – and the decrease for what’s suppressed – occur with exact timing of the report. When awareness switches from one image to another, so does direction of the activation. Higher visual areas interestingly show activity that keeps track of both images, but to a different degree. For example, extrastriate visual areas (V4-V8) show activity that correlates with the stability of consciousness, especially with the preferred image. Parietal areas and the prefrontal cortex (PFC) are activated when switching from one image to another. However, note that the FFA and PPA are both activated but with significantly higher activity in the area that corresponds to the conscious content perceived (FFA for face, PPA for house). To sum it all up, during binocular rivalry, unconscious images are extensively processed along the visual pathway, but the amount of brain activity associated with their processing seems inversely correlated to the activity associated with conscious images in early visual processing. It also appears to be considerably less (almost fully suppressed) than for images consciously perceived in the specialized areas during later visual processing.

Switching between two states of visual awareness (just like in binocular rivalry) can be more elegantly done by using pictures without sophisticated presentation, which are called bistable figures. These are images such as the Necker cube, duck/old lady illusion, Rubins vase/face illusion… In these illusions, the picture stays the same, but the contents of our consciousness flip from one interpretation (“duck”) to the next (“old lady”). Meaningless transition states of a complex pattern without focused attention rarely occur. For now, note that the interpretation of the picture that is not in conscious awareness is gone in unconsciousness! The best interpretation of why conscious brain mechanisms behave this way is that rival and bistable figures are ambiguous, which our consciousness cannot do anything with; it needs a coherent, consistent and meaningful environment to act upon.

In the laboratory, it is possible to make images literally invisible using a technique called masking. It usually takes 60 milliseconds (ms) for a visual stimulus to reach V1 and be detected 100% of the time it is presented. However, if the image is presented for 40 ms, almost no one can detect it. Detection increases around 50% when the image is presented for 50 ms. This is generally considered the threshold for visual awareness of a stimulus; images presented slower or faster are respectively defined as above or below typical visual awareness threshold, i.e. subliminar or supraliminar. By itself, near-threshold presentation (presentation for 50 ± 5ms) can induce bistable perception. However, to make the image invisible, one can have “patches” of similar colour, contrast and luminance, shown in the same area, precede and follow the near-threshold image. The only things perceived in such conditions are two flashes due to the patches; the target image is masked (invisible).

Or is it? Many experiments using words or faces as stimuli show that masked and subliminal visual displays still end up associated with activity in early visual striate areas. Observers swear they did not see anything and behave accordingly when asked to pick what they saw from a list. The experimenter is given a confused look, then observers just shrug and guess. Interestingly, their subsequent performance is significantly higher than chance and ends up 60-65% correct. Clearly, some subliminal information is processed by the brain, but not deep enough to reach consciousness. Most experimental evidence agrees that accuracy of the timing and discrimination is correlated with better-than-chance performance with concurrent activation of V1, but evidence ends there; performance is at chance levels in later visual stages and areas.

There are two important implications of these findings. First, contrary to what was thought just twenty years ago, sensory and perceptual environmental information needs to reach at least V1 to be processed unconsciously. Subcortical levels are not sufficient for the detection of subliminal information. For example, transcranial magnetic stimulation (TMS) stimulation of V1 and ventral areas in blind individuals with retinal damage have shown that basic visual experiences (i.e., phosphenes) can be easily obtained, but not in those with damaged V1. Thus, activity in this area is necessary for processing an environmental stimulus at some level of accuracy, in terms of content and time discrimination. However, and this is the second important implication, higher, more complex cortical networks than V1 must be involved for an environmental stimulus to become fully conscious. In summary, environmental stimuli need to be pushed in a bottom-up fashion from retina to V1 for unconscious processing, and from V1 to higher cortical areas to reach consciousness. V1 is necessary but not sufficient for environmental consciousness along this stream of processing.

What about the higher areas? They’re known to be necessary, but are they sufficient? Can we bypass V1 and reach consciousness? There are two sources of evidence of involuntary changes in consciousness that address the question. Some blindsight patients have part of their V1 badly impaired or destroyed, and report not seeing anything in the area of their visual field corresponding to the lesion in V1. Yet, when asked to guess and respond to simple detection of stimuli in those blind spots, their performance is fairly accurate. This suggests at least partial consciousness; the issue is that they lack integral consciousness. Clinical studies and neuroimaging, in addition to TMS studies, have indicated that when V1 is working perfectly, right parietal, frontal and prefrontal cortical lesions can lead to two types of syndromes: hemi-field neglect and change blindness.

Hemi- (“half) visual field neglect is a pattern of symptoms indicating the ability to attend to only half of the visual field; the other half shows concurrent activation of visual areas such as V1. Sometimes, the demands of the task allow the patient to change their spatial coordinates (replacing the impaired side by the intact hemifield), such as moving the body 180 degrees. This makes the inattention or neglect disappear without consciousness of the patient, even if changing the body’s position in space involves repeatedly watching the same scene.

Change blindness is the phenomenon through which people are unable to see an altered part in a display flashed twice with a blank screen in between. Some versions of this test use changes in very large objects such as airplanes or ships, but the most impressive demonstrations are short videos with a bizarre agent in which the observer’s attention is engaged in a somewhat demanding task. For example, counting the number of passes between basketball players while a moonwalking gorilla traverses the field. Again, neuroimaging evidence and TMS studies show that inattention associated with decreased activation or disruption in parieto-frontal areas predict change blindness. Therefore, overall evidence suggests that V1 is necessary for timing of stimulation and feedback from extrastriate parieto-frontal areas, but not sufficient without these backward feedback connections to mediate conscious access. On the other hand, parieto-frontal networks are necessary for integral visual awareness (linked to deficits in hemi-neglect and change blindness), but not sufficient without V1.

To recapitulate, we know what happens to neural signals associated with consciousness, but how deep is unconscious processing in the brain? How much of the unconscious processing is elaborated and how much does it influence us without our knowing it? The answer to both questions is a lot! The most compelling evidence comes from the unconscious processing of meaning or semantic content.

There has been an accumulation of evidence showing that subliminal pictures and words with strong emotive content, such as angry faces or words such as rape, murder, danger and poison, are detected by the amygdala and other limbic structures even when they are well below threshold or masked. What is remarkable is that when invisible words are presented before other visible words (which might make a combination), a task based on the evaluation of the visible words is influenced by the unconscious processing of the invisible ones. For example, if the participant is subliminally shown the words “very happy” then “war”, making the concept “very happy wars”, brain activity recorded for incongruent semantic information (the ERP signature N400) occurs.

Similar depth of semantic processing is shown through experimental conditions that produce priming or facilitation in the speed or accuracy of performance related to a target stimulus. In these situations, the prime stimulus can be an invisible word or picture followed by a related word (target) or unrelated word (distracter) shown above threshold. For example, the primer could be the subliminal word piano, the target could be flute, and the distractor could be spoon. The word “flute” is likely to be processed faster and more accurately than “spoon”, due to the influence of “piano” even though the latter does not enter consciousness. There have been scores of priming experiments using subliminal words and single/multiple words as targets, showing activation in the ventral visual stream up to the temporal cortex. This suggests that subliminal words receive semantic processing similar to and as exhaustive as visible words.

In sum, there is compelling evidence that unconscious processing involves A) Higher cortical networks, and B) Considerable depth. Current research in cognitive neuroscience has explored a few hypotheses based on these points in quest of greater understanding of the relationship between consciousness, unconsciousness and other fundamental neurocognition. In this context, the most important work has mostly focused on the role of attention and its interactions with consciousness.

3.2. Consciousness and Attention

Following nearly any definition, research agrees that consciousness gives potentially huge opportunity of access to thoughts, perceptions, sensations, etc. While writing this, I can hear birds singing, my dog scratching, a bee buzzing… However, the strange thing is that the actual set of things I am actually conscious of is really restricted. I can be really conscious of one group of things at the time, be it a chunk of organized information (such as the sentences I am reading and writing) or a thought (such as the idea I am trying to communicate). Thus, while there is infinite information hitting my sensory receptors, memory and cognitive system, it is clear that I focus on just one entity that has undergone some preconscious processing, filtering or selection. This latter process of sifting through information, eliminating the irrelevant bits and offering an output accessible to consciousness, is distinct from consciousness though very related to it. Many agree this process can be considered a function of attention, and more specifically selective attention. Do note that attention and consciousness have different functions.

Be careful, and keep in mind these few distinctions. When covering philosophy, I mentioned that one important criterion of consciousness is that it is about something. That is, consciousness is transitive: it can go from one object or entity to another. However, this is only when the higher levels of consciousness are reached with the aid of attention. Lower levels of consciousness, as discussed in relation to the neurological examination, do not really have a specific content; they are said to be intransitive. For example, when we say “The tackled hockey player lays on the ground but remains conscious”, we mean something similar to what seen when covering the vegetative and minimally conscious states. There are signs that the individual is awake (referring to lower subcortical regions regulating the sleep-wake cycle) or vigilant (referring to the excitement of cortico-thalamic networks that support conscious states). Again, wakefulness, vigilance and attention are what we call c-conditions.

However, agreements among researchers ends here. There is significant debate surrounding several research questions: Can we become conscious of something without attending to it? Can we attend to objects that are consciously suppressed, i.e. invisible? Can selective attention and perceptual consciousness have opposing effects? The current mainstream view is that attention and perceptual consciousness are two distinct, but often allied, processes with distinct neurobiological processes. Therefore, an important research objective is to carefully distinguish the neural correlates of consciousness from those of selective attention.

Let us consider vision. In primates, about one million fibers leave each eye and carry raw information on the order of one megabyte per second. To deal with this deluge of data, attention is a mechanism that selects relevant information to the organism while leaving the unselected and unattended data to suffer from benign neglect. Attentional selection is based on either “unprompted/involuntary/bottom-up (attention)/exogenous” or “prompted/deliberate/top-down/endogenous” factors. However, subjects can disregard salient/bottom-up cues when searching for particular objects in a scene with top-down, task-dependent control of attention. Bringing top-down, sustained attention to bear on an object takes time; it selects input defined by a circumscribed region in space (focal attention), a particular feature (feature-based attention), or an object (object-based attention).

In contrast, consciousness has significantly different functions from attention. These range from summarizing all information relevant to the current state of the organism and its environment, making this summary accessible to the planning stages of the brain, detecting anomalies and errors, decision making, language, inferring the internal state of other animals, setting long-term goals, making recursive models, and rational thought.

So, to the extent that one accepts attention and consciousness have different functions, one also has to agree that they cannot be the same process. We can distinguish four cases of processes: voluntary attention & consciousness, voluntary attention & unconsciousness, involuntary attention & consciousness, and involuntary attention and unconsciousness (as indicated in Table 1, adapted from Tsuchiya and Koch [2009]).

Let’s take a look at some examples in Table 1. There’s compelling evidence that we must attend to novel stimuli to become conscious of them, for example in a complex pattern full of distractors (“Where’s Waldo?”). In most of these tasks, we need to go through the entire image and scan all its details to be able to find the desired information. Similarly, when we lack a precise idea of what to look for, we need to engage in step by step serial search. For example, what do you see in Figure 5?

Non-voluntarily attended entities, moving from the retina to V1 and beyond, may not trigger a conscious perception. Nonetheless, this can still be effective and leave traces in the brain that affect other processes, such as priming. For example, some types of masking allow invisible stimuli to cause negative afterimages, and withdrawing from the afterimage inducer strengthens afterimages (Opposite Effects). Therefore, afterimages can be induced without voluntary attention and without seeing the inducer. Other examples of this include visuo-motor reflexes (ex: accommodation and pupillary reflexes) and zombie behaviors. Zombie behaviours are highly trained, rapid, automatic and stereotyped sensorimotor actions, which are likely to run independently of voluntary engagement. Common examples are reaching and grasping, posture adjustment, running and action sequences such as tying shoelaces, playing piano and so on. Rapid visual categorization, i.e. telling animals from inanimate objects or identifying entity sex /gender, is yet another candidate for a process that may require neither voluntary attention nor consciousness.

To approach the (sometimes confusing) idea that behaviours can be unconscious but require conscious attention, consider that people can observe a location for some time and yet fail to see some attributes of an object there. Neuroimaging (fMRI) and psychological experiments have discovered similarities in the processing of invisible stimuli that require voluntary attention, invisible stimuli that attract spatial attention and feature-based attention spreading to an invisible target. However, some interesting ones have shown a priming effect of invisible properties in subliminal pictures, such as sex (nudes) or emotions (words), only when the invisible stimuli are actively attended to.

The best example of consciousness without voluntary attention is provided by the phenomenon of gist: gist is immune from inattention deficits such as change blindness. When a full-background photograph is briefly shown unexpectedly to an observer, subjects can accurately report a summary of its contents. In the 30 ms necessary to get the gist of a scene, top-down attention cannot play much of a role because gist is a property associated with the entire image; anything that enhances local features is going to be of limited use. For example, consider perception of a single object in an otherwise empty display. The most popular neuronal model of attention, biased competition (Desimone and Duncan, 1995), predicts that without competition, little or no attentional enhancement occurs. Take a brief look at Figure 6 then look away – what did you see?

If you said “ruins” or “an archaeological site”, you would be partially right… but your gist would be talking. However, have you seen the iguana? Look again at the picture’s central area to behold a camouflaged iguana. Given that we can’t (currently) experimentally manipulate subject brains/neural matter, it is very difficult to know that all attentional resources have been removed from a particular location or object during experimental conditions. What seems clear is that conscious perception can occur in the near-absence of, or at least without the necessity of, voluntary attention.

Most remarkably, withdrawing attention voluntarily from a stimulus and cloaking it from consciousness can cause counterintuitive effects. When observers engage voluntary attention to try to find two embedded targets within a rapidly flashed stream of stimuli, they often fail to see the second target, a phenomenon called attentional blink. Surprisingly, Olivers and Nieuwenhuis (2005) reported that observers can see both targets better when they are distracted by a simultaneous auditory task or encouraged to think about task-irrelevant events. It seems that sometimes, not putting too much attention and consciousness in a decision can lead us to take better decisions. Some research also suggests this might apply for multiple choice tests!

The distinction between voluntary and involuntary attention is a simple description of two major networks usually designated with technical terms: that is, bottom-up and top-down processing. I did not use these terms because they are somewhat confusingly and arbitrarily used in the literature. When properly used, however, they mean that involuntary attentional tasks correspond to processes that involve feed-forward stages of processing initially involving V1, then continuing in the ventral to inferior temporal or the dorsal to parietal visual pathways. These feed-forward streams of processing end with output that fully describes an object or locale in space where things are happening. They may or not project to the frontal and prefrontal cortex, which is the highest neurocognitive level of processing. The “bottom-up” expression refers to how the information is put together: it goes from single elements (ex: pixels), to features, then more complex feature configurations… Knowledge is built through building blocks that become increasingly larger and more complex. “Top-down” processing associated with voluntary attention works in exactly the opposite fashion, from complex structure to simple features. Very few researchers think it can go to the very simple constitutive elements beyond features. Top-down operations may require feed-forward processing, but a distinctive property of it is that it heavily involves backward feedback from higher to lower brain levels. Practically, this means that the distinctive aspect of top-down processing is that it always involves the frontal and prefrontal cortex as the main player and initial stage of elaboration of information.