***THIS SITE IS A ROUGH WORKING COPY OF THE ENACTIVE THEORY OF COGNITION. THE DIAGRAMS ARE CORRECT, BUT THE TEXT REQUIRES UPDATING.***

This graphical representation by Davis (2012) depicts a theory of consciousness, cognition, and energy. In the theory, consciousness is said to be an emergent property of two flows of information coming from both the brain and the body.

The emergent consciousness then maintains the functioning of the system by providing feedback. A similar mechanism is said to exist for the conservation of energy in physics. Instead of E = MC^2, Davis (2012) postulates that energy is the emergent sum of two fundamentally different types of energy, mental energy, and physical energy. These two energy equations are balanced dynamically to arrive at a third emergent equation, which can be conceptualized as (E = MC^2)^2 (Davis, 2012).

This model depicts the dynamic flow of cognition as consciousness emerges through the fusion of information from the brain and body. Consciousness provides feedback to the system to help maintain its effective functioning. Each feedback loop is variable based on the conscious attention of the agent.

Two states of cogition exist: clamped and unclamped (Glenberg, 1997; Davis, 2017). In clamped cognition, the feedback loops are set to a static amount. In unclamped cognition, the feedback loop ratios change depending on the concious experience of the agent.

Attention is a finite resources, and the role of the consciousness of the agent is delegating attention to these four operations in different ratio or proportions:

1. Simulating future interactions: mental visualization, mental simulation

2. Simulating past interactions: episodic memory, reflection, recollection, recall

3. Exploring opportunities for interaction: object affordances

4. Performing actions

While in a clamped or structural coupling with the environment, the ratios of these feedback loops would remain relatively fixed. However, in early stages of sense-making, or when something happens to violate the expectations of the individual, agents decouple their interaction, unclamp their perception, and manually modulate the way that sensory data is entering the system and the types of predictions the brain is making. Thus, we can say that the feedback loop in each quadrant of the model has (at least) two states, clamped and unclamped.

Once perception is unclamped and consciousness directs attention to a particular quadrant of experience, the shape of the feedback loop changes in this representation (becoming either larger or smaller). The shape has additional meaning to help categorize emergent cognitive states, i.e. phenomenological states of consciousness. There exists a state of potentiality for each quadrant if the agent where to unclamp and devote attentional and cognitive resources to that particular type of cognitive pursuit. These states of potentialities are represented in the initial EMC2 model as the dotted lines surrounding each of the feedback loop to show that there are multiple states and shapes of each feedback loop, as shown in the full depiction of EMC2.

Davis et al. (2015) describe the enactive model of perception as: "the contents of perception vary based on an individual’s position on [a] continuum of cognition (Glenberg 1997). As individuals deviate from the equilibrium in the center of the spectrum, perception becomes partially unclamped (a term coming from Glenberg’s (1997) theory of memory) which loosens semantic constraints on sensory input and memory. Different points on the cognitive spectrum result in a unique perceptual logic that is used to intelligently perceive affordances in the environment....The enactive approach proposes that perceptual intelligence arises through the formation of percept-action pairings that are chunked and internalized for quick retrieval (Noë 2004). Perceptual logic is a proposed cognitive mechanism that filters sensory data, identifies relevant percept-action pairings, and presents these percept-action pairings as affordances to perception." (Davis, 2015)

Clamping Perception

"Research indicates that perception filters irrelevant sensory input to reduce distractions and facilitate everyday cognition (Gasper 2014). When the agent is engaged in a routine task and following well established affordances, sensory data is clamped to filter out unnecessary details and unconventional ways of seeing objects (Glenberg 1997). Everyday cognition is represented in EMC by situating the awareness rectangle in the center of the spectrum of cognition, creating a point of equilibrium. Shifting either to the left or right on this spectrum requires the agent to concentrate on either the details of her mental model or closely inspect details in the environment. At equilibrium, perception is clamped to a combination of sensory input and cognitive input that optimizes routine interactions (Glenberg 1997). When minor problems arise, such as small improvisational adjustments to the action based on environmental feedback, this equilibrium is slightly perturbed. The agent could generate various alternative actions by thinking (moving slightly left on the spectrum) and explore various ideas by interacting with the environment (moving slightly right on the spectrum)." (Davis, 2015)

Clamped Cognition: "The process of maintaining or slightly refining the selected generative model assuming that it is the most accurate representation of the environment. It generally occurs after making sense of a task or activity. Behavioral markers include fluid interactions with minimal hesitation (e.g. embodied play actions, fluid drawing actions)." (Davis et al., 2017)

Unclamping Perception

"If there is a severe disruption to the current task (e.g. a great new idea, distraction, or some kind of failure), it might become necessary to disengage from the current task to re-evaluate the situation. When an individual disengages from a task, perception becomes unclamped and attention shifts to thinking and simulating solutions (moving far left on spectrum) and closely examining the detail of the environment to discover new affordances (moving far right on the spectrum). The degree of concentration devoted to thinking about or acting on the environment determines how far, in either direction, awareness is situated on the spectrum of cognition. At the extreme left of the continuum (thinking) would be closing one’s eyes to try to think deeply about a topic, which removes most sensory input from perception altogether. At the extreme right of the continuum (inspecting) would be an individual fully concentrated on acting skillfully, carefully, and deliberately on the environment and paying close attention to the feedback." (Davis, 2015)

Unclamped Cognition: "The process of changing or replacing the generative model by exploring and reflecting on the environment from different perspectives. It generally occurs during task onset and after surprises during the task. Behavioral markers include hesitation (e.g. eyes closed, confused look) and physically experimenting with the environment and viewpoint of the environment (e.g. futzing, inspecting)." (Davis, 2017)

Sense-Making

"Within the unclamped category, there are two further distinctions that can be made to increase the granularity and explanatory power of our proposed coding technique. Unclamped actions...can be either perceptually-based or physical-based. Perceptual-based sense-making relates to refining the brain's predictive model, which subsequently changes how features in the environment are perceived and interpreted. Since these processes are happening internally, they cannot be directly observed, but individuals experiencing this cognitive state display indirect behavioral markers, such as pausing, hesitating, contemplating, and looking confused, i.e. thinking. Physical sense-making, on the other hand, changes the structure of the environment by manipulating and modifying the environment, or by moving the body and altering what information is available to the senses, i.e. thinking by doing." (Davis, 2017)

Perceptual sense-making: "the cognitive agent is working to internally improve recognition density of its generative mental model, i.e. thinking. Behavioral markers include: hesitation, eyes closed, confusion, and task disengagement in general." (Davis et al., 2017)

Physical sense-making: the cognitive agent is exploring the environment through interaction to decrease disorder in the environment and increase the recognition density of its generative mental model, i.e. thinking by doing. Behavioral markers include: experimentally manipulating resources in the environment and re-positioning the body to change available sensory data. (Davis et al., 2017)

Perceptual Logic

Perceptual logic is a cognitive mechanism that filters all the available object affordances generated by cognition to present only those that are relevant to consciousness in the moment. Object affordances are the interactive potential of objects in the environment, e.g. a cup affords drinking, a pen affords gripping, and a ball affords bouncing (Norman et al.). However, there are an infinite number of object affordances potentially associated with each object in the mind. To our knowledge, perceptual logic is the only proposed cognitive mechanism to select the relevant affordances and present them to consciousness. Perceptual logic is proposed here as a cognitive mechanism to filter all affordances and determine which are relevant in the moment to facilitate effective interaction.

Perceptual logic is proposed to work through combining input from the agent’s environmental sensory data, the data from the current mental model, and the intention of the agent in the moment. These three input mechanisms can be utilized to filter object affordances to determine which of them should be perceptually salient object affordances, which are object affordances the individual perceives and experiences first hand through their perceptual and interactive processes. These perceptually salient object affordances are relevant to the agent’s intention, but may not always be the most optimal for interacting effectively in a scenario. The elements of the intention, sensory data, and the mental model the individual is currently attending to influence which perceptual logic is activated. This determines which affordances are activated as perceptually salient and thereby available to the conscious perception of the individual.

Perceptual logic structures thinking by activating relevant mental models to predict elements of interaction with the environment (sense-making) and other agents within it (participatory sense-making). When there is an adequate mental model developed to effectively guide interaction with the environment, cognition can be said to be clamped upon that mental model. When a surprising event occurs, cognition can become unclamped. During an unclamp cycle, the individual can interact with both their mental model and the environment to determine a more accurate mental model and interaction strategy. Once a mental model and interaction strategy is in place, cognition then clamps onto the novel mental model and interaction strategy. The intention, mental model, and sensorimotor data are all used to determine which perceptual logic should be active. The perceptual logic then dynamically selects the appropriate object affordances to be perceived in the moment.

Neurocognitive Basis of Perceptual Logic

Perceptual logic can theoretically be formed through a stable grouping of dynamically firing neurons that creates an attractor for thought. This attractor seeks out relevant knowledge structures and activates their connected semantic, procedural, and sensorimotor neural networks to determine the most effective interaction profiles of each of the knowledge structures. The interaction profiles that seem promising relative to the intention, directives, and goals of the agent form the object affordances that are presented to consciousness in the moment of interaction, e.g. the perceptually salient object affordances.

Perceptual logic is similar to Gabora’s concept of a neurd (Gabora, 2010) in the sense that it is a stable grouping of neural activity hypothesized to enable thinking in a domain. It is also similar to a simulator in Barsalou’s Perceptual Symbol Systems (Barsalou, 1999). A simulator is a group of neuronal activations surrounding a single class of perceptual experiences that form a concept, e.g. the simulator for a car would contain the sight of the car, the feeling of the metal and steering wheel, the feeling of the seat, the visual appearance of the interior, etc. When a simulator becomes active, the relevant branches of the simulator activate, which each contain the neural correlates of the perceptual experience. Activating a simulator creates a stable conceptual structure of the object or situation at hand. This theoretically occurs by activating the same neurons (or a subset thereof) that were stimulated at the time of the initial perceptual experience. Perceptual logic is the mechanism that determines which branches of a simulator become active in the simulation of a concept. The actively simulated concept would then provide the neural basis for relevant affordances through the stored sensorimotor profiles of each concept.

Types of Perceptual Logic

Perceptual logic can be based in different phenomenological states of cognition. There are three general types of perceptual logic an agent can engage with.

Semantic Perceptual Logic

Semantic perceptual logic pertains to the semantic definition of an object, such as a house that has windows, a door, a chimney, a walkway, a car, and a garage. The object itself suggests items that can be added to it. The detail of this perceptual logic is dependent on how familiar one is with an object and its features.

Procedural Perceptual Logic

Procedural perceptual logic has to do with the experience of producing a creative expression, such as sketchy lines, squiggly lines, or another pattern that can be executed successively. The pattern can be extended or applied to other regions of the artwork. The pattern itself could also be altered to change its application.

This logic contains abstract relationships that exist in a drawing. For example, in th figure above, there is a basic stem and lines radiating from that stem at 45 degree angle. There may be another perceptual logic about only crossing lines at a 90 degree angle. Procedural perceptual logic is about how to traverse through an artistic landscape by interacting with the various regions that are there and the patterns they contain. Procedural perceptual logic is about the way in which something is created. It is about remembering the technique for crafting a design or pattern.

Episodic Perceptual Logic

Episodic perceptual logic pertains to co-occurrences of objects in similar situations from the past. It may entail some semantic relationships and procedural relationships, such as beach scenes contain chairs, which have many features, and also beach sand, which can have repeated patterns.

Levels of Perceptual Logic

In addition to these types of perceptual logic, perceptual logic can exist at different spatial and temporal scales or resolutions. There are at least three spatio-temporal scales of granularity of perceptual logic:

Local Perceptual Logic

Perceptual logic pertaining to the immediate activity and/or region of space the individual is currently interacting with. This relates to the details of a scene and the minor aspects that should all be consistent and balanced in order for the creative product to be balanced. For example, the inner details of an eye might not be consistent and require a revision.

Regional Perceptual Logic

Perceptual logic pertaining to the recent activity and/or the general region of space in which the interaction with the environment is taking place. This factor can relate to one or multiple objects and pertains to whether that object is currently expressed in adequate terms. For example, there may be a face which has only one eye that requires a matching eye for the full face logic to be complete.

Global Perceptual Logic

Perceptual logic pertaining to the whole activity that could extend in both time and space significantly. This perceptual logic has to do with viewing the artwork as a whole. It is a zoomed out picture of all the relationships that are present in a scene. Sometimes, the artist can take a step back to see the artwork from a smaller perspective to judge the overall flow of the artwork.

These spatio-temporal categories of perceptual logic add additional layers of nuance to each of the thinking style perceptual logics listed above. Each type of perceptual logic has a local, regional, and global aspect to it. This creates a matrix of 12 perceptual logics that can be applied in creativity.

Conclusions

The enactive theory of cognition proposesa computationally tractable explanation of consciousness and perception as intimately intertwined processes. Perception is dynamic and variable based on the conntents of conscious experience, such as intention. Perceptual logic was proposed as a cognitive mehanism meant to filter all possible affordances and present only those that are relevant to consicousness. A framework for perceptual logic was presented that demonstrated the different aspects and qualities of perceptual logic and showed its explanatory power as a cognitive mechanism. The resulting model, the enactive model of cognition, depicts a continuous and dynamical cognitive system with input from both the environment and mental model of the agent. This model can be used in creative and co-creative artificial intelligence agents, computaitonal models, and computational theories of cognition.

References

Adamson RE (1952) Functional fixedness as related to problem solving: a repetition of three experiments. J Exp Psychol 44(4):288–291

Barsalou LW (1999) Perceptual symbol systems. Behav Brain Sci 22(04):637–660

Davis, Nicholas. (2012). Enactive Consciousness: A Computaitonal Cognitive Theory of Consciousness Accessed June 2022 on www.enactivetheory.com

Davis, N., Hsiao, C. P., Singh, K. Y., Lin, B., & Magerko, B. (2017, June). Creative sense-making: Quantifying interaction dynamics in co-creation. In Proceedings of the 2017 ACM SIGCHI Conference on Creativity and Cognition (pp. 356-366).

Davis, N., Hsiao, C. P., Popova, Y., & Magerko, B. (2015). An enactive model of creativity for computational collaboration and co-creation. In Creativity in the digital age (pp. 109-133). Springer, London.

Davis, Nicholas. (2022). The Enactive Model of Continuous Cognition, in Co-Creative AI: A Computational Cognitive Theory for the Design and Evaluation of Co-Creative AI Agents. Accessed June 2022: www.enactivecomputing.com

Glenberg, A. M. (1997). What memory is for: Creating meaning in the service of action. Behavioral and brain sciences, 20(1), 41-50.

Stewart JR, Gapenne O, Di Paolo EA (eds) (2010) Enaction: toward a new paradigm for cognitive science. MIT Press, Cambridge, MA

Gabora L (2010) Revenge of the “neurds”: characterizing creative thought in terms of the structure and dynamics of memory. Creat Res J 22(1):1–13

Gaspar JM, McDonald JJ (2014) Suppression of salient objects prevents distraction in visual search. J Neurosci 34(16):5658–5666

Gibson JJ (1979) The ecological approach to visual perception. Lawrence Erlbaum Associates, Hillsdale.

Nersessian N (2008) Creating scientific concepts. MIT Press, Cambridge, MA

Varela, Francisco J., Evan Thompson, and Eleanor Rosch. The embodied mind, revised edition: Cognitive science and human experience. MIT press, 2017.

Figure 9: Each feedback loop expands when consciousness directs attention towards activities relating to each type of feedback

Each quadrant, then, has an additional dimension and layer of potential explanatory and predictive power. The EMC2 models an emergent dynamical system that is governed through those laws. The shape of each feedback loop, then would have a certain number of steady-state configurations that tend to yield productive results. Two phases we can be fairly certain such a process would include is a broad superficial phases and an deep in-depth analysis phase (due to mounting evidence for these distinct type of cognitive processing happening in the brain [REF]). When we consider the temporal dynamics of cognition and analyze what each feedback phase would mean in each quadrant of experience, we arrive at a more nuanced conceptual framework with which to systematically describe conscious experience. Each phase will be described in turn.

Simulating Future Feedback Phases

• Broad: Imagining graduating college

• Deep: Imagining completing one homework assignment

Simulating Past Feedback Phases

• Broad: Thinking what went wrong in previous relationship

• Deep: Recollecting what caused one to get in a car accident so as to avoid doing it again

Affordance Feedback Phases

• Broad: Perceiving the environment now in terms of what will help me at various time scales (such as graduating college)

• Deep: Perceiving the environment in terms of achieving a very precise intention in the moment.

Action Feedback Phases

• Broad: Doing things now that have an effect far in advance, slow feedback time

• Deep: Doing things now meant to have an immediate effect--the deeper the more complex the action and intention

Levers for Tuning Cognition

There is an additional dimension of variance to the system that we might consider as we explore its features, which is the shape of the lense that that the prediction generating engine of the brain and the shape of the body filter for the senses. Theoretically, the lense itself could be shaped in a broad or narrow manner, as well as the body filter, as shown in Figure 1-0.

Figure 10: Demonstrating the varieties of ways that the brain and body could theoretically be ‘tuned’

The shape of the brain lens could theoretically be either wide or deep. If the brain lens was wide, it would correspond to considering many things that did happen in the past as well as many things that could happen in the future as a results of various different interactions, as depicted in Figure 10 modulating brain image 2. (left right brain processing, inhibitory phases of system.) If the brain lens is deep, that would correspond to focusing on only the immediate future and immediate past and how they are related to the current interaction, depicted in Figure 10 on the left side number 3.

Similarly, we could look for that same effect by changing the nature of sensory input. Widening the sensory filter would correspond to looking for distant causes or very future potential in the environment. An artist might engage in this type of perception when he looks at a piece of trash and can see a beautiful artwork in the distant future sitting in front of him, i.e. he knows how to interact with that element of the environment to turn it into something of value.

In turn, deep sensory focus would be perceiving the environment and knowing the precise sequence of actions naturally to achieve a very detailed procedure. The bandwidth of sensory information is very small, but allows attention to be stretched further into the details of one particular interaction and intention. It may turn out that the shape of the brain lens or body filter could increase the probability of different shape formations in the feedback loops, for example, perhaps a wide brain lens that considered distant causes and effects would support and be accompanied by broad mental simulations of the future and past.

Correspondingly the shape of the body filter might support exploration of affordances through actions exploring the environment. Wider filters may lend to exploratory behavior that could all be relevant to an eventual long term goal, whereas deeper filters may allow intense and precise sense-making to occur such that one particular outcome occurs soon. With these different modulations, we can systematically reproduce many states of cognition in a type of emergent network--a continual cognition

This chapter describes the enactive model of continuous cognition (EMC2) that explores the relationship between the brain, body, and consciousness. In particular it shows how consciousness can tune both the brain and body to optimize interaction with the environment and other people in it. EMC2 is an evolution of the enactive model of creativity, which applies the theory of enaction to the domain of creativity by exploring how sense-making is used to gradually define directives that subsequently guide interactions during the creative process.

The contribution of this chapter is presenting a novel modeling convention that represents the temporal and emergent nature of continuous cognition in a visually elegant and computationally tractable manner. We begin by introducing the general conceptual structure for the EMC2 and explain the core components in detail. Next we describe the role of consciousness and sense-making in the model. A simple 2D linear convention is proposed for describing the temporal dynamics of cognition. Different types of feedback processes in the cognitive system are considered next, followed by methods for shaping those feedback loops through conscious thought and action.

Next, we describe the process of ‘intentional perceptual attunement’ or tuning the brain and body to process information in a way that increases the probability the agent will achieve its intention. Then, we examine some cognitive procedures that may systematize the process of intentional perceptual attunement, including meditation, visualization, and creative expression. This discussion leads into the theoretically predicted ‘perceptual placebo effect’ that is strengthened by actively cultivating the ‘magical mentality’ during the cognitive procedures just listed.

Enactive Model of Continuous Cognition

The enactive model of continuous cognition models a dynamical emergent cognitive system that is continually flowing through time. There are three basic components: a brain that acts as a prediction generating machine, a body that acts and senses, and a consciousness that modulates the functioning of those two systems.

The prediction generating machine causes an afferent flow of information outward from the agent and into the world by projecting its simulated version of reality into the agent’s conscious awareness. The sensory apparatus of the agent (the body), in turn, represents an efferent flow of sensory data from the environment that enters the body and is filtered by various biological mechanisms before being presented to conscious awareness. The simulated reality provided by the brain combines with key information in the environment presented by the body to form the agent’s conscious perceptual experience, which would typically be referred to as ‘perception.’ In short, perception has a filter that guides the agent’s conscious awareness to critical parts of the environment to inform the brain’s predicted version of reality. We might say ‘perception’ is constituted by 40% brain simulation and 60% sensory data (an arbitrary but illustrative ratio). This two part process mutually constitutes perception, which is a critically important feature of the theory of enactive perception.

Consciousness can direct the body to change orientation and position in order to obtain different sensory data, as required to validate predictions of the brain’s dynamic model. Additionally, consciousness can direct the prediction powers of the brain through thought, i.e. thinking of different intentions informs the brain about what predictions are relevant.

One justification for this particular structure to perception is the well known limits on the processing capabilities of attention and working memory. Given that conscious attention can only process a limited amount of information, EMC2 (and enaction in general) propose that the brain predicts and simulates the environment that is then combined with relevant sensory data and projected to consciousness as ‘perceived reality.’ The sensory apparatus of the agent is then recruited by consciousness to verify that the predictions and assumptions the brain made in formulating these predictions were accurate. This vastly reduces the processing demand of consciousness and attention. Without having to actually process all of the sensory data, the primary role of consciousness is to modulate the manner in which those predictions are being made and the way in which the body is sensing information in the environment to help achieve its intention.

Core Components of EMC2

To explain the EMC2, we will slowly build out all its components beginning with the three critical ingredients, namely the brain, body, and consciousness, as shown in Figure 4.

Figure 4: The basic components of EMC2

The shape of the brain and body in this model have significance. The brain is shaped like a lens because it is meant to focus and project predictions and simulations to consciousness. The body is shaped like a triangle because it filters a great amount of sensory data and presents only the most ultra relevant information to consciousness. The next quality of this model is understanding that the brain and body both exhibit a dynamic flow of information directed toward the consciousness of the agent, as shown in Figure 5.

Figure 6: 2D linear representation of the manner in which cognition is modulating over time

When the agent is coupled with the environment, it moves in a roughly straight manner along the horizontal timeline. However, in the various stages of sense-making, agents decouple and unclamp perception to devote their attention to both thinking and exploring the world, in parallel or individually.

In this temporal representation of the model, we show the two main influencing factors, i.e. the influence of the brain and body on cognition (more nuanced distinctions will be presented later). During unclamped cognition, when individuals are doing any manner of conscious thought (simulating past or future, to various capacities and with different intentions), the circle representing consciousness rises above the line. The degree to which the circle rises above the line corresponds to the amount of cognitive resources devoted to ‘consciously thinking.’ When individuals are investigating the environment for possible ways to interact and also interacting with that environment, more attention is devoted to the senses than conscious thought. The degree to which the circle goes below the horizontal line would correspond to how closely the individual is inspecting and concentrating on experimentally acting upon the environment.

This linear 2D representation depicts the manner in which an agent can potentially modulate their consciousness. It will greatly aid in describing the sense-making; the initial stages of the sense-making process could be represented as damped harmonic oscillation. As the individual ‘makes-sense’ of the environment, they begin to develop a ‘perceptual logic’ that serves to filter irrelevant parts of the environment. As the perceptual logic becomes more detailed, interaction gradually becomes ‘structurally coupled’ such that the individual does not have to devote much conscious attention to thinking about the task being performed. Once structural coupling occurs, perception is said to be clamped, which means a particular perceptual logic (or set thereof) is active and fixed.

Objective Time and Subjective Experience of Time

Modeling cognition in this manner helps us explain why humans sometimes have a different perception of time than objectively measured time. The agent’s subjective experience of time would correspond, not to the Tobj in the figure, but rather to a new measurement of time that corresponds, approximately, to the length of the actual line in a given interval. The difference between the Tobj and Tsubj in this model should roughly correlate to the ratio of the actual time passed vs. perceived time in an experiment. This is supported and rationalized through reports of individuals fluctuating reports of time while in ‘flow.’ On average, time is reported to speed up (or more accurately not slow down..).

We provide two explanations of this based on the model. First, when the agent is de-coupled from the environment, perception is unclamped, thus they are necessarily processing more sensory or mental information. As the consciousness literally experiences more ‘stuff’ it might seem as if more time has passed because the agent had to exert a lot of effort processing all that data. Second, we predict that in the ‘coupled’ state of interaction, time does not speed up or slow down so much as become irrelevant to the consciousness. There is no need to track time, so it becomes an abstract concept working in the far background of conscious awareness. Literally, cognitive resources have been diverted away from any type of cognitive processes that need or relate to time. One explanation of this is because there is no need to perform any mental simulations of possible future ideal states or any knowledge based on previous experience-- because the agent knows precisely how to interact with the environment based on ‘reading’ the important information in the environment using a well-developed perceptual logic.

For that particular consciousness, at that precise moment in flow, there is literally no past or future, but rather only now, the precise moment in which the interaction is taking place. The agent engaged in flow can be said to be perfectly in tune with a system or process in the environment to such an extent that they can perfectly tune the functioning of their brain, body, environment, and conscious intention in one fluid motion, that involves the dynamically changing (yet predictable) elements of the environment.

Four Quadrants of Experience

Once time is integrated into the model, four quadrants emerge. We might think of these quadrants as different types of experience a consciousness can engage in, or processes a consciousness goes through in order to change its experience in a certain way. Figure 7 shows time and the feedback loops forming the four quadrants of experience. Each quadrant emphasizes a unique category of human experience and they will each be described in turn.

Figure 7: Consciousness provides four different types of feedback to the system relating to mind and body in the future and past

Mental Simulations of Intended Future

Once the agent unclamps perception, there are four basic ways an agent can tune the cognitive system. First, the consciousness can direct its attention to simulating future possibilities and trying to formulate some kind of plan or strategy explicitly. This is most commonly referred to as thinking and it corresponds to the top-right inner ‘wing’ of the EMC2. It provides a feedforward function that serves as an input to the brain to influence the types of things it is generating predictions about--it communicates to the brain what is relevant to the current situation and task at hand.

Mental Simulations of Past Experiences

Along this same vein, the agent can also simulate previous experience to gain additional experience by re-evaluating actions and possible alternatives that may have achieved different outcomes in past interactions. This is commonly referred to as reflection and it corresponds to the top-left ‘wing’ of the EMC2. It provides a feedback function that also serves as input to the brain to influence the formation of new perceptual logic. One strategy for accomplishing that is considering those actions not taken, which relate to the quadrant directly below it in the model--there is a reciprocal relationship between each quadrant in qualitatively different manners.

Explore Object Affordances

A second manner consciousness might modulate the information flow entering the system is through action. The consciousness can direct its attention to the world and view the perceptually salient ways of interacting with the world to help achieve its intention. This creates the experience of perceiving the world in terms of object affordances. Only those affordances that are relevant to the current intention (as filtered by perceptual logic) are perceptually available to the agent. Affordances correspond to the lower-right wing in the EMC2. Moving throughout the environment reveals new affordances, which correspondingly enter back through the sense of the agent, providing another feedback loop.

Act on the World

Finally, the agent can restructure its environment through interactions in a way that modulates the information flow by changing the source of that information, i.e. the environment itself. This is commonly referred to as action. Actions, by their definition happen proceeding into the past, which is why they are in the lower-left wing of the model. Only the very precise current piece of an action occurs in the precise ‘present’ of consciousness. Actions can be understood as stemming from the consciousness through the body and leaving a trace, or mark in the world in the the wake of their causal effects. This trace or mark re-enters the information flow of the consciousness through the senses as yet another feedback loop.

Consciousness Distributes Attention

Attention is a finite resources, and the role of the consciousness of the agent (when in unclamped cognition and to a certain extent clamped cognition) is delegating attention to these four operations in different ratio or proportions:

1. Simulating future interactions

2. Simulating past interactions

3. Exploring opportunities for interaction (affordances)

4. Performing actions

While in a clamped or structural coupling with the environment, the ratios of these feedback loops would remain relatively fixed. However, in early stages of sense-making, or when something happens to violate the expectations of the individual, agents decouple their interaction, unclamp their perception, and manually modulate the way that sensory data is entering the system and the types of predictions the brain is making. Thus, we can say that the feedback loop in each quadrant of the model has (at least) two states, clamped and unclamped.

Further, once perception is unclamped and consciousness directs attention to a particular quadrant of experience, the shape of the feedback loop in this representation has additional meaning to help categorize emergent cognitive states. There exists a state of potentiality for each quadrant if the agent where to unclamp and devote attentional and cognitive resources to that particular type of cognitive pursuit. These states of potentialities are represented in the initial EMC2 model as the dotted lines surrounding each of the feedback loop to show that there are multiple states and shapes of each feedback loop, as shown in the full depiction of EMC2 in Figure 8.

Figure 8: Full depiction of EMC2

Shaping Feedback Loops

Once we notice this particular feature of the cognitive model, we might be able to push the idea of the shape of feedback loops to a logical extreme to explore the implications. There exists at least two shapes that the feedback loop can take, namely broad or deep. Broad simulations represent a surface exploration far into the future, for example thinking about planning for graduating college would be a good example of this. Deep simulations represent an in depth analysis of one particular point in time, such as doing one homework assignment for a particular class. The detail and clarity of visualization for one slice in time represents a fundamentally different type of skill than the broad simulation of planning for graduation (a student could presumably be perfectly good at one of these while being terrible at the other).