embodied, active cognition
"being there"
"Ninety percent of life is just being there" - Woody Allen

Mind as controller
The main role of the mind is controlling the body, not logical reasoning about stored data
aka embodied cognition, wild cogniton,
integration of perception, cognition and action into an enaction framework (Varela et al 1991)
- Clark (1997), Introduction

"Enactive knowledge is not simply a multisensory mediated one, but knowledge stored in the form of motor reponses and acquired by the act of "doing". It is a form of cognition inherently tied to actions, as in the handcrafter as way of knowing. It is an intuitive non-symbolic form of learning"

There is research into new human-computer interfaces called enactive interfaces, using interactions rather than the symbols and icons that dominate current human-computer interfaces.

embodiment wiki (comprehensive)

Andy Clark

Andy Clark: Papers Available Online

Natural Born Cyborgs: Minds, Technologies and the Future of Human Intelligence (2003)
Mindware: An Introduction to the philosophy of cognitive science (2000)
Being There: Putting Brain, Body and World Together Again (1997)

Natural Born Cyborgs by Andy Clark (Edge article, good intro to his ideas)
" Cognitive technologies are best understood as deep and integral parts of the problem-solving systems that constitute human intelligence. They are best seen as proper parts of the computational apparatus that constitutes our minds. If we do not always see this, or if the idea seems outlandish or absurd, that is because we are in the grip of a simple prejudice: the prejudice that whatever matters about mind must depend solely on what goes on inside the biological skin-bag, inside the ancient fortress of skin and skull. But this fortress has been built to be breached. It is a structure whose virtue lies in part in it's capacity to delicately gear its activities to collaborate with external, non-biological sources of order so as (originally) to better solve the problems of survival and reproduction. ...

The conjecture, then, is that one large jump or discontinuity in human cognitive evolution involves the distinctive way human brains repeatedly create and exploit various species of cognitive technology so as to expand and reshape the space of human reason. We, more than any other creature on the planet, deploy non-biological elements (instruments, media, notations) to complement (but not, typically, to replicate) our basic biological modes of processing, creating extended cognitive systems whose computational and problem-solving profiles are quire different from those of the naked brain. Human brains maintain an intricate cognitive dance with an ecologically novel, and immensely empowering, environment: the world of symbols, media, formalisms, texts, speech, instruments and culture. The computational circuitry of human cognition thus flows both within and beyond the head.

Such a point is not new, and has been well-made by a variety of theorists working in many different traditions. I believe, however, that the idea of human cognition as subsisting in a hybrid, extended architecture (one which includes aspects of the brain and of the cognitive technological envelope in which our brains develop and operate) remains vastly underappreciated. We simply cannot hope to understand what is special and distinctively powerful about human thought and reason by merely paying lip-service to the importance of this web of surrounding technologies."

"BEING THERE" by Andy Clark

review: A stroll through the world of animats and humans
This review praises the book overall but also criticises Clark for insufficient discussion of the implications of enactivist theory "against realist and objectivist views of the world" ie. Clark embraces realism, unlike Varella, but does not explain clearly why


Ch 1. Outing the Mind
Ch. 2 The Situated Infant
Ch 3. Mind and World: The Plastic Frontier
Ch 4 Collective Wisdom, Slime-Mold-Style
Intermission: A Capsule History
Ch 5 Evolving Robots
Ch 6 Emergence and Explanation
Ch 7 The Neuroscientific image
Ch 8: Being, Computing, Representing
Ch 9. Minds and Markets
Ch 10 Language the Ultimate Artifact
Ch 11 Minds, Brains and Tuna: A Summary in Brine
Epilogue: A Brain Speaks


Preface: Deep Thought Meets Fluent Action
Descarte: vision of mind separate from body and world
"Ghost in the Machine" (Gilbert Ryle, 1949)
"Every intelligent ghost must contain a machine" (Aaron Sloman)

Matter / mind division persists in the way we study brain and mind, excluding as 'peripheral' the roles of the rest of the body and the local environment
It persists in the choice of problem domains we model, eg. chess by Deep Thought, when we still can't get a real robot to successfully navigate a crowded room and we still can't fully model the adaptive success of a cockroach
Pure Thought solutions are different from Embodied Thought solutions, eg. in getting a computer controlled machine to assemble tight fitting components
More fruitful to think of brains as controllers for embodied activities
Abandon the idea of neat dividing lines between perception, cognition and action
Abandon the idea of an executive centre of higher level reasoning
Abandon research methods that divorce thought from embodied action taking

Varela F., Thompson E and Rosch E. (1991) The Embodied Mind: Cognitive Science and Human Experience

Reading Hubert Dreyfus, What Computers Can't Do helped motivate Andy Clark to explore alternative models - the connectionist or parallel distributed processing approaches and explore biologically plausible images of mind and cognition

Introduction: A Car with a Cockroach Brain
"Ninety percent of life is just being there" - Woody Allen

What is human intelligence?
The mind is NOT a logical reasoning device coupled with a store of explicit data
The biological mind is, first and foremost, an organ for controlling the biological body

first approach: CYC (Douglas Lenat, ) begun in 1984
The bottleneck for adaptive intelligence is knowledge

CYC is good for expert systems but not an artificial mind
CYC has no coupling b/w the system and real world environment (senses, body)

second approach: cockroach
Hence the importance of the cockroach, those creatures of little mind who are neverthess very hard to swat. The roach has the abilties to:
- sense the wind disturbance caused by the motion of an attacking predator
- does not avoid contact with other roaches
- does not simply run at random but takes into account its own initial orientation, the presence of obstacles, the degree of illumination and the direction of the wind

How the cockroach does this is now understood (5)
To do it through explicit data storage and search would be too slow for the cockroach to escape

Ch 1. Outing the Mind

1.5 A Feel for Detail?

animate vision (29-31)

Ch. 2 The Situated Infant
2.2 Action Loops
slopes, darts experiments shows that some learning is context specific, geared to specific actions, generalised transfer does not occur

Crawlers learnt from experience to avoid steep slopes (20 degrees or more). But when they learned to walk they forget, two-thirds of the new walkers plunged down steep slopes. Steep slope awareness was action specific!

This phenomena not restricted to infants. Study of human adaptation to special glasses that shift the image to the right or the left. Subjects were then asked to throw darts at a board. Over time adaptation occured and they could aim as before. But the adaptation was motor loop specific. When asked to throw underhand or with non dominant hand there was no comparable improvement. There was adaptation for a particular throw, not generalised to other types of throws

Perception may be geared, from the outset, to specific action routines

2.3 Development without Blueprints

2.4 Soft Assembly and Decentralised Solutions

to explain the balance b/w individual variation and developmentally robust achievements we need the notion of soft assembly

human walking is soft assembly - it naturally compensates for icy sidewalks, blisters and high heeled shoes

centralised control via detailed inner models is inimical to such fluid adaptation

development of reaching behaviour in infants (lots of individual variation in how it is achieved)

2.5 Scaffolded Minds

affinity between soft assembly and the use of external scaffolding

Vygotsky type scaffolding, using language at the zone of promimal development

extend this to scaffolding by the inanimate environment

007 Principle: Know only as much as you need to know to get the job done - do not store or process information in costly internal ways when you can farm it out to the environment (this is a principle of parsimony)

2.6 Mind as Mirror vs. Mind as Controller

cognition might exploit real world action so as to reduce computational load

decentralised soft assembly in which mind, body and world act as equal partners in determining adaptive behaviour

models of representation as mirroring or encoding cf models of representation as control

... the brain should not be seen as primarily a locus of inner descriptions of external states of affairs; rather, it should be seen as a locus of inner structures that act as operators on the world via their role in determining actions

eg. Maja Mataric's neurobiology inspired robot rat navigates its environment
subsumption architecture
one layer: robot follows walls while avoiding collisions
second layer: detects and records landmarks
third layer: construct a map

Ch 3. Mind and World: The Plastic Frontier

3.1 The Leaky Mind

Mind is a leaky organ, forever escaping its "natural" confines and mingling shamelessly with body and with world

also see Clark's Law in 2004 Edge, "What's Your Law"
Clark's Law
Everything leaks.
There are no clear-cut level distinctions in nature. Neural software bleeds into neural firmware, neural firmware bleeds into neural hardware, psychology bleeds into biology and biology bleeds into physics. Body bleeds into mind and mind bleeds into world. Philosophy bleeds into science and science bleeds back.The idea of levels is a useful fiction, great for hygienic text-book writing and quick answers that defend our local turf but seldom advance scientific understanding).

brain as associative engine (explain?) and its environmental interactions as an iterated series of simple pattern completing computations
How can this account for the scale and depth of human cognitive success?
Part of the answer: language and culture sculpts our behaviour

3.2 Neural Networks: An Unfinished Revolution


useful for ... recognition of handwritten zip codes, visual processing, face recognition, signature recognition, robotic control, planning and automated theorem proving

but artificial inputs and outputs (58)

3.3 Leaning on the Environment
neural networks substitute pattern recognition for classical reasoning

How is long term planning and sequential reasoning possible?
long multiplication 7222 * 9422
pencil and paper
series of simple pattern completions coupled with external storage

Human brains are special - we both structure and exploit our external environment in ways that other animals cannt. But the computational differences may be less than is commonly believed

Examples from David Kirsch:
dieter: 3/4 of 2/3 of a cup of cheese - arrange 2/3 of a cup of cheese in a circle, split into 4 quadrants and serve 3 quadrants
(papert also discussed kitchen maths)

to repair alternator, take it apart and place pieces in a linear or grouped array, to make it easier to put it back together

ie. managing objects in space to simplify the information processing required by the brain

3.4 Planning and Problem Solving

distinction b/w pragmatic and epistemic action
pragmatic - alter the world to achieve a physical goal, eg. peel potatoes before boiling them
epistemic - act on the world to alter the nature of a mental task we want to perform (Kirsh & Maglio, 1994)
Think, for example, of moving Scrabble tiles around on their tray to see what words you can spell. Doing so changes the nature of the task from one of trying to come up with words inside one's head to one of completing patterns, a task our connectionist brains are good at. We find such external manipulations useful, suggesting that our in-the-head computational resources do not easily perform these tasks

Tetris - advanced players find it faster and more reliable to physically rotate zoids in preference to internally imagining the rotation

Kirsh & Maglio - we need to replace the separation of physical space and information processing space with unified physico-informational space

use of written notes (plans) for reordering and shifting focus of attention (more in Ch. 10)

3.5 After the Filing Cabinet

Where is the mind? .... Every thought is had by a brain. But the flow of thoughts and the adaptive success of reason are now seen to depend on repeated and crucial interactions with external resources. The role of such interactions ... is clearly computational and informational: it is to transform inputs, to simplify search, to aid recognition, to prompt associative recall, to offload memory and so on. ... human reasoners are distributed cognitive engines: we call on external resources to perform specific computational tasks, much as a networked computer may call on other networked computers to perform specific jobs. (69)

Brain and world collaborate in ways that are richer and more clearly driven by computational and informational needs than was previously suspected ...

... although specific thoughts remain tied to individual brains, the flow of reason and informational transformation it involves seem to criss cross brain and world. Yet it is this flow of ideas I suspect we most strongly associate with the idea of the mind as a seat of reason and of the self. ... The true engine of reason, we shall see, is bounded neither by skin nor skull

Ch 4 Collective Wisdom, Slime-Mold-Style

4.2 Two Forms of Emergence

direct emergence - relies on properties of and relations b/w individual elements, environmental conditions play only a background role
eg. temp and pressure emerge from interactions b/w molecules of a gas (multiple homogeneous)
eg. hydrogen + oxygen --> water (heterogeneous)

traffic jam
two simple rules will produce a traffic jam
1. slow down to avoid collision with car ahead
2. speed up if travelling under speed limit

indirect emergence -
stigmergic algorithm
stigma (sign),ergon (work) - the use of work as the signal for more work
eg. nest building behaviour of some termites
construction of arches
arches built through simple responses to local environment (chemical trace on mudballs)

complex adaptive behaviours arise without the need for leaders, blueprints or central planners

4.3 Sea and Anchor Detail

Edwin Hutchins
division of labour amongst crew for navigation
stigmergic procedure, waiting for local environmental prompt

external structures do large amount of work - technology such as nautical slide rule
technology makes computational problems more tractable to perceptual, pattern completing brains

work space structured to reduce complexity, eg. order of charts


4.4 The Roots of Harmony

4.5 Modeling the Opportunistic Mind

rational reconstruction won't do, input-output mapping of a series of problems

Nature is heavily bound by achieved solutions to previously encountered problems (81), cf. George Siemens: "Capacity to know more is more critical than what is currently known" (p.5, original connectivism paper)

embodied, active cognition
key features:
- real world, real time focus - biologically realistic time frames
- awareness of decentralised solutions
- extended vision of cognition and computation - spread over space and time, combination of inside and outside the head of individuals

key challenges:
- tractability in the face of cognitive leakage
- advanced cognition - ch 9&10
- identity - leakage of personal identity

Intermission: A Capsule History pp. 83-4
three stage progression of cognitive science

first stage: classical cognitivism
- memory as retrieval from a stored symbolic database
- problem solving as logical inference
- cognition as centralised
- the environment as (just) a problem domain
- the body as input device

second stage: connectionist artificial neural networks
replaced the first three characteristics with
- memory as pattern re-creation
- problem solving as pattern completion and pattern transformation
- cognition as increasingly decentralised

the body and the world remained marginalised

third stage: Embodied active cognition
connectionist trio plus
- the environment as an active resource whose intrinsic dynamics can play an important problem-solving role
- the body as part of the computational loop

to take body and world seriously enhances emergenist perspective
adaptive success depends as much on interactions among body, world and brain as in inner processes

challenges to third approach
- finding the right vocabulary to describe inner / outer processes

Ch 5 Evolving Robots
the use of simulated evolution as a means of generating control systems for robots

5.2 An Evolutionary Backdrop

Human design is modular, carefully decomposed into parts
Evolution does not work like that, it will produce cheap, distributed solutions to problems provided that its intermediate forms are robust
Francois Jacob likens evolution to a tinkerere rather than an engineer

Ch 6 Emergence and Explanation

6.1 Different Strokes?

What kind of tools are required to make sense of real-time, embodied, embedded cognition?

6.2 From Parts to Wholes
three (general) styles of cognitive scientific explanation

Componential Explanation (reductionist):

"Catch and Toss" Explanation: environment is still just treated as a source of inputs to the brain, but such inputs can lead to actions that simplify subsequent computations

Emergent Explanation: (106-113)

emergence, self organisation, circular causation
convection rolls when heating cooking oil in a pan
self organisation of slime mold cells (Ch. 4)
movement of a panicking crowd

collective variables - track properties that depend on the interaction of multiple components
eg. temp, pressure of gas; rate of acceleration of panicking crowd

definitions / descriptions:

one sense of emergence: parts of an entity
- collective behaviour of a large ensemble of simple components none of which is playing a leading or special role in controlling or orchestrating the process of pattern formation
- interesting, non centrally controlled behaviour ensues as a result of the interaction of multiple simple components within a system
- the action of the parts cause the overall behaviour and the overall behaviour guides the action of the parts

another sense: organism-environment interaction

6.3 Dynamical Systems and Emergent Explanation
6.4 Of Mathematicians and Engineers

6.5 Decisions, Decisions

6.6 The Brain Bites Back

Ch 7 The Neuroscientific image

Ch 8: Being, Computing, Representing

8.1 Ninety Percent of (Artificial) Life?

8.2 What is this thing called Representation?

8.3 Action Oriented Representation

Ch 9. Minds and Markets
9.1 Wild Brains, Scaffolded Minds

humans are special because of amazing capacity to create and maintain special external structures

three main moves
1 individual reasoning is a fast, pattern-completing style of computation
2 problem solving offloaded onto external structures and processes, which are social and institutional (rather than brute physical like earlier examples in the book)
3 public language for both social co-ordination and as tool for individual thought

"... reduce the loads on individual brains by locating those brains in complex webs of linguistic, social, political and institutional constraints"

"dissipate reasoning"
"diffuse knowledge and wisdom through social structures"

brain + external scaffolding = mind (inference engine)

9.2 Lost in the Supermarket

rational agent of classical economics: eg. bean-selecting agent when shopping in the Supermarket
The agent has some preexisting and comprehensive set of weighted preferences about quality, cost etc.
the paradigm of substantive rationality

substantive rationality works best in highly scaffolded cases

9.3 The Intelligent Office?

9.4 Inside the Machine

Organisations, factories, offices, institutions are large scale scaffolds for individual cognitive success. On the other hand there is individual communication and individual problem solving which alter the large scale scaffolds.

Reciprocal relationships have been modelled by Hutchins, Cogniton in the Wild, using neural networks
Constraint-satisfaction networks consist of excitatory links connecting mutually supportive features and inhibitory links connecting mutually inconsistent features
eg. a feature like "is a dog" coded by excitatory links to "barks" and "has fur" and inhibitory links to "meows" and "is a cat"

Once such a network interprets input (eg. all dog feature units become active) it becomes difficult to dislodge.
This corresponds to confirmation bias, the tendency to ignore, discount or creatively reinterpret evidence that goes against a hypothesis or model that is already in place (eg. a solitary "meow")

MORE (complex and inconclusive)

9.5 Designer Environments

Ch. 10 Language: The Ultimate Artifact

10.1 Word Power

the emphasis on language as a medium of communication tends to blind us to the role of language as a tool that alters the nature of the computational tasks involved in various kinds of problem solving
- the view of language as a computational transformer
cf scissors which fit the human hand and allow us to perform new tasks (two way fit)
designed artifacts as amplifiers of intelligence and repositories of achieved knowledge and wisdom
Dennett's Gregorian creatures

language enables us to reshape difficult tasks into formats better suited to basic computational capacities of human brain

10.2 Beyond Communication


language as a computational transformer that allows pattern completing brains to tackle otherwise intractable classes of cognitive problems

private speech
scaffolded action, within the zone of proximal development
scaffolded means requires some kind of external support, meaning use of tools or the knowledge and skills of others - all cases in which external structures are coopted to aid problem solving (broader than some definitions of scaffolding)
physical, cognitive and social augmentation
use of a compass and pencil to draw a circle
role of other crew members to allow a ships pilot to steer a course
infant taking first steps, while supported by a parent
talking through a challenge such as tying shoelaces

private speech dovetails with scaffolding
child can conduct a dialogue with self, talking through a task that was previously explained
the function of this private speech is to guide behaviour, to focus attention, to guard against errors
language is a tool for structuring and controlling action (not merely for information exchange)

self directed speech allows us to highlight the most puzzling features of new situations

Gauker 1990 "How to Learn a Language like a chimpanzee"
public language as a tool for effecting change in ones environment
cause-effect relationship, linguistic signs produce certain effects

different ways to view language as a tool:
- directly causal, like reaching out a hand and grabbing a cake
- the self directed utterance of words and phrases has certain effects on our own behaviour
( Dennett (1991), Consciousness Explained Ch 7&8 and (1995) Darwins Dangerous Idea Ch, 13)
- as a means of altering the shape of computational problem spaces (see 10.3)

How does self directed speech work? How can we tell ourselves something we don't already know!!

attempts at explanation:

inner thinking is literally done in inner speech
one does not have a private thought and then write it down, the thinking is the writing

or (Clark modification), treat the writing as an environmental manipulation that transforms the problem space for human brains

linguaform inputs have distinctive effects on some inner computational devices


10.3 Trading Spaces
trading external symbol structures against time and labour intensive internal computations

Some notation systems are superior to others, eg. Arabic numerals cf Roman

use of external symbol media to offload memory onto the world: texts, diaries, notebooks

use of labels as environmental simplification, eg. signs

Use of language in coordinating actions, eg. lists

inner rehearsal of speech in manipulating our own attention and guiding our allocation of cognitive resources

the role of reaction and linguaform reflection in expert performance at the computer game tetris

the problem of path dependence, certain ideas can only be understood once other ideas are in place
(language allows us to transcend this problem)

two types of scaffolding - mythic and theoretic

summing up:
- we trade culturally achieved representation against individual computation
- we use words to focus, clarify, transform, offload and control our own thinking
- language is part and parcel of reason itself

10.4 Thoughts about Thoughts: The Mangrove Effect

self-evaluation, self-criticism,
true intelligence, off line thinking
thinking about thinking

10.5 The Fit of Language to the Brain

is language like a paper clip?

10.6 Where does the Mind Stop and the Rest of the World Begin?

some human actions are like thoughts
epistemic actions

certain harms to the environment have moral significance that we normally associate with harm to the person

we are prone to confuse the mental with the conscious

the Naked Mind

language is so ubiquitous it is almost invisible

Ch 11 Minds, Brains and Tuna: A Summary in Brine

Epilogue: A Brain Speaks

I am John's brain ...

I do not have John's thoughts. John has John's thoughts, and I am just one item in the array of physical events and processes that enable the thinking to occur

John thinks I am the point source of his ideas - ideas which he thinks he captured on paper as a mere convenience and a hedge against forgetting ... But my role is more delicately constituted than John's simple image suggests ... I acted as a mediating factor in some complex feedback loops encompassing John and selected chunks of his local environment. I spent the day in a variety of close and complex interactions with a nummber of external props. ... My role, as best as I can recall, was to support John's rereading of a bunch of old material and notes, and to react to those materials by producing a few fragmentary ideas and criticsms. These small responses were stored as further marks on paper and in margins. Later on, I played a role in the reorganisation of these marks on clean sheets of paper, adding new on-line reactions to the fragmentary ideas. The cycle of reading, responding and external reorganisation was repeated again and again. At the end of the day, the "good ideas" with which John was so quick to credit me emerged as the fruits of these repeated interactions between me and the various external media. Credit thus belongs not so much to me as to the spatially and temporally extended process in which I play a role

I have allowed myself to speak as if I was a unified inner resource ... this is an illusion which the present literary device encourages ... I am not one inner voice but many. I am so many inner voices that the metaphor of the inner voice must mislead, for it suggests inner subagencies of some sophistication and perhaps possessing a rudimentary self-consciousness ... I consist only of multiple mindless streams of highly parallel and often relatively independent computational processes. I am not a mass of little agents

he supposes that I store information in clusters that respect what he calls "concepts"

John's language, introspections and oversimplistic physicalism incline him to identify my organisation too closely with his own limited perspective

... John really knows very little about me. Think of me as the Martian in John's head.

other reference:

Varela F., Thompson E and Rosch E. (1991) The Embodied Mind: Cognitive Science and Human Experience