The way in which the components of associative learning - classical conditioning and operant conditioning - contribute to learning in the fruit fly Drosophila melanogaster, has been extensively studied by Brembs (2000), Brembs and Heisenberg (2001) and Heisenberg, Wolf and Brembs (2001), in experiments using a flight simulator.

Fruit flies display many kinds of behavioural flexibility in their patterns of bodily movement, or "ethogram". In the experiments, the range of the fruit flies' bodily movements was drastically restricted in a highly unnatural fashion. Each fly, after being immobilized by cold-anaesthesia and glued by the head and thorax to a triangular copper hook, was left overnight with some food (sucrose) in a small moist chamber. The next day, it was attached to a torque meter via a clamp and performed tethered flight in the centre of a cylindrical panorama. The torque meter measured the fly's tendency to turn around its vertical body axis - i.e. its yaw torque. The fly was held horizontally, as if cruising at high speed. The head was glued to the thorax. The fly was unable to turn or shift its gaze or its orientation, but was free to bend its abdomen, extend its proboscis, beat its wings and move its legs. The torque meter to which the fly was attached measured the difference between the fly's left and right wing movements, and interpreted this as an "intended" side-to-side turn.

Most of the experiments were designed to monitor the flexibility of a single behavioural variable in the fly: yawing (movement from side to side). Fruit flies, when allowed to fly freely, do not fly at just one angle, but spontaneously vary their angle of flying over the entire yaw torque range. Fluctuations in yaw torque may be slow or rapid (jerky body movements, known as "spikes"). Torque spikes, generated when the fly reduces its wing beat amplitude on the side to which it "intends" to turn, enable the fly to make sudden turns to the left or right. In the experimental set-up, when the fly "attempted" to make a side-turn, it did so by generating torque spikes.

The flight simulator used in the experiments was a computer-controlled feedback system in which a fly was allowed to control, by its yaw torque, the rotations of a cylindrical panorama surrounding it. In some experiments, the cylindrical arena was used to present a visual stimulus - either a colour or a pattern (a conditioned stimulus). The fly's tendency to perform left or right turns (yaw torque) during tethered flight was fed into the computer. The computer controlled the background visual display of colors and/or patterns perceived by the fly. Thus although the fly could not actually turn left or right, the rotations of the panorama could be used to provide the fly with artificial sensory feedback from its "intended" turns, which mimicked the feedback it would receive in free flight from turning. It also regulated a heat beam which could be focused onto the fly. The heat beam served as an unconditioned stimulus - in this case, a "negative reinforcer" or "punishment". Thirty seconds of continuous exposure to a heat beam would incinerate the fly. A torque meter measured the fly's angular momentum around its vertical body axis - its yaw torque.

The description of the heat beam used in the experiments as a "punishment" raises the question of whether a fly's ability to be conditioned is grounded in its capacity to experience subjective mental states (e.g. pain and fear). Does the heat hurt the fly? And does the fly consciously fear an aversive stimulus? These questions will be further discussed in chapter 4. For the time being, it should be noted that avoidance behaviour alone cannot serve as an indicator of pain or fear: bacteria, whose behaviour is readily explicable using a goal-centred intentional stance, also avoid aversive stimuli.

Brembs (2000) tested fruit flies in four distinct "learning modes", where they were subjected to different combinations of classical and operant conditioning. Two modes were mono-dimensional tasks, where the fly had to form just one association. In operant conditioning, the association was between its behaviour (B) and a "punishing" unconditioned stimulus (US); in classical conditioning, between a visual pattern on the cylindrical panorama (CS) and the US. The other two modes were composite or multi-dimensional tasks, as the fly had the opportunity to form two associations: either B -> US and CS -> US (switch mode) or B -> CS and CS -> US (flight simulator mode).

Brembs (2000) measured the fly's ability to learn from pure operant conditioning by subjecting the fly to heat when it turned too far to the left or right. In this "yaw torque" mode of learning, the fly learned to avoid heat by restricting its yaw torque range, for example, to the left domain if straying into the right domain was punished. The fly thus formed an association between its behaviour (B) of initiating a right turn, and an unconditioned stimulus (US): heat. However, it had to learn this "blind" - i.e. without the help of any external guidance indicating whether it was flying inside or outside the permitted range. Heisenberg, Wolf and Brembs (2001) comment:

This is a remarkable feat, as ... in no natural situation would continuous turning to one side bring the fly permanently out of a heated zone (2001, p. 1).

In Brembs' classical conditioning experiments, a certain visual pattern (CS) on the cylindrical arena was followed by a punishment (US): the fly was subjected to heat. In this set-up, there was nothing the fly could do to avoid the punishment, but subsequent testing showed that it had learned to associate the pattern (CS) with the punishment (US).

In flight simulator mode (fs-mode), an attempt was made to realistically simulate the changes in the fly's visual field that the fly's turns would cause in free flight, except that the flight simulation was maintained at a constant speed at an undefined altitude. The cylindrical arena was decorated with four black T-shaped patterns of alternating orientation - two upright and two inverted - evenly spaced on the wall so that each quadrant had one T-shape. The arena's angular speed was made proportional to the fly's yaw torque, but the direction of rotation was opposite to that of the fly's turning. By adjusting its yaw torque, the fly was able to stabilise the arena and control the direction of its simulated flight (i.e. fly "straight" relative to the patterns on the wall). The fly was punished by heat (the US) when it flew into a forbidden quadrant, marked with an upright T, and it learned that quadrants associated with an inverted T were safe. In this learning mode, the fly was unable to form a direct association between its behaviour (B) and the heat (US), because the same behaviour (torque spikes) could either bring the fly into the heat or out of it, depending on which quadrant it was in. Instead, the fly had to form two associations. First, it had to associate its behaviour (B) of attempting to fly in the direction of a forbidden quadrant, with the CS (upright T) in that quadrant. Second, it had to associate the CS (upright T) with the aversive property of the US (heat), and learn that the former predicted the latter. This kind of learning is known as operant stimulus conditioning. It proved to be considerably more effective than either classical or pure operant conditioning alone. It was also a more natural way of learning. Whereas in the pure operant conditioning experiments, the fly had to restrict its range and continuously turn to one side to avoid the heat, in flight simulator mode, it simply had to keep its left and right yaw torque spikes within the safe zones.

In switch mode (sw-mode), the fly received two forms of feedback - heat (the US) and a change in the colour (from blue to green) or pattern (from an upright to an inverted T) of the panorama (the CS) - when its yaw torque exceeded the allowable range. For instance, if the fly's yaw torque value fell into the left domain, heat was switched on and the upright T pattern was placed in front; flying in the right domain heat switched the heat off and the arena was quickly rotated by 90 degrees, shifting the inverted T to the front. Thus the fly underwent parallel operant conditioning (a combination of operant and classical conditioning), where it not only had the opportunity to associate its behaviour (B) with a US (heat), but also had the chance to associate a particular display - a colour or pattern (CS) - with the US. Unlike flight simulator mode, the fly had to respond in a very artificial way to avoid the heat: it had to keep flying around in circles to avoid being fried. The requisite behaviour was thus the same as for pure operant conditioning. Brembs wryly comments:

I find it hard to imagine a situation where flying in clockwise circles gets a fly incinerated, but counterclockwise circles prevent that (personal email, 11 August 2003).

In other research (Brembs, 2003), it was shown that flies could stabilise a rotating cylindrical arena by modulating their thrust instead of their yaw torque. Flies learned after a few seconds to adjust their thrust to an arbitrary level (chosen by the experimenter) that corresponded to no rotation of the arena. Less thrust caused the arena to rotate one way (say, counter-clockwise), while more thrust caused it to rotate the other way (clockwise).

Summary of Brembs' key research findings

In Brembs' experiments (2000), classical conditioning and operant conditioning were shown to be independent learning processes.

It was also demonstrated that operant conditioning could assist (or boost) learning via classical conditioning, and that classical associations could be formed during operant training.

Flies performed much better in fs-mode than in classical conditioning, where they were powerless to control their circumstances. "As expected, the more natural complex learning tasks are easier to solve than the more artificial single-association tasks" (Brembs, 2000, p. 30).

Generally, flies learned best in natural situations involving complex associations than in artificial set-ups with single-association tasks (pure operant conditioning or classical conditioning).

Flies were also able to transfer colour or pattern preferences acquired during one mode of training (sw-mode) to another mode of learning (fs-mode).

It was shown that flies could remember patterns without heat reinforcement and compare them to other patterns later.

Finally, the flies' ability to learn compound stimuli was investigated by using both colors and patterns as visual cues. It was shown for the first time that Drosophila could learn compound stimuli and recall the individual components independently and in similar proportions (Brembs, 2000, p. 31). Interestingly, it was discovered that "flies acquire, store and retrieve the two compound stimuli 'colors' and 'patterns orientations' separately. They do not store them only as a compound" (Brembs, 2000, p. 30).

1.2 Merfeld's model of efference copy

Merfeld (2001, p. 189) summarises the history of the notion of an efference copy:

Sperry (1950) and von Holst and Mittelstaedt (1950) independently suggested that motor commands must leave an image of themselves (efference copy) somewhere in the central nervous system that is then compared to the afference elicited by the movement (reafference). It was soon recognised that the efference copy and reafference could not simply be compared, since one is a motor command and the other is a sensory cue (Hein & Held 1961; Held 1961).

Merfeld (2001) has developed his own model of how the nervous system forms an internal representation of the dynamics of its sensory and motor systems:

The primary input to this model is desired orientation, which when compared to the estimated orientation yields motor efference via a control strategy. These motor commands are filtered by the body dynamics (e.g., muscle dynamics, limb inertia, etc.) to yield the true orientation, which is measured by the sensory systems with their associated sensory dynamics to yield sensory signals. In parallel with the real-world body dynamics and sensory dynamics, a second neural pathway exists that includes an internal representation of the body dynamics and an internal representation of the sensory dynamics. Copies of the efferent commands (efference copy) are processed by these internal representations to yield the expected sensory signals, which when compared to the sensory signals yield an error (mismatch). This error is fed back to the internal representation of body dynamics to help minimize the difference between the estimated orientation and true orientation (2001, p. 189).

1.3 Definition of Operant Agency

Note: This definition is the same as that in Part C of chapter two. I have included it here purely for the sake of convenience, so that the reader can compare the conditions for operant agency with those for the other three forms of agency described in this Appendix: navigational agency, tool agency and social agency.

Definition - "operant conditioning"
An animal can be described as undergoing operant conditioning if the following features can be identified:

(i) innate preferences or drives;

(ii) innate motor programs, which are stored in the brain, and generate the suite of the animal's motor output;

(iii) a tendency on the animal's part to engage in exploratory behaviour;

(iv) an action selection mechanism, which allows the animal to make a selection from its suite of possible motor response patterns and pick the one that is the most appropriate to its current circumstances;

(v) fine-tuning behaviour: efferent motor commands which are capable of stabilising a motor pattern at a particular value or within a narrow range of values, in order to achieve a goal;

(vi) a current goal: the attainment of a "reward" or the avoidance of a "punishment";

(vii) sensory inputs that inform the animal whether it has attained its goal, and if not, whether it is getting closer to achieving it;

(viii) direct or indirect associations between (a) different motor commands; (b) sensory inputs (if applicable); and (c) consequences of motor commands, which are stored in the animal's memory and updated when circumstances change;

(ix) an internal representation (minimal map) which includes the following features:

(a) the animal's current motor output (represented as its efference copy);

(b) the animal's current goal (represented as a stored memory of the motor pattern or sensory stimulus that the animal associates with the attainment of the goal); and

(c) the animal's pathway to its current goal (represented as a stored memory of the sequence of motor movements or sensory stimuli which enable the animal to steer itself towards its goal);

(x) the ability to store and compare internal representations of its current motor output (i.e. its efferent copy, which represents its current "position" on its internal map) and its afferent sensory inputs;

(xi) a correlation mechanism, allowing it to find a temporal coincidence between its motor behaviour and the attainment of its goal;

(xii) self-correction, that is:

(a) an ability to rectify any deviations in motor output from the range which is appropriate for attaining the goal;

(b) abandonment of behaviour that increases, and continuation of behaviour that reduces, the animal's "distance" (or deviation) from its current goal; and

(c) an ability to form new associations and alter its internal representations (i.e. update its minimal map) in line with variations in surrounding circumstances that are relevant to the animal's attainment of its goal.

If the above conditions are all met, then we can legitimately speak of the animal as an intentional agent which believes that it will get what it wants, by doing what its internal map tells it to do.

Definition - "operant agency"
Operant conditioning is a form of learning which presupposes an agent-centred intentional stance.

Animals that are capable of undergoing operant conditioning can thus be said to exhibit a form of agency called operant agency.

2. Navigational Agency

2.1 Definition of Navigational Agency

The following definition has been adapted from the definition of operant agency in section 1.3 above.

Definition - "navigational agency"
We are justified in ascribing agency to a navigating animal if the following features can be identified:

(i) innate preferences or drives;

(ii) innate motor programs, which are stored in the brain, and generate the suite of the animal's motor output;

(iii) a tendency on the animal's part to engage in exploratory behaviour, in order to locate food sites;

(iv) an action selection mechanism, which allows the animal to make a selection from its suite of possible motor response patterns and pick the one that is the most appropriate to its current circumstances;

(v) fine-tuning behaviour: efferent motor commands which are capable of steering the animal in a particular direction - i.e. towards food or towards a visual landmark that may help it locate food;

(vi) a current goal (long-term goal): the attainment of a "reward" (usually a distant food source);

EXTRA CONDITION:
(vi*) sub-goals (short-term goals), such as landmarks, which the animal uses to steer itself towards its goal;

(vii) visual sensory inputs that inform the animal about its current position, in relation to its long-term goal, and enable it to correct its movements if the need arises;

(viii) direct or indirect associations (a) between visual landmarks and local vectors; (b) between the animal's short term goals (landmarks) and long term goals (food sites or the nest). These associations are stored in the animal's memory and updated when circumstances change;

(ix) an internal representation (minimal map) which includes the following features:

(a) the animal's current motor output (represented as its efference copy);

(b) the animal's current goal (represented as a stored memory of a visual stimulus that the animal associates with the attainment of the goal) and sub-goals (represented as stored memories of visual landmarks); and

(c) the animal's pathway to its current goal, via its sub-goals (represented as a stored memory of the sequence of visual landmarks which enable the animal to steer itself towards its goal, as well as a sequence of vectors that help the animal to steer itself from one landmark to the next);

(x) the ability to store and compare internal representations of its current motor output (i.e. its efferent copy, which represents its current "position" on its internal map) and its afferent sensory inputs. Motor output and sensory inputs are linked by a two-way interaction;

[NOT NEEDED:
(xi) a correlation mechanism, allowing it to find a temporal coincidence between its motor behaviour and the attainment of its goal;]

(xii) self-correction, that is:

(a) an ability to rectify any deviations (or mismatches) between its view and its internally stored image of its goal or sub-goal - first, in order to approach its goal or sub-goal, and second, in order to keep track of it;

(b) abandonment of behaviour that increases, and continuation of behaviour that reduces, the animal's "distance" (or deviation) from its current goal; and

(c) an ability to form new associations and alter its internal representations (i.e. update its minimal map) in line with variations in surrounding circumstances that are relevant to the animal's attainment of its goal.

If the above conditions are all met, then the animal can be said to exhibit what I will call navigational agency. Such an animal qualifies as an intentional agent which believes that it will get what it wants, by doing what its internal map tells it to do.

2.2 Case study: Navigational agency in insects

The current state of research into spatial learning in insects remains fluid. What is not disputed is that some insects (especially social insects, such as the ants, bees and wasps) employ a highly sophisticated system of navigation, and that they employ at least two mechanisms to find their way back to their nests: path integration (also known as dead reckoning) and memories of visual landmarks. A third mechanism - global (or allocentric) maps - has been proposed for honey bees, but its status remains controversial.

I argue in section (a) that path integration does not require the use of minimal maps and hence does not qualify as a case of intentional agency. In section (b), I examine the way insects use visual landmarks to steer by. The evidence suggests that this kind of navigation does indeed require "minimal maps". Additionally, the continual self-monitoring behaviour of insects navigating by landmarks suggests that they are indeed in control of their bodily movements, and that they construct their own visual maps using a flexible learning process. I conclude that insects that navigate using visual landmarks are bona fide intentional agents. Finally, in part (c) I outline the current evidence for and against the hypothesis that some insects use cognitive maps to navigate.

2.2(a) Path integration (dead reckoning)

The best studied mechanism is path integration, which allows insects to navigate on bare terrain, in the absence of visual landmarks. Collett and Collett (2002, p. 546) describe it as follows:

When a honeybee or desert ant leaves its nest, it continually monitors its path, using a sun and polarized light compass to assess its direction of travel, and a measure of retinal image motion (bees) or motor output (ants) to estimate the distance that it covers. This information is used to perform path integration, updating an accumulator that keep a record of the insect's net distance and direction from the nest.

The insect's working memory of its "global" position - its distance and direction from the nest - is continually updated as it moves. An insect's global vector allows it to return home in a straight line from any point in its path. Tests have shown that if a desert ant returning to its nest is moved to unfamiliar terrain, it continues on the same course for a distance roughly equal to the distance it was from the nest. Finding no nest, it then starts a spiral search for the nest. If obstacles are placed in its path, the ant goes around them and adjusts its course appropriately (Collett and Collett, 2002, p. 546; Gallistel, 1998, p. 24; Corry, 2003). When ants have to navigate around obstacles, they memorise the sequence of motor movements corresponding to movement around the obstacle, thereby cutting their information processing costs (Schatz, Chameron, Beugnon and Collett, 1999).

Can we describe the navigation systems of insects as representational? If so, is it a map-like representation, like the one I proposed for operant conditioning, and does it qualify an insect to be a belief-holder?

Is path integration (dead reckoning) representational?

Gallistel (1998) regards the nervous system of an insect as a computational system and argues that the ability of insects to perform dead reckoning is strong evidence that their brains are symbol processors, which can represent their movements. The ability of insects' nervous systems to perform computations using the azimuth position of the sun and the pattern of polarised light in the sky, although impressive, does not entail that they have mental states (Conclusion C.3) - as we have seen, even viruses can compute.

More importantly, Gallistel contends that because there is a functional isomorphism between the symbols and rules in an insect's nervous system and events in the outside world, we can call it a representation. The nervous system contains vector symbols: neural signals which represent the insect's net displacement from the nest, and are continually updated as the insect moves about. The current sum of the displacements made by the insect while moving represents its present position and at the same time tell it how to get home.

As Gallistel puts it:

The challenge posed by these findings to those who deny symbol processing capacity in [insect] brains is to come up with a process that looks like dead reckoning but really is not (1998, p. 24).

Insects' ability to hold a course using a sun-compass and to determine distance from parallax are other examples of insect representation cited by Gallistel.

It was argued earlier that a bona fide representation should contain within itself the possibility of mis-representation (Conclusion R.1). The case of the displaced ant, which sets a path that would normally return it to its nest, seems to meet this criterion. Likewise, the honeybee has a built-in visual odometer that estimates distance travelled by integrating image motion over time. The odometer misreads distances if the bee collects food in a short, narrow tunnel (Esch, Zhang, Srinivasan and Tautz, 2001).

Corry (2003) points out that isomorphism is not a sufficient condition for a representation: the ability of system A to track system B does not mean that A represents B. However, he proposes that if an individual (e.g. an ant) uses a system (e.g. its internal home vector system) which is functionally isomorphic with the real world, in order to interact with the real world, then the system can be said to represent the real world. This condition is satisfied in the case of dead reckoning by an ant. Can we call such a representation a belief?

Ramsey proposed that a "belief of the primary sort is a map of neighbouring space by which we steer" (1990, p. 146). However, if we look at the path integration system alone, the representation fails to meet even the minimal conditions which I suggested in the section above for a minimal map, as neither the goal nor the current position is represented: only the directional displacement from the goal is encoded. Without an internal representation of one's goal, we cannot speak of agency, or of control. A naive foraging insect, relying solely on path integration, is merely following its internal compass, which is continually updated by events beyond its control.

The same remarks apply to the ability of monarch butterflies to navigate from as far north as Massachusetts to central Mexico, in autumn. Scientists have recently discovered that the monarch uses an in-built sun-compass and a biological clock to find its way (Mouritsen and Frost, 2002). Once again, the butterflies appear to be lack any internal representation of their goal (a place with warm, tropical weather where they can spend the winter, as they do not hibernate) or their destination (Mexico). Thus we cannot describe them as agents when they migrate.

2.2(b) Landmark navigation

The other way by which insects navigate is the use of landmarks which they observe en route and at their home base. Naive foragers initially rely on path integration to find their way home, but when they repeat their journey, they learn the appearance of new landmarks and associate local vectors with them, which indicate the distance to the next landmark. With experience, these local vectors take precedence over the global vectors used in dead reckoning.

[T]he basic structure of each route segment is a landmark and an associated local vector or some other stereotyped movement (for example, turning left)... [T]he primary role of a landmark is to serve as a signpost that tells the insect what to do next, rather than as a positional marker... (Collett and Collett, 2002, p. 547).

A landmark navigation system can also work hand-in-hand with path integration: for instance, the Mediterranean ant uses the latter as a filter to screen out unhelpful landmarks, learning only those that can guide subsequent return trips to the nest - particularly those that are close to the nest (Schatz, Chameron, Beugnon and Collett, 1999). Visual landmarks help keep path vector navigation calibrated, as short local vectors are more precise than a route-long global vector, while the learning of visual landmarks is guided by path integration (Collett and Collett, 2002).

It is still not known how insects encode landmarks, and what features of the image are stored in its memory. However, it is agreed that ants, bees and wasps not only memorise landmarks, but guide their final approach to their goal by matching their visual image of the landmark with their stored image (snapshot?) of how it looks from the goal. It appears that when comparing their current view to a stored image, they use the retinal positions of the edges, spots of light, the image's centre of gravity and colour. They also learn the appearance of an object from more than one distance, as their path home is divided into separate segments, each guided by a separate view of the object. Additionally, consistency of view is guaranteed because the insect, following the sun or some other cue, always faces the object in the same direction. Finally, the insect's view of the distant panorama from a landmark can help to identify it (Collett and Collett, 2002).

Can landmarks serve as maps to steer by?

Path integration on its own works perfectly well even for an untrained insect: it does not require the insect to associate a motor pattern or sensory stimulus with attaining its goal. Navigation by landmarks, on the other hand, requires extensive learning. The location and features (colour, size, edge orientation and centre of gravity) of each landmark have to be memorised. Multiple views of each landmark have to be stored in the insect's brain. Additionally, some insects use panoramic cues to recognise local landmarks. Finally, a local vector has to be associated with each landmark (Collett and Collett, 1998, 2002). The fact that insects are capable of learning new goals and new patterns of means-end behaviour means that they satisfy a necessary condition for the ascription of mental states (Conclusion L.12) - though by itself not a sufficient one, as we have seen that associative learning can take place in the absence of mental states (Conclusion L.14).

While navigating by landmarks, insects such as ants and bees can learn to associate their final goal (the nest) with views of nearby landmarks, which guide them home, even when their final goal is out of sight. These insects often follow "a fixed route that they divide into segments, using prominent objects as sub-goals" (Collett and Collett, 2002, p. 547).

It is a matter of controversy whether insects possess a global (or allocentric) map of their terrain, which combines multiple views and movements in a common frame of reference (Giurfa and Menzel, 2003; Collett and Collett, 2002; Harrison and Schunn, 2003; Giurfa and Capaldi, 1999; Gould, 1986, 2002). However, even if an insect has "only a piecemeal and fragmented spatial memory of its environment" (as suggested by Collett and Collett, 2002, p. 549), it clearly meets the requirements for a minimal map. Its current position is represented by the way its nervous system encodes its view of the external world (either as a visual snapshot or as a set of parameters), its short-term goal is the landmark it is heading for, and the path is its local vector, which the insect recalls when the landmark comes into view (Collett and Collett, 2002, pp. 546, 547, 549). The map-like representation employed here is a sensory map, which uses the visual modality.

What are the goals of navigating insects?

For a navigating insect, its long-term goals are food, warmth and the safety of the nest, all of which trigger innate responses. Short term goals (e.g. landmarks) are desired insofar as they are associated with a long term goal. Even long term goals change over time, as new food sources supplant old ones.

Additionally, insects have to integrate multiple goals, relating to the different needs of their community (Seeley, 1995; Hawes, 1995). In other words, they require an action selection mechanism. For example, a bee hive requires a reliable supply of pollen, nectar, and water. Worker field bees can assess which commodity seems to be in short supply within the hive, and search for it.

Are navigating insects agents?

The statement by Collett and Collett that "the primary role of a landmark is to serve as a signpost that tells an insect what to do next" (2002, p. 547), recalls Ramsey's claim that a belief is a map by which we steer. Should we then attribute agency, beliefs and desires to an insect navigating by visual landmarks?

Corry (2003) thinks not, since the insect is not consciously manipulating the symbols that encode its way home. It does not calculate where to go; its nervous system does. Corry has a point: representations are not mentalistic per se. As we have seen, the autonomic nervous system represents, but we do not say it has a mind of its own. Nevertheless, Corry's "consciousness" requirement is unhelpful. He is proposing that a physical event E (an insect following a home vector) warrants being interpreted as a mental event F (the insect is trying to find its way home) only if E is accompanied by another mental event G (the insect must be consciously performing the calculations), but he fails to stipulate the conditions that have to be satisfied in order for G to be met.

I would suggest that Ramsey's steering metaphor can help us resolve the question of whether insects navigate mindfully. The word "steer", when used as a transitive verb, means "to control the course of" (Merriam-Webster Online, 2004). Thus steering oneself suggests control of one's bodily movements - a mindful activity, on the account I am developing in this chapter. The autonomic nervous system, as its name suggests, works perfectly well without us having to control it by fine-tuning. It is presumably mindless. On the other hand, Drosophila at the torque meter needed to control its motor movements in order to stabilise the arena and escape the heat. Which side of the divide does insect navigation fall on?

Earlier, I proposed that an animal is controlling its movements if it can compare the efferent copy from its motor output with its incoming sensory inputs, and make suitable adjustments. The reason why this behaviour merits the description of "agency" is that fine-tuned adjustment is self-generated: it originates from within the animal's nervous system, instead of being triggered from without. This is an internal, neurophysiological measure of control, and its occurrence could easily be confirmed empirically for navigating insects.

In the meantime, one could use external criteria for the existence of control: self-correcting patterns of movement. The continual self-monitoring behaviour of navigating insects suggests that they are indeed in control of their bodily movements. For instance, wood ants subdivide their path towards a landmark into a sequence of segments, each guided by a different view of the same object (Collett and Collett, p. 543). Von Frisch observed that honeybees tend to head for isolated trees along their route, even if it takes them off course (Collett and Collett, p. 547). Insects also correct for changes in their environmental cues.

Two facts may be urged against the idea that a navigating insect is exercising control over its movements, when it steers itself towards its goal. First, insects appear to follow fixed routines when selecting landmarks to serve as their sub-goals (Collett and Collett, 2002, p. 543). Second, cues in an insect's environment (e.g. the panorama it is viewing) may determine what it remembers when pursuing its goal (e.g. which "snapshot" it recalls - see Collett and Collett, 2002, p. 545). In fact, it turns out that no two insects' maps are the same. Each insect's map is the combined outcome of:

(i) its exploratory behaviour, as it forages for food;

(ii) its ability to learn about its environment;

(iii) the position and types of objects in its path;

(iv) the insect's innate response to these objects; and

(v) certain fundamental constraints on the kinds of objects that can serve as landmarks (Collett and Collett, 2002, pp. 543, 548-549).

As each insect has its own learning history and foraging behaviour, we cannot say that an insect's environment determines its map.

We argued above that control requires explanation in terms of an agent-centred intentional stance, as a goal-centred intentional stance is incapable of encoding the two-way interplay between the agent adjusting its motor output and the new sensory information it is receiving from its environment, which allows it to correct itself. In a goal-centred stance, the animal's goal-seeking behaviour is triggered by a one-way process: the animal receives information from its environment.

Finally, we cannot speak of agency unless there is trying on the part of the insect. The existence of exploratory behaviour, coupled with self-correcting patterns of movement, allows us to speak of the insect as trying to find food. Studies have also shown that when the landmarks that mark an insect's feeding site are moved, insects try to find the place where their view of the landmarks matches the view they see from their goal.

The above findings lead me to conclude that visual navigation using landmarks does indeed constitute a form of intentional agency, subject to the list of conditions laid down in section 2.1 above.

2.2(c) Cognitive maps in insects?

Menzel, Brandt, Gumbert, Komischke and Kunze (2000) claim that some insects, e.g. (honeybees) possess two spatial reference systems:

(i) an egocentric (observer-centred) system, also known as specialised route memory (SRM), which makes use of path integration and a sequence of visual landmarks; and

(ii) an allocentric (world-centred) system, which combines multiple views and movements in a common frame of reference.

The authors refer to the latter system as a general landscape memory (GLM). They suggest it is learned by bees during their reconnoitring flights around a new hive, and is only activated when the SRM cannot be used. They speculate about how the GLM is implemented at the neural level: it may simply list each landmark with its vector to the hive, or it may include multiple sequential views of landmarks, acquired during reconnaissance flights, or it may store the landmarks in a spatial memory, with a graph structure.

Giurfa and Capaldi (1999, p. 237) define a cognitive map as "a form of spatial memory, in which the geometrical relationships between defined points in space are preserved", from which it follows that "a cognitive map should allow novel short-cutting to occur".

Experiments designed to test novel short-cutting in bees produced conflicting results during the 1980s and 1990s. Recently, studies have shown that honeybees can use novel routes (Menzel, Brandt, Gumbert, Komischke and Kunze, 2000). Tests were performed near a prominent landmark (a steep, isolated hill). Foraging bees were captured at the training station and released at different sites, up to 180 degrees from their original bearings. Nevertheless, the bees generally managed to re-orient themselves towards the hive. Bees released at much greater distances, which they had never visited before, were unable to do this.

Novel short-cutting has been claimed by other researchers (e.g. Giurfa and Menzel, 2003), who summarise their findings as follows:

Furthermore, route-trained bees carrying a trans-ponder enabling harmonic radar to locate them were captured and released at a novel site, either when leaving the feeder to return to the hive, or when arriving empty at the feeder... Both groups of bees first flew their usual vector when released at a novel site, but then headed towards the hive after a phase of circling flights. Again, beacon orientation and navigation according to landscape features were excluded. Most importantly, bees sometimes also decided to fly back to the feeder first rather than directly to the hive. These and additional experiments can be explained by assuming that during their orientation flights, bees learned different locations in their surroundings and attached to the landmarks characterizing these locations local vectors pointing towards the hive...[T]he fact that bees foraging at a distant and constant feeder could decide to fly first back to the feeder rather than directly to the hive indicates a form of spatial memory in which some geometrical relationships between defined points in space are preserved, in agreement with Tolman's seminal paper...

However, alternative explanations have been proposed, and the existence of cognitive maps or even a GLM is disputed (Collett and Collett, 2002; Harrison and Schunn, 2003; Giurfa and Capaldi, 1999).

[A]ll around the nest, different landmark constellations might be linked to different flight vectors. In this way, bees might find their way home from any novel release site within this area, providing that it lies between two identifiable sites that give rise to different, established flight vectors. This type of vector map is different from a cognitive map... (Giurfa and Capaldi, 1999, p. 241).

Recently, Harrison and Schunn (2003) have tried to establish that egocentric processing is computationally cheaper than allocentric or exocentric processing, and that the cognitive map hypothesis is redundant, even for rats.

2.3 Navigational agency in octopuses

Photo of a giant Pacific octopus. Copyright Greg Dombowsky and Dive BC Publications.

What is a cephalopod?

Cephalopods are a class of the phylum Mollusca (molluscs) and are therefore related to bivalves scallops, oysters, clams, snails and slugs, tusk shells and chitons. Cephalopods include the pelagic, shelled nautiloids and the coeleoids (cuttlefish, squid and octopods, the group to which octopuses - not octopi - belong). Among molluscs, cephalopods are renowned for their large brains, while other molluscs (e.g. bivalves) lack even a head, let alone a proper brain.

Spatial learning

Cephalopods find their way around by remembering landmarks, as well as the distances they have travelled:

Cephalopods are certainly adept at navigation. Mather and a team of volunteers have mapped the travels of fist-sized O[ctopus] vulgaris as they forage off the coast of Bermuda. The animals venture from their dens on complicated trips lasting up to three hours, and return by different, more direct routes. Although O. vulgaris usually ends up no more than 9 metres from home, other species of octopus can find their dens after journeys of up to 120 metres - over a landscape that easily disorients human scuba divers (Hamilton, 1997, p. 35).

Like some insects, octopuses can navigate flexibly:

A series of disruptions of the foraging trail showed that they could make detours and suggested they were using vision to follow prominent features of the landscape of the rocky bottom (Mather and Anderson, 1998).

The ability of insects to navigate using landmarks has already been identified as indicative of cognitive mental states (see Part C), irrespective of whether they use allocentric cognitive maps.

Additionally, laboratory tests have shown that octopuses are fast learners that adapt quickly to reversals. They can rapidly learn the location of an escape burrow in an arena and retain this information for a week. When the burrow location is rotated 180 degrees, they display relearning (Langley, 2002).

Octopuses can navigate simple mazes. Lauren Hvorecny and Jessica Grudowski are currently researching learning in octopuses, to determine if they can solve a conditional discrimination maze problem (in maze configuration A, go to hole A; in maze configuration B, go to hole B). The results are still being analysed. The only invertebrate that has demonstrated this type of complex learning is the honeybee.

3. Tool Agency

Beck (1980) proposed that in order to qualify as a tool user, an animal must be able to modify, carry or manipulate an item external to itself, before using it to effect some change in the environment (Mather and Anderson, 1998). Beck's definition of a tool is commonly cited in the literature, so I shall use it in my definition of tool agency. At the same time, I would like to note that Beck's yardstick, taken by itself, cannot tell us whether the tool-using behaviour it describes is fixed or flexible, and is an insufficient criterion for intentional agency.

3.1 Definition of tool agency

Definition - Tool Agency
An animal can be described as using a tool intentionally if the following features can be identified:

NEW CONDITION: a tool - that is, an item external to the animal, which it modifies, carries or manipulates, before using it to effect some change in the environment (Beck, 1980);

(i) innate preferences or drives;

(ii) innate motor programs, which are stored in the brain, and generate the suite of the animal's motor output;

(iii) a tendency on the animal's part to engage in exploratory behaviour, by using its tools to probe its environment;

(iv) an action selection mechanism, which allows the animal to make a selection from its suite of possible motor response patterns and pick the one that is the most appropriate for the tool it is using and object it is used to get;

(v) fine-tuning behaviour: an ability to stabilise one of its motor patterns within a narrow range of values, to enable the animal to achieve its goal by using the tool;

(vi) a current goal: the acquisition of something useful or beneficial to the individual;

(vii) sensory inputs that inform the animal whether it has attained its goal with its tool, and if not, whether it is getting closer to achieving it;

(viii) associations between different tool-using motor commands and their consequences, which are stored in the animal's memory;

(ix) an internal representation (minimal map) which includes the following features:

(a) the animal's current motor output (represented as its efference copy);

(b) the animal's current goal or end-state (represented as a stored visual memory involving a tool that the animal associates with attaining its goal; and

(c) the animal's pathway to its current goal (represented as a stored memory of a sequence of movements, coupled with sensory feedback, which allows the animal to steer its tool towards its goal);

(x) the ability to store and compare internal representations of its current motor output while using the tool (i.e. its efferent copy, which represents its current "position" on its internal map) and its afferent sensory inputs;

[NOT NEEDED:
(xi) a correlation mechanism, allowing it to find a temporal coincidence between its motor behaviour and the attainment of its goal;]

(xii) self-correction, that is:

(a) an ability to rectify any deviations in motor output from the range which is appropriate for attaining the goal;

(b) abandonment of behaviour that increases, and continuation of behaviour that reduces, the animal's "distance" (or deviation) from its current goal; and

(c) an ability to form new associations and alter its internal representations (i.e. update its minimal map) in line with variations in surrounding circumstances that are relevant to the animal's attainment of its goal.

If the above conditions are all met, then we can legitimately speak of the animal as an intentional agent which believes that it will get what it wants, by doing what its internal map tells it to do.

3.2 Case study: Tool agency in cephalopods

Photo of a giant Pacific octopus. Copyright Greg Dombowsky and Dive BC Publications.

Taxonomy and comparative anatomy

Cephalopods are a class of the phylum Mollusca (molluscs) and are therefore related to bivalves scallops, oysters, clams, snails and slugs, tusk shells and chitons. Cephalopods include the pelagic, shelled nautiloids and the coeleoids (cuttlefish, squid and octopods, the group to which octopuses belong). (Authorities agree that the plural of octopus cannot be octopi, as the word is Greek, not Latin.)

The design of a mollusc's nervous system is quite different from that of a vertebrate, despite fundamental similarities at the neuronal level:

The vertebrate central nervous system comprises one main nerve cord that has swollen at one end to create a brain. Most molluscs, on the other hand, have dual nerve cords running like a set of railway tracks along the length of the body (Hamilton, 1997, p. 32).

Among the molluscs, there is an enormous degree of variability in the complexity of the nervous system. Cephalopods are renowned for their large brains, while other molluscs (e.g. bivalves) lack even a head, let alone a proper brain. Most molluscs have a relatively "simple" central nervous system, with five or six pairs of ganglia.

In the cephalopods alone among the molluscs, evolution has also constructed a brain. It has greatly expanded the forwardmost pairs of ganglia and moved them closer together to create a tightly packed mass of lobes that lies between the eyes and encircles the oesophagus (Hamilton, 1997, p. 32).

The brain-to-body weight ratios of cephalopods exceed those of other invertebrates, as well as most fish and reptiles. Additionally, their brains are anatomically complex. However, mammals and birds far outstrip cephalopods in the complexity of their brains (Anderson and Wood, 2001; Hamilton, 1997).

In contrast with molluscs such as clams and oysters, which are passive filter feeders, cephalopods live in a challenging environment, where they have to hunt down mobile prey and avoid predators. They have sophisticated sense organs, a complex rapid movement system, an ability to rapidly change colour and (in the case of cuttlefish and squid) a wide range of social signals (Broom, 2001).

What do cephalopods think with?

So far, we have assumed that cephalopods think with their brains.

Octopus arms are highly manoeuvrable. Courtesy BBC.

...[A]n octopus moves its arms simply by sending a "move" command from its brain to its arm and telling it how far to move.

The arm does the rest, controlling its own movement as it extends.

"There appears to be an underlying motor program... which does not require continuous central control," the researchers write (Noble, 2001).

I suggest that there is no inherent reason why intelligence should be tied to a brain alone. We should be open to the possibility of creatures who also think with their arms, especially when "each arm is controlled by an elaborate nervous system consisting of around 50 million neurons" (Noble, 2001).

The learning abilities and adaptive behaviour of cephalopods compare favourably with those of insects and some vertebrates. The following discussion focuses principally on the well-studied common octopus, Octopus vulgaris.

Tool agency in octopuses?

According to Beck's criterion for tool use, octopuses use both rocks and water jets from their siphons as tools to modify their environment. After selecting a place for a home, an octopus usually has to modify it extensively to render it suitable for habitation, as it is usually clogged with sand and the shape may not be appropriate. The octopus gathers up excess sand and small rocks and carries them out to the entrance of its new home. Once there, it lets go of the rocks and blasts them all away with a jet of water from its siphon. After removing sand and small rocks from its burrow, an octopus may be left with a large entrance, so it goes outside, picks up some small rocks and brings them back to the home, piling them up at the entrance (Mather and Anderson, 1998).

Octopuses also use water jets to get rid of nuisances:

After capturing crab prey, the octopus will usually kill them and hold one or several under the arm web, dissolve the cartilage holding the joints together, digest out the meat, and keep the exoskeleton bits. When it's finished, it will take the remains to the den entrance and jet the lot out into what becomes a midden. If a scavenging Serranid fish comes by to eat these remains, the octopus may jet a blast of water to remove the 'pest' from the vicinity (Mather and Anderson, 1998).

However, the behaviour described above may turn out to be a fixed pattern of behaviour, and even Mather and Anderson (1998) admit that octopuses use tools "in a very simple way". While there is no doubt that octopuses are anatomically capable of fine-tuning their motor movements, what needs to be shown here is that they actually do so while manipulating tools.

Although octopuses do not appear to fine-tune their behaviour when cleaning their dens, they aim their jets carefully at passing scavengers or human observers - rather like a water gun (Mather and Anderson, 2000). This could be an instance of fine-tuning, and hence bona fide agency.

Flexible behaviour in octopuses?

Mather and Anderson (2000) describe how octopuses will use a variety of techniques to open a clam shell, switching readily from one to another in the event of failure. Giant Pacific octopuses switch strategies to open different shellfish - smashing thin mussels, prying open clams, and drilling tougher-shelled clams. When clams were wired shut with stainless steel wire, the octopuses couldn't pull them apart, so they switched to drilling and chipping. The authors comment:

They were intelligently adapting the penetration technique to the clam species presented and the situation in which they were placed.

The above interpretation is reasonable. Unfortunately, the range of behaviours involved here is too narrow to decide whether the octopuses were acting intentionally or in a hit-and-miss fashion.

Mather and Anderson (1998) describe a recent experiment where two octopuses were able to fine-tune the force of their water jets to control the movement of some coloured toys floating in their tank. The octopuses did not manipulate the toys in a stereotypical fashion; indeed, their behaviour seemed spontaneous, even playful. This tool-using behaviour by octopuses appears to have been both flexible and fine-tuned, but follow-up studies are warranted.

Object play in octopuses?

Mather and Anderson (1998) define play as "activity having no immediate benefits and structurally including repetitive or exaggerated actions that may be out of sequence or disordered", and reported observing some octopuses playing with objects. Scigliano (2003) describes their experiment:

Anderson tested for play by presenting eight giant Pacific octopuses with floating pill bottles in varying colors and textures twice a day for five days. Six octopuses examined the bottles and lost interest, but two blew them repeatedly into their tanks' jets. One propelled a bottle at an angle so it circled the tank; the other shot it so it rebounded quickly and on three occasions shot it back at least 20 times, as if it were bouncing a ball.

However, Boal questions the authors' interpretation, and suggests that the behaviour may reflect boredom (like a cat pacing), rather than creativity. More recently, another researcher, Ulrike Griebel, offered common octopuses a variety of objects, from Lego assemblies to floating bottles on strings. Some octopuses took toys into their nests and toted them along while fetching food. Griebel suggests that this "might be an early stage of object play" (Scigliano, 2003).

Mather and Anderson (2000) argue that "[p]lay involves the detachment of actions from their primary context, and such flexibility is both a basis and a sign of intelligence, whether it be shown in a person or a fish or an octopus." The key insight here is that the player self-selects a new goal and performs the actions to achieve this goal rather than the natural end of the behaviour. Genuine play is therefore cognitive. However, as we shall see in chapter 4, "cognitive" does not necessarily mean "phenomenologically conscious".

At present, then, there are promising leads, but the evidence is insufficient to determine whether tool use in octopuses is accompanied by cognitive mental states. I would suggest that future research should focus on ascertaining whether octopuses are capable of satisfying conditions (iii), (v) and (ix) - i.e. whether they use tools to explore their surroundings, fine-tune their movements when manipulating tools, and possess an internal minimal map encoding their current motor output, current goal and pathway for getting there.

4 Social agency

4.1 Definition of Social Agency

Definition - "agency in a social context"
An animal can be described as displaying agency in a social context if the following features can be identified:

NEW CONDITION: a role model or knowledgeable individual;

NEW CONDITION: sensory capacities: the ability to discriminate between individual members of its own species (conspecifics), as well as between members and non-members of its group;

NEW CONDITION: memory capacity: the ability to keep track of the status of individuals within one's group, and remember one's past interactions with them (book-keeping);

NEW CONDITIONS: learning: the ability to learn from observing the behaviour of other individuals (observational learning) and to acquire new knowledge that is specific to one's group (traditions);

NEW CONDITION: representation: the ability to represent another individual in its group as a useful, reliable role model, to be followed in the pursuit of important objectives such as food;

(i) innate preferences or drives;

[NOT NEEDED? (ii) innate motor programs, which are stored in the brain, and generate the suite of the animal's motor output;]

(iii) a tendency on the animal's part to engage in exploratory behaviour;

(iv) an action selection mechanism, which allows the animal to make a selection from its suite of possible motor response patterns and pick the one that is the most appropriate to its current social setting;

(v) fine-tuning (controlled, modulated activity): the ability to model its behaviour on that of a knowledgeable individual (the role model), and to adjust its social behaviour to take account of differences between the individuals in its group, as well as changes in a given individual's behaviour;

(vi) a current goal for the animal, which is (at least qualitatively) the same as the goal which its role model is currently pursuing or has pursued in the past;

(vii) sensory inputs that inform the animal whether it has attained its current goal, and if not, whether it is getting closer to achieving it;

(viii) associations between stored memories of the different individuals in the animal's group and the (good or bad) consequences of following their example, as well as direct associations between different motor commands and their consequences, which are stored in the animal's memory;

(ix) an internal representation (minimal map) which includes the following features:

(a) the animal's current state, which includes both its current spatial relation to its role model, and its current motor output (represented as its efference copy);

(b) the animal's current goal (represented as a stored memory of a sensory stimulus which the animal associates with the attainment of that goal); and

(c) the animal's pathway to its current goal (represented as a stored memory of the individual which can reliably lead the animal to its goal - i.e. the role model);

(x) the ability to store and compare internal representations of its current motor output (i.e. its efferent copy, which represents its current "position" on its internal map) and its afferent sensory inputs;

[NOT NEEDED:
(xi) a correlation mechanism, allowing it to find a temporal coincidence between its motor behaviour and the attainment of its goal;]

(xii) self-correction, that is:

(a) an ability to rectify any deviations in its social behaviour from that which is appropriate for attaining its current goal;

(b) abandonment of social behaviour that proves to be unproductive (e.g. when the animal's expectations of another individual are disappointed), and continuation of behaviour that helps the animal obtain its current goal; and

(c) an ability to form new associations and alter its internal representations (i.e. update its minimal map) in line with variations in surrounding circumstances that are relevant to the animal's attainment of its goal.

If the above conditions are all met, then we can legitimately speak of the animal as an intentional agent which believes that it will get what it wants, by doing what its internal map tells it to do.

4.2 Case study: Social agency in fish

According to Brown and Laland, "research over the last 50 years has demonstrated that social learning is common amongst fish, birds and mammals, and should now be regarded as a regular feature of vertebrate life... Documented cases of social learning in fish are now commonplace" (2003, p. 281). In particular, "social learning plays a role in fish (i) anti-predator behaviour; (ii) migration and orientation; (iii) foraging; and (iv) mate choice, and ... it is facilitated by eavesdropping" (2003, p. 281).

Brown and Laland (2003) explain the rationale for social learning in fishes:

In making decisions, such as how to find food and mates or avoid predators, many animals utilise information that is produced by others. Such individuals are referred to as 'eavesdroppers' in the signal-receiver literature (McGregor 1993) and 'observers' in the social-learning literature (Heyes and Galef 1996). Socially transmitted information may simply be a by-product of the demonstrating individual's behaviour or a signal targeted towards a particular audience (2003, p. 280).

Bshary, Wickler and Fricke (2002) argue that most kinds of social behaviour interpreted as cognitive in primates can be found in certain fish as well. For instance, features such as individual recognition, acquisition of new behaviour patterns by observational learning, transmission of group traditions, co-operative hunting, tactical deception (cheating), tit-for-tat punishment strategies, reconciliation, altruism and social prestige, formerly thought to be unique to primates or at least mammals, can all be found in fish societies. Specifically, research cited by Bshary, Wickler and Fricke (2002) shows that each kind of behaviour described in my model of social agency above can be found in fish. My only qualification is that (as the authors point out), the evidence is assembled together from different species of fish, mainly because there have been relatively few studies of fish cognition to date. Further research is needed to determine whether any particular species satisfies all of the criteria. Still, the current evidence for a rudimentary social intelligence in fish looks strong.

In my model of social agency in fish, I outlined a proposed set of sufficient criteria for agency in a social context, looked at the rationale for social learning, and noted its widespread occurrence across vertebrate species. Evidence that at least some species of fish satisfy my criteria for social agency is provided below. To date, I have not been able to establish whether any particular species meets all of the criteria, although the evidence for cleaner fish, sticklebacks and guppies looks impressive.

Behaviour modelled on that of a knowledgeable individual

Individuals (especially juveniles) learn to model their behaviour on that of experienced adults, mainly by accompanying them and observing how they behave (Bshary, Wickler and Fricke, 2002).

Sensory discrimination between individuals and/or categories of individuals

There is abundant evidence in the literature of individual recognition in fish:

Individual recognition based primarily on optical cues ... has been demonstrated experimentally in a variety of species... There is even evidence that in damselfish, individuals can recognise one another on purely acoustical cues... In summary, individual recognition can safely be assumed to be widespread across fish families (Bshary, Wickler and Fricke, 2002).

In addition to individual recognition, cleaning symbiosis provides an example of a case where the ability to categorise individuals on the basis of their observed characteristics is especially useful:

In cleaning symbiosis, so-called client fish trade the removal of parasites and dead or infected tissue against an easy meal for so-called cleaner fish... Cleaning symbiosis is particularly promising for comparative studies as cleaner fish are found in many different fish families and can differ markedly in the degree to which they depend on interactions with clients for their diet... Full-time cleaners like the cleaner wrasse (Labroides dimidiatus) may have about 2,300 interactions per day with clients belonging to over 100 different species... There is strong evidence that cleaners can categorise their 100-or-so client species into resident species that have access to their local cleaner only, due to their small territory or home range, and other species that have home ranges that cover several cleaning stations. As predicted by biological market theory (Noe et al. 1991), clients with choice options between cleaners almost invariably have priority of access over clients without choice at cleaning stations (Bshary, Wickler and Fricke, 2002).

Memory for individuals and their track record

The ability to remember individuals over long periods of time is of fundamental importance for social learning. Bshary, Wickler and Fricke (2002) cite evidence that an anemonefish can recognise an individual that it has not seen for 30 days. (So much for the myth that fish have only a 3-second memory!)

According to Bshary, Wickler and Fricke (2002), some fish can also monitor changes in the status of individuals and track relationships within their groups.

Bshary, Wickler and Fricke (2002) describe experiments showing that some fish species are capable of engaging in book-keeping (remembering their partners' behaviour during past interactions) with several partners at once:

The most famous example of co-operation in fish is probably the inspection of nearby predators by one or several fish that leave the relative safety of their school to do so (Pitcher et al. 1986). During inspection, pairs of sticklebacks, Gasterosteus aculeatus, and guppies, Poecilia reticulata, among others, approach the predator in alternating moves. A series of experiments led to the conclusion that these fish solve a so-called "prisoner's dilemma" (Luce and Raiffa 1957). In a prisoner's dilemma, two players have the option of either co-operating with or cheating their partner. Cheating the partner yields a higher benefit than co-operation irrespective of what the partner does, but if both partners co-operate then they receive a higher benefit than if both cheat, hence the dilemma. Milinski (1987) and Dugatkin (1988) proposed that fish solve the prisoner's dilemma by playing a "tit-for-tat" strategy, which states that a player starts co-operatively and does in all further rounds what the partner did in the previous round (Axelrod and Hamilton 1981). This interpretation is not yet entirely resolved (see review in Dugatkin 1997) but discussions about the interpretation led to a few experiments with very interesting additional results. Milinski et al. (1990a) could show that individual sticklebacks prefer specific partners to others, which implies that school members recognise each other. In addition, partners build up trust in each other during repeated inspections, that is, they hesitate less in approaching a predator when accompanied by a partner that co-operated in the past (Milinski et al. 1990b). Similar results have been found in guppies (see review in Dugatkin 1997). These data imply that these fish species are capable of book-keeping (remembering their partners' behaviour during past interactions) with several partners simultaneously (2002).

Observational learning of new practices

Juvenile fish learn what to eat by observing conspecifics:

There is some evidence that young fish learn what to eat by observing adults. Fish definitely learn horizontally from conspecifics what to eat under lab conditions. Templeton ... found that juvenile rock bass ... that saw a trained conspecific eating a novel food item would readily consume that food later when, alone, they were tested for the first time. Without prior observations, these juveniles did not attack the prey... (Bshary, Wickler and Fricke, 2002).

Hatchery-reared Atlantic salmon acquire new kinds of feeding behaviour and learn to target new kinds of prey, simply by observing knowledgeable conspecifics. Salmon raised in hatcheries tend to prefer taking prey from the surface, because of long-term conditioning in the hatchery environment. This can cause high mortality rates from starvation when they are released into the wild, because their choice of prey near the surface is restricted and energetically costly. However, after six days of watching "demonstrators" through a clear perspex partition, naive salmon changed their feeding habits and were able to feed from the bottom (Brown, Markula and Laland, 2003).

Social enhancement of foraging has been reported in species as different as salmon, rock bass, Alaska pollack and brown trout (Brown and Laland, 2003).

Fish can also learn novel techniques for obtaining food from observation of knowledgeable conspecifics. Juvenile European sea bass learned to press a lever to get food, simply by watching other fish that had been previously trained to do this (Brown and Laland, 2003).

Learning (group traditions)

Schools of fish have their own "traditions" relating to their choice of sites for resting sites, migration routes and food sources, and this knowledge is transmitted through social learning (Bshary, Wickler and Fricke, 2002). For instance, juvenile French grunts learn the migration route from their resting grounds to feeding sites by following older individuals, and bluehead wrasse have preferred mating sites that stay the same over many generations (Brown and Laland 2003).

Bshary, Wickler and Fricke describe the mechanism by which traditions are perpetuated in guppies:

Laland and Williams ... conducted laboratory experiments and showed experimentally that guppies learn the way to hidden food sourced from knowledgeable conspecifics. The conspecifics had been trained to use only one of two ways to the food source. Naive fish were added and learned the way to the food source by schooling with the others. Members of the original school could be replaced successively and the school still preferentially took the originally learned way to the food source. The fish thus built up a tradition. Using principally the same experimental set up, Laland and Williams ... went one step further and showed that even maladaptive behaviour can spread through a population due to social learning. In their study, a longer and therefore more costly way to a foraging site was still preferred over a short way 3 days after all original trainers had been removed" (Bshary, Wickler and Fricke, 2002).

Innate goals

Brown and Laland (2003) mention four general categories of goals, in relation to which social learning is known to take place amongst fish: predator avoidance; migration and orientation; foraging for food; and mate choice. There is an ever-growing body of evidence that juvenile fish engage in extensive social learning of skills relating to all of these goals (Laland, Brown and Krause, 2003).

Fine-tuning (controlled, modulated activity):

Individuals carefully tailor their own social behaviour towards an individual, in accordance with their observations of that individual's past interactions with other individuals.

Male Siamese fighting fish ... monitor aggressive interactions between neighbouring conspecifics and use the information on relative fighting ability in subsequent aggressive interactions with the males they have observed...(Brown and Laland, 2003, p. 285).

Individuals also adjust their behaviour towards a specific individual on the basis of their own previous interactions with that individual - a practice known as book-keeping. Cleaner fish engage in book-keeping: they provide better than average service to dissatisfied clients that "punished" (aggressively chased) them during their last interaction. As Bshary, Wickler and Fricke (2002) point out, punishment can only work if there is individual recognition. This means that cleaner fish must be able to keep track of the behaviour of each their clients (up to 100 individuals!), and modulate their behaviour towards each of them.

Cleaner fish also provide tactile stimulation to predatory clients, possibly as a form of pre-conflict management, or towards clients it has cheated in the past (Bshary, Wickler and Fricke, 2002).

Additionally, cleaner fish behave much more attentively (or "altruistically") towards their clients if they are being watched by bystanders who have the option of switching to another cleaning station. The reason is that an observer will copy the behaviour of the previous client, and either invite for inspection if it witnessed a positive interaction, or flee the approaching cleaner if it saw the last client run away as well. The true rationale for cleaner "altruism" is a selfish one: the opportunity to recruit a new customer and get access to more food (Bshary, Wickler and Fricke, 2002).

Internal representations

The mechanisms by which fish represent their social interactions with other individuals are not known, but fish are certainly able to form internal representations of the status and fighting ability of other individuals in their group, as well as the reliability of former partners (Bshary, Wickler and Fricke, 2002). Presumably, when copying the goal-oriented behaviour of a knowledgeable individual, they must be able to represent the activity of following the role model's example as a means of attaining its own ends, which are (qualitatively) the same as its own. Alternatively, in simpler cases (e.g finding hidden food by following a knowledgeable individual), the observer may simply represent the model itself as a kind of "moving signpost" pointing to its goal (i.e. the model itself is viewed as a means to the individual's end).

Self-correction

Fish are certainly capable of altering their social behaviour when their expectations of another individual are disappointed. As we saw above, Bshary, Wickler and Fricke (2002) cited evidence that sticklebacks and guppies adopt a tit-for-tat strategy towards their partners: a partner that fails to co-operate is punished the next time round. Cleaner fish who cheat their clients by removing extra food (healthy tissue) as well as dead or infected tissue, are "punished" (chased aggressively).

Other evidence for intentional agency in fish

According to Bshary et al. (2002):

there is an array of behaviours found in a variety of fish families (categorisation, cheating, punishment, manipulation of individuals and altruism) which are usually thought of as unique to primates;

there are instances of interspecific cooperative hunting between giant moray eels and red sea coral groupers;

co-operative hunting between conspecific predators is widespread in fish, and different individuals play different roles;

some fish appear to be able to use cognitive maps of their environment. For instance, inter-tidal gobies acquire an effective memory of the topography of its home pool as well as that of surrounding pools, because at low tide, it often has to jump into these pools without being able to see where it is going. Other fish appear to use landmarks for homing;

some fish use advanced foraging techniques - e.g. removing obstacles to reach hidden prey, and using their spatial intelligence to gain access to prey;

a few fish are capable of tool-using behaviour in the strict sense of the word (Beck, 1980), where an animal directly handles an object in order to obtain a goal. South American cichlids are a case in point;

some fish also build complex nests and bowers (Laland, Brown and Krause, 2003).

Finally, contrary to claims by Varner (1998), fish are indeed capable of progressive adjustments in multiple reversal trials - as long as olfactory stimuli are used (Mackintosh and Cauty, 1971, cited by Wakelin, 2003). Earlier, we examined arguments that creatures which show improvements in serial reversal learning were capable of meta-learning, insofar as they had to develop primitive hypotheses about changes in their surroundings. We tentatively concluded that this was the most reasonable interpretation of the experimental evidence, and that the behaviour should be described using an agent-centred intentional stance.

4.3 Case study: Evaluation of the evidence for social agency in Octopuses

Observational learning in octopuses?

Octopus arms are highly manoeuvrable. Courtesy BBC.

Fiorito and Scotto (1992) reported that an octopus in a research laboratory in Naples learned to choose a red ball instead of a white one, simply by watching another octopus. (Actually, octopuses prefer red over white, but the opposite preference has also been induced in recent experiments.) The discovery of observational learning, if confirmed, would be remarkable, as octopuses are short-lived, solitary creatures that usually meet only to copulate, and as even some mammals are incapable of this learning feat (Hamilton, 1997). However, other researchers, including Jean Boal, have tried without success to replicate the results (Mather, personal email, 8 September 2003). Commenting on the original experiment, Woods (2003) writes:

A critique by Biederman and Davey of the Fiorito and Scotto experiment can be found in Science vol. 259 (March 12, 1993). The critique questions: if the observational octopuses attacked the ball more often since it was a familiar item (i.e. octopuses are hesitant to attack novel stimuli), [and] if observational learning or rapid imitation occurred (what was the role of the stimuli and the role of the demonstrator octopus - why were these not controlled for?). I should mention that Fiorito defends the experiment in the same issue of Science. The bottom line, at least in my mind, is that the Fiorito and Scotto experiment failed to prove observational learning since other factors were not controlled for. I certainly would not rule out the possibility of observational learning in cephalopods - after all they are the most advanced invertebrates. On the other hand, octopuses are not very social so there may not be much of a chance for them to evolve the ability to learn by observing other octopuses.

On methodological grounds alone, it would be imprudent to ascribe observational learning to cephalopods. There is another reasons to question the cognitive interpretation of the octopuses' behaviour: learning can also be triggered by events that convey no technical (means-end) information, suggesting that the skills are latent within the octopus and not learned.

Late last year at Woods Hole, Boal, Hanlon and graduate student Kim Wittenberg allowed animals to observe trained cuttlefish attack and eat a crab, and then compared their performance in the same situation with a naive animal. The observers did learn more quickly how to hunt down a crab. But they also hunted better if they had previously seen only a crab without a predation event, or even if they had simply smelt that a crab was kept hidden behind a partition. 'If smelling a crab means you perform better than if you hadn't smelled one before, and watching a predation event is no better than simply smelling a crab,' says Boal, 'then we're talking [about] some kind of releaser of an innate behavior' (Hamilton, 1997, p. 35).

5. A special case of combined agency in the jumping spider Portia