Meta-learning in insects: serial reversal learning

Although learning in the broadest sense (i.e. the acquisition of new skills) may not necessarily require mental states, it has been argued that meta-learning ("learning to learn") is a critical capacity that presupposes an ability to evaluate and correct one's actions. A creature with this ability would qualify as what Dennett (1997, p. 112ff.) calls a "Popperian creature". Such a creature is one level above a Skinnerian creature, which is capable of learning from its trial-and-error mistakes and successes, and can associate information about one kind of event with information about another kind. A Skinnerian creature may stumble upon a "smart move", but it cannot predict what works and what does not. Its first move may be a fatal one, if it is unlucky.

A "Popperian creature" can avoid such an outcome, because it can foresee the consequences of its actions in the "inner environment" of its imagination, which lets the creature manipulate information in its memory, about its external environment. In this inner environment, try-outs or simulations can be executed without harming the animal, allowing it to select the best course of action and make a smart first move in its external ("real") environment. The advantage of foresight is that it "permits our hypotheses to die in our stead", as Popper put it (Dennett, 1997, p. 116). The creature can make a smart first move, because it can think about smart moves.

Varner (1998), following Bitterman (1965), has suggested an experimental way to identify meta-learning in animals and resolve the question of whether they have mental states. He has argued that if they are genuinely learning, (and not merely mechanically associating), they should be forming hypotheses about the changes in their environment. He has proposed that reversal tests offer a good way to test animals' abilities. Multiple reversal tests involve repeatedly reversing the reward pattern in simple learning experiments. For instance, a rat is first presented with two levers and rewarded for pressing the left lever instead of the right. When the rat has learned to press the left lever all the time, the reward pattern is reversed. Once the rat has learned the new reward pattern, it is reversed again, and so on. Varner suggests that if an animal shows no improvements in the time it takes to adjust to subsequent reversals, that suggests an inflexible, non-cognitive mechanism is governing its behaviour. By contrast, Bitterman predicted that an animal that can form hypotheses should take longer to learn the new pattern the first time it is reversed, but should adjust more and more rapidly to subsequent reversals, as it learns to quickly revise its expectations.

Varner's proposal invites two questions. First, is rapid reversal learning a sign of intelligence? Second, does progressive improvement in multiple reversal tests indicate the presence of mental states?

The ability to adapt rapidly to changes sounds like a mind-like feature. However, the consensus from animal behaviourists is that it need not be so. According to Ben-Shahar (personal email communication, 19 August 2003), the rapid reversal learning of honey bees surpasses even that of pigeons and rats. However, Ben-Shahar cautions against the use of reversal learning per se as a measure of intelligence in animals, as the rapid reversal learning appears to be an adaptive trait for some animals, and adaptive behaviour is not necessarily intelligent:

I'm not convinced that reversal learning is necessarily directly related to intelligence. It is possible that for some species, reversal is highly adaptive, and hence the good performance. In bees one could speculate that reversal is very important to an animal that forages on unstable resources. In bees and other social species this is even more critical since they use communal foraging strategies. Bees will follow other bees to resources previously identified. If these have dried out the new forager has to look for new resources fast or she will come back empty - a big waste of time and money (personal email communication, 19 August 2003).

We can formulate the following negative conclusion:

S.21 The capacity for rapid reversal learning in an animal does not, by itself, warrant the ascription of mental states to it.

The second and more interesting question is whether the existence of progressive adjustment in multiple reversal learning trials indicates intelligence.

The ability to improve in multiple reversal learning trials is readily explained by the hypothesis that the animal is forming a hypothesis about changes in its environment. I have not been able to find a non-cognitive explanation as to why such improvement might occur. Certainly, the fact that the cognitive explanation makes a highly specific prediction (that the animal should take longer to learn the new pattern the first time it is reversed), which has been experimentally confirmed, tends to bear out a mentalistic interpretation. It should be borne in mind, however, that even if the behaviour cannot be accounted for in terms of associative learning, that does not necessarily make it cognitive.

Even if progressive adjustment shows that an animal has mental states, that does not necessarily make it a Popperian creature. The ability to formulate primitive hypotheses need not imply the ability to foresee the consequences of one's actions in the "inner environment" of one's imagination.

S.22 Progressive adjustments in serial reversal tests constitute good prima facie evidence that an animal is trying to adjust to sudden changes in its environment, by rapidly revising its expectations.

Basing his arguments on research by Morton Bitterman (1965), Varner has claimed (1998, p. 32) that progressive adjustment in multiple reversal learning trials is found only in reptiles, birds and mammals. Since then, it has become apparent that fish (Wakelin, 2003) and honeybees (Komischke, Giurfa, Lachnit and Malun, 2002), are also capable of this kind of learning. Komischke, Giurfa, Lachnit and Malun (2002) compared the responses of bees that had experienced reversals with those of bees that had not experienced such reversals when both were confronted with a new reversal situation. They found that bees that had experienced three previous reversals were better in solving the final reversal task than bees with no previous reversal experience. They also showed that one reversal learning trial was enough for bees to perform better in the final reversal task.

The evidence to date from serial reversal learning suggests that honeybees, at least, are capable of learning to learn. This ability may turn out to be widespread among insects, but very little research has been done with most groups of insects. Brembs claims that serial reversal learning in insects is not confined to honeybees:

Drosophila can reversal learn and if the pattern-heat contingency is reversed, learning is faster (personal email communication, 11 August 2003).

However, neither Brembs nor Drosophila researcher Josh Dubnau was able to supply a reference to serial reversal learning by Drosophila melanogaster in the published literature. Dubnau admitted that "not many people have looked carefully at reversal learning in flies" (personal email communication, 14 August 2003).

The evidence from serial reversal learning is thus of limited value. At most, it suggests that honeybees are capable of meta-learning, while saying nothing about other insects.