When our evolutionary ancestors first crawled onto land, their brains only half-filled their skulls

Most of us would recognize a human brain, but what about the brain of a frog or fish? Given the vast diversity of life on Earth, there are some weird and wonderful brains out there.

Despite this, the basic plan of most vertebrate (animals with a backbone) brains is similar, consisting of three major regions: the fore-, mid- and hindbrain. But the variation in size and shape between these regions (and others) is what makes studying vertebrates so fascinating.

Research published today by my colleagues and me in Frontiers in Ecology and Evolution indicates some of our earliest ancestors—which were likely still taking their first steps on land—had brains that only filled about half the space in their skulls.

The hurdles of studying ancient brains

Growing and maintaining brain tissue is energetically expensive for animals. The relative size of different regions of the brain is thought to be guided by a concept known as "the principle of proper mass."

This states the more important a sense or brain region is to an animal, the more likely it is that region will be enlarged compared to others. After all, it's pointless to spend lots of energy growing a visual processing center if you're a blind, cave-dwelling animal.

While studying the brains of living beasts is straightforward, this is much more complicated for extinct animals. When organisms fossilize, their soft tissues (including the muscle and brain) tend to decompose quickly, leaving only the hard parts of the skeleton to be preserved.

This means experts have to study the internal parts of the skull (or the "braincase") and the void within it, which is called an "endocast". Palaeoneurology is the study of endocasts and fossil brains. This field was founded about a century ago by palaeontologist Tilly Edinger.

In palaeoneurology's early days, scientists had to rely on lucky discoveries of skulls cracked in half to reveal the inside, or they had to destroy specimens to create an endocast. These days, researchers can use advanced scanning methods to create endocasts digitally, without damaging the specimen.

The tremendous tetrapod transition

Today, half of all vertebrate life can be classified as tetrapods—backboned animals with four limbs bearing digits (fingers and toes). This includes humans, frogs, crocodiles, parrots and kangaroos, among many, many others.

The animals responsible for giving rise to this huge array are known as "stem tetrapods." These brave beasts first ventured out of the water and onto land more than 360 million years ago.

For fish to transition into land-dwelling tetrapods required many changes to their body and brain. Fins made way for limbs, and gills gave way to lungs that could breathe air. What's more, the terrestrial realm would have presented an entirely new sensory environment to navigate.

How did the brains of stem tetrapods change in response to this major ecological shift?

We have some clues gleaned from skeleton parts, such as enlarged eye sockets that coincide with the transition from fish to tetrapods on land. One study has suggested a change in eye position and size would have led to a million-fold increase in visual acuity in early tetrapods, compared with their fishy relatives.

If so, we would expect such changes to be reflected in the visual processing regions of the brain. But how can we test this?

Interpreting ancient endocasts

One of the trickiest things about interpreting endocasts is that not all brains fill the endocast space fully. While most birds and mammals have brains that closely match the shape of the endocast, other animals such as fish, amphibians and reptiles typically don't.

In fact, the coelacanth fish is famous for having an extremely tiny brain that takes up just 1% of its endocast.

I've been studying the spatial relationship between the brain and endocast in animals that fall into a category called the stem tetrapod "extant phylogenetic bracket". These are the closest living relatives of the extinct stem tetrapods in the evolutionary tree.

This category includes the coelacanth fish, lungfishes and amphibians such as frogs, salamanders and caecilians.

Using a scanning method referred to as diceCT, we scanned the heads of these animals and measured the brains within to compare the brain's size and shape with that of the endocast.

A series of papers published by myself and my colleagues have shown lungfish, unlike the coelacanth, have brains that fill a far greater portion of their endocast (starting from about 40% and possibly up to 80%).

Within amphibians, salamander brains looked quite similar to lungfish brains and filled their endocasts in a similar way.

In our most recent paper, however, we investigated frogs and caecilians and found they have brains that are generally more tightly fitted inside their endocasts.

Reconstructing old brains

This knowledge helps us more accurately estimate the likely brain sizes of extinct stem tetrapods. Also, considering the ecology and habitats of the living relatives helps us further narrow down these values.

For instance, frogs and caecilians are highly specialized amphibians and are therefore unlikely to accurately represent the biology of the first tetrapods. Similarly, the coelacanth is a deep-water marine fish, whereas the earliest tetrapods lived in shallow and marginal environments.

This leaves us with salamanders and lungfish as the best living examples of what the neural biology of the first tetrapods may have looked like. With this in mind, we can reasonably assume the brains of our early ancestors that took those first steps onto land filled about 40-50% of their endocasts—with the rest likely full of fatty tissue or fluid.

Combining this with spectacular 3D-preserved fossils, we can begin to reconstruct the brains of these animals from millions of years ago.

Moreover, understanding how their brains changed during the vital juncture from water to land will help us understand a giant step in our own neural evolutionary history.