Evidence In The Classroom

Teachers are often bombarded with a wealth of claims about classroom teaching techniques, but it can be difficult to figure out whether these are actually effective or not, and whether there is any research evidence behind them.

When it comes to claims about neuroscience­-based teaching techniques, unfortunately there isn’t much convincing evidence out there at the moment. There is a huge disconnect between what neuroscientists know about the brain, and how that knowledge might actually be applied in a practical, day-to-day setting. That doesn’t mean that neuroscience will never have a place in the classroom – it just means that we have to be careful about any firm claims that we come across.

How to assess claims that you come across

At some point in your career, you will be faced with a claim that some sort of neuroscience-based method or programme will revolutionise the way you teach, or the way your students will learn. When that happens, here are some points to think about.

Be aware of your own biases

This is perhaps the most important thing to bear in mind. Just because something sounds right doesn’t mean that it is right. We are more likely to be critical of things we disagree with, and uncritical of things that fit with our own worldview. Particularly when it comes to the science of learning, we need to make sure that we remain objective when faced with a claim that, at face value, seems to be sensible and fit in with our own experiences – we need actual evidence showing that it works. Regardless of whether you think the claim makes sense, or is a good idea, or fits with what you already know, try to remain objective, and ask questions 1 and 2 below.

1. Who is making the claim?

Are they a practicing research scientist, entrepreneur, or someone with a passion or interest in teaching? Background motivations behind why the person is making the claim can be important.

  • Is the person a researcher with a track record of expertise in the area that they are talking about? If not, then you need to ask what they are basing their claim on. Most researchers will have either a university-based or personal website that contains a list of their academic publications. It’s worth having a quick look at these to get a general idea of their area of research expertise, and whether it fits in with the claims they are making.
  • Do they have a financial conflict of interest?  Often, if someone makes a claim about a new teaching technique and has a lesson package to sell based on it, they may be biased towards providing only a positive view of it. This isn’t always an easy thing to figure out, but again personal websites can be useful here. Check to see whether the person making a claim has any vested interests in companies, or makes money through giving talks or consultancy. Often these can be signs that they may be introducing a bias.

2. Is the claim based on peer-reviewed, published research?

If the answer to this is ‘no’, then you should ask why this is the case, and whether it might be published in an academic journal in the future. If it isn’t going to be, then you should be very wary of the claim. While peer-reviewed research is by no means perfect, it does at least provide a series of checks that help us to determine whether a claim is objective or robust. But even if the answer is ‘yes’, we should ask further questions:

  • Has the research been replicated? If so, who by? Repeating a study to determine whether we can get the same results using slightly different groups of participants or research procedures is a foundation of good science. Replicated research – ideally, by independent groups of scientists and not the original person making the claim – provides more convincing evidence that the research is valid. If the study hasn’t been replicated, this doesn’t invalidate the results – it just means that we haven’t yet got enough evidence that the results are correct. The quickest way to check is to ask the person making the claim – if the work is genuine, then they should be more than happy to provide evidence that their results are reliable. Alternatively, you can ask a scientist who was independent to the study – you will have met some through the Learning Zone, and there are lots on social media who are always happy to help.
  • How big was the sample of participants? Did the research test a sample of 10 people, or more than 100? Larger samples are preferable, because they are less susceptible to random errors.
  • What was the new technique compared with in the study? If a study claims that a new teaching technique is effective, it’s not enough to simply compare it against more traditional forms of teaching. The reason for this is that you are introducing something novel into the classroom, and it might be the simple fact that you’re doing something different with your students that makes it look as though the new technique is effective. In this situation, we can’t be sure that we’re not seeing something akin to the placebo effect happening. Proper control conditions ensure that whatever mechanism that makes the technique effective is properly assessed.
  • Are the data open and freely available? There is a drive in science to make data, research materials and analysis software freely available to anyone who wishes to see or use them. If they are not, then other researchers can’t always be sure that mistakes haven’t been made in data analysis, or, in extreme cases, that the data haven’t been tampered with to produce a desired result. Therefore research with freely available data should be more trusted.

Written by Dr Pete Etchells, Reader in Psychology and Science Communication at Bath Spa University, UK.

How can I run my own study at school?

Dr Richard Churches from the Education Development Trust helps teachers to run their own randomised controlled trials (RCTs) at school.

You can read an introduction to teacher-led RCTs, listen to a podcast where Dr Churches talks about RCTs, or buy the book. For a more in-depth look at different experimental methods, take a look at this booklet from Dr Churches.

Look out for updates – over the next few months the successful research protocols will be hosted online, and the Education Development Trust will be looking for teachers who want to replicate the studies and submit their findings.

The Education Endowment Foundation have an evaluation guide which similarly aims to introduce the key principles of conducting small-scale evaluations in school.

The Centre for Educational Neuroscience have a what works guide to the things to keep in mind when evaluating new school practices.