DevLearn Digest 9: Why you don’t always need a baseline
Welcome to our new-look DevLearn Digest, now hosted on Substack.
We have an announcement before we get into the surprisingly controversial topic of baselines.
Try out the DevLearn Chatbot now
You may have heard the buzz – including from our last DevLearn Digest – around AI Chatbots. Now you can test out your very own AI expert in MSD and MEL with the DevLearn Chatbot! It can answer technical questions, design results frameworks, write proposals and copy-edit text. Try it out by clicking here.
Please note that chatbots are still an experimental technology – the DevLearn Chatbot is not always accurate or comprehensive, and so its answers should be carefully checked before putting any reliance on them. With that caution in mind, please give it a try and let us know what you find helpful.
Baselines – myth and reality
Let’s start this section by asking a question: are the three following statements about baselines true or false?
- A baseline study is an essential component of monitoring and evaluation in any programme.
- It shows the situation before the project starts, which is required to track what has changed.
- Consequently, baseline studies need to be conducted at the beginning of the programme.
These are all common wisdom about baseline studies – in fact, I copied them from various online guidance papers. As you might guess from the title of the newsletter, however, I disagree. While baselines are a useful tool for monitoring, evaluation and learning (MEL), they are not essential, they are not required to track changes, and they do not need to be conducted at the beginning of the project.
Are baseline studies essential?
As a purely empirical observation, the majority of baseline studies I have seen – representing millions of pounds of research – have been wasted. In one of my first-ever assignments, as a nervous junior consultant in Nigeria, I was proudly told that the programme had conducted an extensive baseline study. When I asked to see it, I was (less proudly) told that they had lost the data. While this is an extreme case, the realities of changing programme modalities, staff turnover, and poor data management mean that baseline studies are often lost or forgotten by the time of the end of the programme.
Of course, the fact that baseline studies are often badly managed does not mean that they are not needed. So let’s move on to ask…
Are baseline studies required to track changes?
There are four common situations where baseline studies are not required.
The first is where the baseline starts off as zero. Market development programmes often try to introduce new ideas and business models into an area. For example, it might be introducing a new crop or a new type of seed. If you want to know the baseline for key indicators, such as the number of people using improved seed, or income from starting a new business, there’s no need to measure it – you know that it’s zero.
The second situation is where the baseline is so easy to remember that you can use recall. This means asking respondents to remember baseline data when you conduct your end-line survey, rather than running a separate study. This is particularly relevant for projects that aim to create jobs. Most people can remember their work history, even after several years have passed. Consequently, rather than measuring employment status at the beginning of a project, it’s often fine to leave measurement till the end.
The third is when the context is evolving rapidly. The point of a baseline is to provide a point of comparison – but these comparisons can be quite misleading in practice. For example, imagine that you did a baseline for a one-year livelihoods project in February 2020 – your baseline would tell you very little about the project, but a lot about the impact of Covid-19. Doing a baseline in countries with fast-moving economic challenges (such as Ghana or Sri Lanka) or rapidly changing conflict dynamics (such as South Sudan or Northern Nigeria) might simply not produce information that is relevant to your project.
The fourth is when the programme itself is changing. Particularly common with market systems development programmes, initial baselines may be based on a set of interventions that are never actually implemented, or that turn out to be unsuccessful. This makes the baseline data redundant.
Should baselines be done at the beginning of the project?
I have seen projects with great pressure, from their donors or senior management, to get the baseline done as quickly as possible before programme implementation can start.
The problem with this approach is that you can’t do robust baselines until you know exactly who might benefit from an intervention. You can survey a random sample of people in a target area – but they are likely to be significantly different to people who actually participate in your intervention, making comparisons impossible. So you need to wait until the potential beneficiaries have been identified, which is likely not going to be until the project is set up and running, and activities have started.
When working with the private sector, as in a market systems development project, the challenge of timing is even more acute. Even once the project is running, it can take companies months to identify their suppliers and customers. For many consumer-facing firms, there is no way to identify the ‘beneficiary’ ahead of time – you find out who they are when they walk through the shop door.
For all these reasons, we need to rethink how we perceive baselines. A baseline study can be a valuable tool in helping us understand change. However, it is only one out of a number of options available when building an effective MEL system. It is not a pre-requisite, not the only way to track change, and nor should it be built into every project. Baseline studies should be used when appropriate, and ignored otherwise. And if you do end up running a baseline survey, make sure not to lose the data.
Final thoughts
We hope you enjoyed the newsletter! If you did, please feel free to share it on LinkedIn and Twitter, and let us know by email. If the topic resonated then we might write a follow-up on practical approaches to developing good baselines – let us know if that would be appreciated.
Best wishes
The DevLearn Team