If you are a school-research lead then this post is for you. In particular, if you are interested in using evidence – be it research, school data, stakeholder views and practitioner expertise – within your school, then this post will help you understand the different stages of the evidence-informed inquiry process. To help with this, I will be drawing upon the work of Chris Brown, (UCL Institute of Education), Kim Schildkamp and Mireille D. Hubers (both of the University of Twente) and their presentation COMBINING THE BEST OF TWO WORLDS: INTEGRATING DATA-USE WITH RESEARCH INFORMED TEACHING FOR SCHOOL IMPROVEMENT which they made at the recent International Congress for School Effectiveness and Improvement conference held earlier this year in Glasgow.
The rest of this post will: first, summarise the thesis underpinning the development of Brown et al’s model; second look in more detail at the proposed cycle of evidence-based inquiry; third, identify a number of potential limitations/design flaws within the proposed cycle; fourth, put forward an alternative model which is based upon evidence-based practice.
The underpinning thesis
The main ‘thesis’ underpinning Brown et al’s presentation was a call to integrate data-based decision-making (DBDM) and research-informed teaching practice (RITP) into a comprehensive professional learning-based approach that is designed to enhance teaching quality, a result increased student achievement. As such, the model is attempting to use the strengths of each approach to offset the weaknesses of the other. Brown et al summarise the offsetting benefits of each approach as follows:
- RITP is not based on a real need in the field vs DBDM starts with the vision and goals of a specific school, focusing on a context-specific problem.
- DBDM data can inform educators about problems in their school, but what causes these problems vs RITP educators can draw upon a variety of effective approaches to school improvement
- RITP is the ‘one size does not fit all’ vs DBDM based on data, schools develop a context-specific solution
- DBDM is that data can be used to pinpoint possible causes of a problem, but educators may still not know the best available course for school improvement vs RITP picking a promising solution, based on an existing evidence base (slide 23).
On initial reflection, this model provides an incredibly useful way of thinking about the relationship between data and research evidence within the inquiry cycle, with the local data setting the scene for the necessary research evidence. Indeed, one of the things I have been thinking about lately is where ‘local data-collection’ fits within the models of evidence-based practice and evidence-based medicine which I have previously described and advocated. And this model clearly places local data prior to seeking the research evidence.
Potential limitations
However, for me, the model has two limitations to be taken into account: first, the determination of causes coming before the collection of data; second, the potential for increasing the risk of confirmation bias. So let’s now examine each of these limitations in more detail.
Causes before data or vice versa
The eight stage model presented places the stage of determining possible causes of the problem before collecting data about causes, which to me seems to be out of sequence and in reverse to order to what they should be. Indeed, there are other models of data led inquiry where data collection comes before diagnosis – and this is illustrated in the four stage model of inquiry popularised by Roger Fisher and William Ury in their classic book Getting to Yes
In this model, the data stage focuses on identifying what’s wrong, and what are the current symptoms, and then moves to identify possible causes in the diagnosis phase, with the next two phases being direction setting and action planning. For me, and I know when you use these cycles you move back and forth between the stages – it seems far more sensible to place the data collection before the diagnosis phase. In addition, and I thank Rob Briner for this observation, the proposed process seems to separate the use of the research evidence from the diagnosis phase. Yet, why would you not use research evidence to help you identify possible causes of the symptoms being experienced.
The potential for cognitive bias
By placing both vision and goal setting next to determining causes and the drawing of conclusions prior to the search for the solution in the research evidence, may lead to confirmation bias i.e the tendency to selectively search or interpret information in a way that confirms your perceptions or hypotheses. So determining the vision and goals may lead to certain problems to be highlighted as they are consistent with the already established, whereas other problems or data inconsistent with the ‘vision’ may be ignored. Furthermore, given that conclusions are drawn before searching for research evidence – this may lead to research evidence being sought which is consistent with the conclusions that have been already been drawn.
A possible alternative
Barends, Rousseau and Briner (2014) provide an extremely useful definition of evidence-based practice, which identifies a six-stage process in making decisions which are based upon evidence. In this model the search for and acquiring of evidence happens at stage two, with research evidence, organisational data and stakeholder views being accessed. This leads to the subsequent appraisal and aggregation of the evidence resulting in the incorporation of the evidence into the decision-making and action planning process.
A seven-stage cycle of evidence-informed inquiry
Some final words
Brown et al make a powerful case for combining DBDM and RITP and have developed a potentially useful initial synthesis of the two processes.
However, the proposed model has some inherent limitations, be it diagnosis taking place before data collection and increased possibilities of confirmation bias taking place. Finally, an alternative model – the 7 A’s of evidence-informed inquiry – is put forward as a way to think about the process of linking DBDM and RITP.
This is a re-blog post originally posted by Gary Jones and published with kind permission.
The original post can be found here. Read more from Gary by clicking here.
You need to Login or Register to bookmark/favorite this content.
Be the first to comment