As someone working with educational technology in both research and industry, I am constantly facing the problem of how to evaluate the impact of any technology intervention. With my focus being on the development of higher-level thinking skills, this makes it even more challenging, as developing such skills takes time and are not easily measured by the traditional pre-test and post-test approach.
During my investigation of possible approaches to help with such evaluation, I came across the theories of CHAT (Cultural Historical Activity Theory) in “Vygotsky’s Neglected Legacy: Cultural-Historical Activity Theory” by Roth and Lee, and ‘Expansive Learning’ by Engeström, which also builds on CHAT. While I won’t go through the details of these, as that would make for a long and boring article (and partly because I haven’t fully understood them yet), I’ve already gained some very useful insights on what to look at when evaluating learning.
The main point for me was that it enforced my view that we should move away from judging how successful any intervention is based solely on the ‘subject-specific’ knowledge that students have gained (or were expected to gain) – despite that being the easiest bit to measure, and probably why everyone uses it as they analytical metric.
CHAT theory widens our view of what to look at in any learning session. This includes, in addition to subject knowledge, skills such as:
• Collaborating with others.
This largely depends on how the roles were divided within the task, which CHAT formally terms ‘division of labour’. Students may develop and demonstrate different types of social skills within a group. These are worthy of observing and developing as working in groups is a key skill that is required and valued by employers.
• Using available tools to reach the desired objective.
Here, CHAT helps in expanding our interpretation of tools, e.g. to go beyond technology, and to look at the use of language or different thinking skills as tools to achieve a goal.
• The ability to adapt to, and work within, the rules/ constraints of the community and environment where learning is expected to happen.
Such rules can be those imposed by the classroom and the teacher, the school, or even the community outside the school.
In addition to these, CHAT emphasises the importance of looking into issues related to the individuals themselves, e.g. motivation, confidence, behaviour, and identity. As highlighted by Roth and Lee, these are integral to cognition, knowing, and learning; they are not independent or peripheral factors. Therefore, focusing on the material being taught rather than the students who should be learning, leads to ignoring many of the integral parts related to cognition – rendering a teaching process that is highly inefficient.
Improvements in any of these points is a considerable achievement, which can be overlooked when only measuring progress on subject-specific knowledge.
The concepts of learning by expanding, on the other hand, help in appreciating the benefits of each of the above points from an interesting perspective: that of expanding action possibilities (and thus opportunities). Things can look different when looked at in this way, and that’s why I am a theories fan: getting different lenses to see the world through. We can evaluate the value of any skill in terms of whether it expands our opportunities, e.g. learning and employment. This is obvious for basic skills such as reading and writing.
One of the arguments I particularly liked was about working with peers with a similar set of skills and knowledge. Peer learning has always been discussed as very useful – because different people have different skills and knowledge, so people learn from each other. According to this view, working with someone with the same set of skills and knowledge will not be of benefit to learning. However, if we look at working with such peers in terms of whether this expands their range of possibilities, our view will change.
Solving a problem with peers expands not only the range of actions that they can do but also the actions they ‘will have to do’. Students will have to make explicit and explain, some of their thinking to others in the process: this is likely to lead to arguments, explanations, and debates that are very useful to the learning process. This argument is even more evident in situations where students ‘learn by doing’, e.g. labs or field trips. There are certain things that can’t be done individually e.g. measuring large areas or using certain devices. Working with peers will expand the range of possible actions to make things otherwise not possible, possible.
The same expiation of the possible actions/opportunities argument can be applied to demonstrate possibly less- recognised values of many other skills, or the use of different tools in learning such as smartphones, or search engines.
So while my understanding of CHAT so far helped me look at a much wider set of aspects (including the variety of skills used in a learning session), Expansive Learning gave me another way to look at how such skills can benefit students in their learning process.
The conclusion for me, and as far as evaluating new technology or learning approach intervention, is that learning about, or using, skills and tools that expand opportunities of learning, should not be treated as inferior to demonstrating progress in subject-specific knowledge. Now just need to figure out how this can help with my evaluation challenge.
This article was first published in the September 2015 Edition of UKEdMagazine. Read the magazine freely online by clicking here.
Dr. Ahmed Kharrufa is the Director of Reflective Thinking and a Research Associate of Newcastle University. His research interest is in how technology can support learning, with a focus on collaboration. Ahmed has developed Digital Mysteries, which involves students problem solving together while developing critical thinking and communication skills.