EEF: Core Knowledge
It is almost impossible to extensively discuss all the studies done by the EEF. In a previous blog I summarised the reports from July 2015 and in this Google spreadsheet I have tabulated all the EEF reports. One study I thought did not get much ‘airplay’ was the “Word and World reading programme” which:
“aimed to improve the reading comprehension and wider literacy skills of children aged 7–9 from low income families. The programme focused on improving the vocabulary and background knowledge (sometimes labelled ‘core knowledge’) of pupils, through the use of specially designed ‘knowledge rich’ reading material, vocabulary word lists, a read-aloud approach, and resources such as atlases and globes. The programme is based on the rationale that children need background knowledge to be able to comprehend what they read, and that improving background knowledge is an effective way to help struggling readers.”
I was interested in this project because to be honest I had heard a lot of Hirsch’s work in books by, for example, Daisy Christodoulou but not yet read a lot about actual ‘Core Knowledge’ inspired interventions (I had read some info, I think it was the Durham university press release, that she also was involved in the delivery/training of the programme). I agree with many that sometimes knowledge has been undervalued. To become an expert, you need knowledge, one particular poignant example is in educating mathematics teachers: they really need more maths knowledge than ‘just one step ahead’ of what they are teaching (at both GCSE and A level). But it also isn’t the case that when you have knowledge everything else follows automatically. With that in mind I was curious how this intervention would fare. The programme was developed and delivered by The Curriculum Centre, a charitable organisation which is part of Future Academies.
The first thing that struck me on the report page was that the study was classified as a ‘Pilot Study’ and further that “The study did not seek to assess impact on attainment in a robust way”. For almost £150k I would expect there to be a bit more ambition? The three aims of the evaluation (pilot?) were (i) to assess the feasibility of the approach and its reception by schools. (ii) to assess the promise of the approach and provide recommendations that could be used to improve the approach in the future. (iii) to provide recommendations that could be used to design any future trial, including an assessment of the appropriate size of any future trial. Especially the third aim seems a bit premature, although granted, in the report the answer to the question “Is the approach ready for a full trial without further development?” is no. This is justified because the results will show there are some big challenges.
The report has some very interesting sections:
- There is an overview of previous ‘Core Knowledge’ research. This overview shows very mixed results with extremely positive but also extremely negative effects. There are numerous issues with potential bias as well, which makes the evaluators conclude “Although widely implemented, the evidence base linking the CK approach to improved literacy is currently underdeveloped. Evaluations to date have commonly adopted matched designs and have been developer led or funded.”. I think this is sufficient grounds to study further.
- Having reiterated the aims on the start page, I was surprised in the report to see the ‘likely magnitude of the effect’ as objective. Again, it seems set up to provide further funding for a largescale effectiveness trial.
- The sample concerned eight primary schools, with a further eight schools in the same areas acting as control. There were two year-groups in each school (Year 3 and Year 4). It was further assumed that there would be 90 pupils in each school (1.5 classes in each year group, and an average of 30 pupils per class), yielding a total of 720 pupils (90 pupils each in eight schools) in each group.
- I often am a bit worried about control groups who use ‘regular practice’ because this might not be a homogeneous approach. I know it is suggested that randomization partly ‘solves’ this but nevertheless I would like to know more about these ‘regular practices’. Note that this also is important from an intervention point of view: it could be that schools that have a similar approach as the intervention already (it is suggested in the report this was not the case, but it was for EEF growth mindset study) might not improve much.
-
There is a section on ‘significance’ in the report which originally, as one of the evaluators mentioned, also was in the ‘Philosophy report’ (note this can also be seen in some references in the reference list which are not in *that* report but are in this one).
The last sentences seem rather dismissive of approaches used in most of the EEF reports. - The intervention was well received, but what I wonder is whether this was to be expected as, as far as I can see Future Academies mainly featured. I could imagine that, although there were no ‘knowledge’ programme in place, there might already be a certain culture. Of course, this is perfectly fine, but I think ‘teacher reception’ of a programme only is a small element of its total appeal.
- The section on ‘lesson implementation’ also was very interesting. It seemed to show that the implementation was generally well conducted, which seems a bit contradictory with a later point. But the most fascinating point to me was:
“It appeared that the highly prescriptive and structured lessons were both an advantage and a disadvantage. Most teachers said they liked the fact that the lessons were planned for them and there was minimal preparation on their part; some, however, adhered so closely to the prescribed programme that the lessons appeared contrived and there was little opportunity for open discussions. In contrast, where teachers attempted to initiate discussions, their lack of general knowledge and confidence in taking the discussions beyond the text was sometimes apparent.”
One of the conclusions addresses this lack of subject knowledge:
“In some lessons, teachers’ subject knowledge did not appear to be sufficient to support an in-depth discussion with pupils about some of the topics within the programme curriculum. This suggests that additional training or support materials may have been beneficial.”
I think it’s a bit unfair to say that teachers’ subject knowledge did not appear to be sufficient (apart from the fact that we are dealing with a self-selected set of teachers from one specific academy chain) as (i) the intervention was quite prescriptive, and (ii) the recommendation shows that it might be missing some features in the design of the intervention. There were more of those ‘areas of improvement’, for example in visuals and the quality of the workbooks.
- In light of the first bullet it is remarkable that the Curriculum Centre (TCC) designed the teacher survey themselves. And it could have been much better. The report shows:
- There is quite a long list of factors supporting implementation and also a longer list with barriers. The teacher turnover within participating schools was striking.
- Finally the effects, which in the ‘web page’ conclude “did not indicate a large positive effect”, actually indicate a very small (probably non-significant 😉 negative effect. I think the conclusions presented on the webpage are a bit coloured.
The picture for FSM and gender differences are slightly different but not very notable.
Overall, I feel it can be said that the Core Knowledge intervention was not effective, although teachers felt it was and liked the intervention. There also seemed to be many things that could and should be improved in the intervention. In the meantime it can hardly be said there is evidence to suggest Core Knowledge might be more effective than ‘regular practice’ (whatever that may be). Sure, teachers in the participating schools like the intervention, but is this enough to warrant its implementation? The recent ‘evidence informed’ developments would suggest not, after all many myths are widely accepted. The suggestion that teachers lack subject knowledge, if true, might result in recommendation for teachers’ subject knowledge but I think it’s a bit ‘easy’ to suggest that this might have impeded the implementation of the intervention. Designers of an intervention need to take the teachers into account; after all they need to deliver the programme. This should be a feature of the complete intervention. So the overall judgement at the moment is that it is not effective and many aspects of the intervention should be improved. I think it would be strange if results like these would culminate in a larger effectiveness trial.
In my book ‘knowledge’ remains a very important, maybe the most important, ingredient in developing both skills and understanding. There are good reasons to assume this, as I will try to elaborate on for mathematics education in a future post about ‘memorisation and understanding’. But even the way you organise such ‘knowledge’ through interventions needs empirical evidence. Unfortunately, this EEF report on the Hirsch inspired ‘Core Knowledge’ does not provide such evidence.
Reblogged this on The Echo Chamber.