Caroline Bowen – “What makes good research evidence?”

In writing Making Sense of Interventions for Children with Developmental Disorders, Pamela Snow and I used word “scientific” 99 times, and “evidence” 333, so it is no surprise that Dorothy Bishop wrote in the Foreword that the stance of the book is scientific and evidence-based.

We couple “evidence” with hand-picked adjectives, cautioning sternly against testimonial “evidence” (Dixon, 2017) and puff pieces on the “our research” tabs of certain websites, as anecdotal, bogus, inadequate, limited, minimal, slim, or based on anecdata. We explain optimistically that for some interventions evidence is emerging, hopeful, nascent, or at least theoretically plausible. And, we point the way to compelling research evidence that is current, authoritative (ideally, peer-reviewed; though you may miss noteworthy ideas and hypotheses if you only read “luxury journals”), and published in quality outlets that seek independent, scientifically-derived data and interpretations from well-designed studies.

While writing, we grew adept at spotting the tell-tale signs of pseudoscience: the marketing hype, the fulsome endorsements, the promise that one intervention would fix a raft of disorders, brain diagrams with flashing synapses, and talk of the “latest frameworks”, “unique concepts and strategies”, “cutting-edge breakthrough paradigms” and “new ways of thinking”. We contrasted pseudoscience with science, which builds conservatively on reputable theory and established evidence-bases, reporting its findings without razzamatazz.
 

In recent years, health has seen strong growth in the use of evidence-based decision-making in clinical settings, and patients/clients/students, their families, and the wider community benefit from efforts to enact Evidence-Based Practice (E3BP, where the superscript “3” denotes that the practitioner, the client and family, and the evidence are all essential to the equation). The health sector sometimes messes up, but at least in industrialised countries we can realistically expect evidence-informed thinking behind decisions about care, and for those receiving care, and/or their caregivers, to have a meaningful voice in the process. We can question the line of thinking, seek second opinions, and can often check the evidence for ourselves (Pagnamenta & Joffe, 2016).

As a field, Education has a less unified position on the evidence trail. Some teachers dismiss sociologists’ contentions that it is a dissimilar journey, where usual conceptualisations of evidence are considered irrelevant. Popular postmodern concepts of democracy, “strengths-based” curricula and “different discourses”  are embraced with their diverse “meanings of evidence” (for example, Biesta, 2010) and different definitions of what constitutes Evidence-Based Education (EBE).

Perhaps that means that appropriate responses to “What makes good research evidence?” can be: “It depends who is asking” and “Is that the right question?” Of course, we can ask “Is the UVW Reading Programme evidence-based?” or “Is XYZ-Therapy backed by evidence?” We do better, however, to seek more nuanced answers, by asking specifically, “What is the level of evidence for UVW and XYZ” (Ebbels, 2017). Among the interventions that thinking, mainstream professionals might consider implementing, most of the “good guys” have a way to go especially in terms of knowledge translation and treatment fidelity, and most of the “bad guys” aren’t all bad.

“Health” must not present, or think of itself, as “superior” because of its duty to E3BP, since by genuinely embracing it, the health sector simultaneously assumes obligations to intellectual humility, curiosity, learning and leadership. Accordingly, practitioners aligned with E3BP or EBE stay abreast of the literature, are flexible thinkers and lifelong learners, who rate scholarship, change their minds and their clinical or teaching practice in response to new and relevant research findings, theoretical shifts and societal change. Conversely, SLPs/SLTs and teachers who are uncommitted to pursuing the E3BP or EBE pathways respectively, tend to preserve an entrenched course, regardless of how their professional milieu changes.

In all discussions of levels-of-evidence or “good research evidence” with families and colleagues, E3BP and EBE devotees must emphasise that there are two main kinds of non-evidence-based practice. First, practices for which the science argues decisively against the treatment or teaching methodology is “in”; and second, supported only by opinions, anecdotes, testimonials, methodologically weak published studies or “grey literature”.

At a minimum, in detecting good research evidence for an intervention, we must seek at least one current, relevant, independent peer-reviewed supporting publication from at least one high-quality journal, and know that replication of those findings, reported in journals of good standing, moves the intervention up to a more trustworthy level. Then, we must ask, “Can I apply it in my world?”

REFERENCES

Biesta, G. J. J. (2010). Why ‘What Works’ Still Won’t Work: From Evidence-Based Education to Value-Based Education. Studies in Philosophy and Education, 29(5), 491-503.

Bowen, C. & Snow, P. (2017). Making Sense of Interventions for Children with Developmental Disorders. Guildford: J&R Press.

 

Caroline, Courtenay Norbury and Pamela Snow at the NAPLIC conference in Birmingham on May 6th 2017

 

Caroline Bowen is an Australian Speech-Language Pathologist.

Written by Caroline Bowen at 00:00

3 Comments :

Julie Steele said...
Dear Caroline,
I am only a HLTA in a primary school, however I have supported SLCN pupils for nearly 20yrs and am now the resident HLTA for SLCN. With the knowledge gained and support from my local SLT I created a Targeted Language Intervention (TLI) programme to assess and support EYFS pupils in their first year. The programme focuses on comprehensive and expressive language. 1) pupils are assessed (baseline) for comp & expressive language (those needing it are referred to SLT); I use a online assessment tool recommended by our local SLT dept; those scoring less than age appropriate are put into ability groups and receive TLIs 20 mins twice a week for term 1; this is in addition to 10 mins, 1:1 non-directed play support emphasising targeted language in context; pupils are reassessed at Easter; those now age appropriate are taken off the programme and those still not age appropriate will continue with the programme (in addition to any SLT targets recommended); pupils are reassessed at the end of the year - those coring age appropriate leave the programme, those still not age appropriate continue into year 1. The data from our first year (2016) showed that out of a cohort of 60 pupils 32 scored below age appropriate (including 4 EAL pupils) - by Easter there were only 18 scoring below age appropriate (1 EAL) - by the EOY only 8 required continued SLCN support into year 1. We have continued with this each year now with very similar results (we work in what is called a deprived ward with above average number of families in receipt of benefits/single parents/ low income/ low ability parents).
My programme encourages expressive language enrichment in small friendly groups, in a quite area - followed up with 10 min 1:1 language support in class / playground. Teachers and TAs also supported the 10 min 1:1s. It can be implemented by anyone with a short amount of training (simple one-sided sheet of paper with instructions) and can even be divided into 2 x15min sessions if there is a time constraint. Most resources are already in the classrooms, only the recording sheet is specific, and this is to record data and to show progress (Ofsted etc). The teachers now include language groups in their planning and both Teachers and TAs have targeted groups of children to support. We found that not only did this programme support the SLCN pupils, it also helped EAL and ASD pupils (visual based & hands-on resources). In addition to this, we can challenge the more able pupils to extend and enrich their language too.
My question is - do you think there is any way I can get this programme out into the public domain and into other schools? Or will people laugh at me because I am not a teacher, nor do I have any titles/ letters after my name?
Best Regards
Julie Steele
April 15, 2018 03:09
Wow, nice post! Well written and informative!
September 20, 2017 03:52
Annoyed that there's no 'like' button so registering my 'like' thus.
May 27, 2017 01:27

Comment

Tags