Caroline Bowen – “What makes good research evidence?”

In writing Making Sense of Interventions for Children with Developmental Disorders, Pamela Snow and I used word “scientific” 99 times, and “evidence” 333, so it is no surprise that Dorothy Bishop wrote in the Foreword that the stance of the book is scientific and evidence-based.

We couple “evidence” with hand-picked adjectives, cautioning sternly against testimonial “evidence” (Dixon, 2017) and puff pieces on the “our research” tabs of certain websites, as anecdotal, bogus, inadequate, limited, minimal, slim, or based on anecdata. We explain optimistically that for some interventions evidence is emerging, hopeful, nascent, or at least theoretically plausible. And, we point the way to compelling research evidence that is current, authoritative (ideally, peer-reviewed; though you may miss noteworthy ideas and hypotheses if you only read “luxury journals”), and published in quality outlets that seek independent, scientifically-derived data and interpretations from well-designed studies.

While writing, we grew adept at spotting the tell-tale signs of pseudoscience: the marketing hype, the fulsome endorsements, the promise that one intervention would fix a raft of disorders, brain diagrams with flashing synapses, and talk of the “latest frameworks”, “unique concepts and strategies”, “cutting-edge breakthrough paradigms” and “new ways of thinking”. We contrasted pseudoscience with science, which builds conservatively on reputable theory and established evidence-bases, reporting its findings without razzamatazz.

In recent years, health has seen strong growth in the use of evidence-based decision-making in clinical settings, and patients/clients/students, their families, and the wider community benefit from efforts to enact Evidence-Based Practice (E3BP, where the superscript “3” denotes that the practitioner, the client and family, and the evidence are all essential to the equation). The health sector sometimes messes up, but at least in industrialised countries we can realistically expect evidence-informed thinking behind decisions about care, and for those receiving care, and/or their caregivers, to have a meaningful voice in the process. We can question the line of thinking, seek second opinions, and can often check the evidence for ourselves (Pagnamenta & Joffe, 2016).

As a field, Education has a less unified position on the evidence trail. Some teachers dismiss sociologists’ contentions that it is a dissimilar journey, where usual conceptualisations of evidence are considered irrelevant. Popular postmodern concepts of democracy, “strengths-based” curricula and “different discourses”  are embraced with their diverse “meanings of evidence” (for example, Biesta, 2010) and different definitions of what constitutes Evidence-Based Education (EBE).

Perhaps that means that appropriate responses to “What makes good research evidence?” can be: “It depends who is asking” and “Is that the right question?” Of course, we can ask “Is the UVW Reading Programme evidence-based?” or “Is XYZ-Therapy backed by evidence?” We do better, however, to seek more nuanced answers, by asking specifically, “What is the level of evidence for UVW and XYZ” (Ebbels, 2017). Among the interventions that thinking, mainstream professionals might consider implementing, most of the “good guys” have a way to go especially in terms of knowledge translation and treatment fidelity, and most of the “bad guys” aren’t all bad.

“Health” must not present, or think of itself, as “superior” because of its duty to E3BP, since by genuinely embracing it, the health sector simultaneously assumes obligations to intellectual humility, curiosity, learning and leadership. Accordingly, practitioners aligned with E3BP or EBE stay abreast of the literature, are flexible thinkers and lifelong learners, who rate scholarship, change their minds and their clinical or teaching practice in response to new and relevant research findings, theoretical shifts and societal change. Conversely, SLPs/SLTs and teachers who are uncommitted to pursuing the E3BP or EBE pathways respectively, tend to preserve an entrenched course, regardless of how their professional milieu changes.

In all discussions of levels-of-evidence or “good research evidence” with families and colleagues, E3BP and EBE devotees must emphasise that there are two main kinds of non-evidence-based practice. First, practices for which the science argues decisively against the treatment or teaching methodology is “in”; and second, supported only by opinions, anecdotes, testimonials, methodologically weak published studies or “grey literature”.

At a minimum, in detecting good research evidence for an intervention, we must seek at least one current, relevant, independent peer-reviewed supporting publication from at least one high-quality journal, and know that replication of those findings, reported in journals of good standing, moves the intervention up to a more trustworthy level. Then, we must ask, “Can I apply it in my world?”


Biesta, G. J. J. (2010). Why ‘What Works’ Still Won’t Work: From Evidence-Based Education to Value-Based Education. Studies in Philosophy and Education, 29(5), 491-503.

Bowen, C. & Snow, P. (2017). Making Sense of Interventions for Children with Developmental Disorders. Guildford: J&R Press.


Caroline, Courtenay Norbury and Pamela Snow at the NAPLIC conference in Birmingham on May 6th 2017


Caroline Bowen is an Australian Speech-Language Pathologist.

Written by Caroline Bowen at 00:00


Annoyed that there's no 'like' button so registering my 'like' thus.
May 27, 2017 01:27