Running Records are an uninformative waste of teacher time
33 Replies
I’ve been doing lots of assessment of my clients’ skills in the following areas lately:
- Receptive and/or expressive language
- Articulation
- Phonological awareness
- Phonological/auditory memory
- Rapid Automatised Naming
- Word and pseudoword reading accuracy and efficiency
- Spelling.
These allow me to identify problems in their reading and spelling systems, and work out how significant/severe these problems are, and what to do about them.
I use the robust, evidence-based Simple View of Reading (SVR) to guide my decision-making.
Wherever possible, I use valid, reliable, standardised tests for assessment. However, I once administered a Running Record to a child with selective mutism, because she would talk to me, but not other adults at school (we were working on it). Her class teacher thus asked me to administer the assessment required by the school, which (sad face) used a multi-cueing model of reading and a text level gradient approach to reading assessment.
Running Records, like most widely-used book-levelling systems (LLI, PM, Reading Recovery, etc), focus on language meaning much more than language structure (speech sounds, their spellings and meaningful word parts), though language structure is the main basis of our writing system.
Running Records should be relegated
Running Records are based on the now-discredited multicueing/three cueing model of reading. They’re highly subjective, and take far too long to find out not much. I don’t know why any school is still using them in 2019, let alone why they’re still on my state’s Education Department’s website.
The Running Record I administered years ago didn’t tell me much about the child’s reading skills that I didn’t already know because I knew about her listening, speech, phonemic awareness and word-level reading. I could have almost completed the assessment form without listening to her read, except some of its questions came from Planet Multicueing, and made no sense to me. I did what I could, gave it back to the nice teacher, and heard no more.
The three cueing system and the four humours
Asking a speech pathologist or teacher who understands the current reading science to administer a Running Record is a bit like asking a modern doctor to assess the Four Humours.
The Four Humours were the four bodily fluids of ancient and mediaeval medicine – black bile, yellow bile, phlegm and blood – thought to influence temperament and health. They were thought to require balancing in amount and strength for good health, and surpluses or imbalances in the four humours were thought to affect not only health but also personality.
A modern doctor asked to assess the Four Humours would simply say, “that’s not how the body works, so I can’t do that”. They’d insist on using a scientifically-based model of health, and prefer objective assessment tools developed in accordance with this model: stethoscope, thermometer, blood tests and so on, not rely on their own observations, let alone prescribe the leeches, infusions and vomiting which made sense to doctors using Four Humours thinking.
Everyone now knows that the Four Humours aren’t how the human body works, but many of today’s teachers still don’t know that children do not learn to read words by using meaning, structure and visual cues. The three cueing system/multicueing model of word-level reading is still the basis of Running Records, and these are still widely used in schools.
You can watch someone administering an assessment along Running Record lines here:
First the child is told to read the book silently, and reminded to look carefully at the pictures, as “making connections between the text and illustrations is an important reading strategy”. The video does not say that children should be encouraged to sound out words they don’t instantly recognise, and discouraged from guessing words from pictures. Once words are identified accurately, pictures can be used to support story comprehension and enjoyment (the oral language part of reading), but should not be used as word identification crutches.
Next, the child is asked to summarise the story, and then read it aloud. The teacher ticks each word the child gets right on the scoresheet, telling the child any words he doesn’t know, and writing down the child’s errors. The teacher then asks the child some comprehension questions, some of which require inference and the application of general knowledge. Amusingly, the child in this video got the first, literal, supposedly-easiest question wrong, and the rest right.
The teacher then goes back through the form and codes and counts the child’s errors and self-corrections, and “in terms of whether they used meaning, whether they used structure or whether the child used the visual information system”.
Since reading scientists have shown that context and syntax play almost no part in accurate word identification, and the multi-cueing model on which this coding system is based has been debunked, this is a waste of precious teacher time.
Stanislas Dehaene (who will be a keynote speaker at the Language, Literacy and Learning Conference in Perth in April) gives a nice video explanation of how the brain actually learns to read here, and David Kilpatrick unpacks the psychological and educational research on this here.
The Running Record form then has a section for recording “Reading Behaviours Observed” which starts with a check box for “Concepts about Print established”. Concepts about Print means things like holding the book up the right way, turning the pages in the right direction and looking at the book not licking it. I’ve never met a school-aged child without a severe intellectual disability who doesn’t do these things correctly, so perhaps this is one of those educational hooray-everyone-gets-a-tick-for-something things.
There’s a check box for “recognised high-frequency words in the text”, although the form doesn’t seem to identify which words are the high-frequency ones, clarify how many of them the child must recognise before the box is ticked, or explain whether to count high-frequency words that are sounded out (i.e. identified) rather than being instantly recognised. On Planet Multicueing, they don’t know about orthographic mapping.
There is a check box for “Applied knowledge of letter-sound relationships to accurately decode some words”, but it’s unclear whether this box should be ticked for words the child reads correctly without having to sound them out, or just some (two? five? ten?) words for which the child can be heard saying individual sounds and then blending them (correctly?) into a word.
Further check boxes allow the assessor to subjectively evaluate “Strategies” such as “attending to meaning” and “searching for print details” as well as “fluency”, though as David Kilpatrick explains at 1:07:45 on the clock in the video here, “the lion’s share of your fluency is determined by your sight vocabulary”, and fluency is not really its own separate Thing (Dr Kilpatrick will be speaking at LDA seminars in Australia this coming August, details will be here soon).
What Running Records do not tell us
Running Records do not attempt to assess children’s phonological awareness. Poor phonological awareness is the most common underlying cause of reading problems, so this is quite an oversight. The version in the video above doesn’t even mention phonemes, it just refers to “visual” errors. There also seems to be nowhere on a Running Record for children’s articulation errors, though these are often a sign of phonological processing weakness.
A Running Record is not designed to tell us which phoneme-grapheme correspondences a child knows, although I can tell you from watching the child in the video read that he seems to be able to decode words with “short” vowels and basic consonant spellings with VC, CVC, VCC, CVCC and CCVC structures, knows about past tense -ed, open syllables with “long” vowels like “no”, “he” and “my”, and consonant digraphs “ck”, “sh”, “th”, “ss” and “ff”.
However, I could tell you that kind of thing from watching him read any book written for a child around his age. However, it doesn’t matter which book he reads, I won’t be able to tell you whether he knows all the graphemes and word types he’s been taught, unless I use an assessment specifically designed to sample them all, such as a word reading test.
The child in the video was also able to read some words containing other digraphs: “down”, “please”, “ride”, “bike”, “you”, “your”, “too”, “looked”, “said” and “little”, but further assessment would be needed to find out whether he can read unfamiliar words containing these digraphs. He didn’t know the digraphs “ay” in “today” or “al” in “walk”, but other common digraphs were not sampled.
The child read “back” as “dack”, then changed it to “duck” so he is still reversing some letters, and sometimes guesses words, in this case on the basis of sound not meaning. This suggests some Set for Variability skills, and he was also able to correctly work out the words “path” and “grass” after sounding them out with /a/ as in “cat” (Set for Variability is discussed from 32:22 on the video clock here). I don’t know how to formally assess Set for Variability, but I wish I did.
Running Records tell us nothing about a child’s phonological memory or working memory, though these can have a massive impact on a child’s learning. Running Records also don’t help us understand kids’ oral vocabulary or Rapid Automatised Naming skills, though deficits in these areas are strong predictors of ongoing reading difficulties, more details are here.
Like many young school-aged children, the child in the video says “holded” not “held” and “hisself” not “himself”, so is still learning irregular past tense verbs and reflexive pronouns, but this kind of incidental linguistic information is not captured by the Running Record.
Finally, and most importantly, Running Records don’t provide more than a subjective comparison of a child’s skills with other children the same age. They don’t and can’t tell us, clearly and objectively, who is falling behind and needs intervention. This still requires teacher judgement, and if the reports I get from parents are anything to go on, many teachers are still condemning children to ongoing reading failure by saying “wait and see“.
Humour me
There’s a lovely Creative Commons emoticon version of the four different personality types thought to result from too much of each of the Four Humours:
If I wasn’t far too busy I’d make stickers of these saying “and don’t forget the four humours!” and make them available to readers of this blog to stick on all mentions of the Three Cueing System/Multicueing on school noticeboards, until the blasted things are all taken down and shredded. A kind of fun-though-going-on-a-bit-too-long form of BS Bingo, to keep our spirits up.
Tests I use most
Here are the tests I’ve been using a lot lately, with their age ranges, what they assess, and how long each one takes:
| Test | Age range (years:months) | Administration time | Information provided |
| Clinical Evaluation of Language Fundamentals 5 (CELF-4) | 5:0 to 21:11 | 30-45 minutes for core subtests | Listening and speaking (receptive and expressive language) |
| Articulation Survey | 3:5 to 7:11 | 10-15 minutes | Consonant sound production |
| Comprehensive Test of Phonological Processing 2 (CTOPP-2) | 4:0 to 24:11 | 40 minutes | Phonological awarenessPhonological memory
Rapid Automatised Naming |
| Rapid Automatised Naming/Rapid Alternating Stimulus tests (RAN/RAS) | 5:0 to 18:11 | 5-10 minutes | Rapid Automatised Naming |
| Phonological Awareness Screening Test (PAST) | Preschool – adult | About 10 minutes | Phonological awareness and proficiency |
| Castles and Coltheart 2 (CC2) | 6:0 to 11:6 | About 10 minutes | Word and pseudoword reading accuracy |
| Test of Word Reading Efficiency 2 (TOWRE-2) | 6:0 to 24:11 | 10 minutes or less | Word and pseudoword reading efficiency |
| Test of Written Spelling – 5 (TWS-5) | 6:0 to 18:11 | 20 minutes | Spelling |
The recommended times on some of these tests are pretty generous. The actual testing part of the TOWRE-2 is two 45-second tasks, so it’s possible to do it in well under 10 minutes with a cooperative older child or adult with good oral language.
These tests give me a rich source of information about how far a client’s reading and spelling-related skills are behind their peers, what intervention they need, and whether they are likely to have long-term problems.
Most of these tests are expensive, clinical ones that can’t be used by classroom teachers, sometimes the test forms cost ~$10 each! However, the CC2 and other MOTIF tests are free and can be used by teachers, and the Articulation Survey is inexpensive and can be used by teachers.
Form A of the PAST and its instructions also seem to be freely available online, though I’d recommend buying the book Equipped for Reading Success (now available in Australia for $99.95 from Silvereye) because it contains all four forms, relevant theory and plenty of finely-graded phonological awareness activities.
I have suggested a few minor edits to the PAST author, Dr David Kilpatrick (did I mention he will be presenting seminars in Australia for LDA in August?), to make this test more relevant to Australian English, where deleting the “l” sound from the word “love” does not result in the word “of”. Fingers and toes crossed he can find time for this, and if anyone has time to go through his One Minute Activities and identify the Americanisms that don’t work in our accent, I’d be happy to share suggested changes to these here, so that people can make their copy of his book more relevant for use with Aussie kids.
My list of assessments that are more consistent with the reading science than Running Records is here, and please let me know if you have a good one that isn’t listed, so I can add it.
Thanks to a nice, frustrated teacher called Therese whose email prompted me to write this post, and to Heidi G for proofreading it.
33 responses to “Running Records are an uninformative waste of teacher time”
Leave a Reply
The Spelfabet shop has affordable, downloadable spelling teaching resources.
“Guided Reading” needs a whole rethink in Australia. Google this term and you will see why.
Teachers need an instructional reading lesson format and record sheet to replace the usual guided reading process and running record sheet. We need something that can be built in to current practice and something that can promote reading science within each reading lesson.
Could Spelfabet create such a template?
Hi Natalie, I agree, there is a whole educational artifice constructed around the incorrect idea that learning to read is just like learning to speak, and it needs to be pulled down and rebuilt around the scientific evidence. Would the Multilit Reinforced Reading program be of any use as a tool to replace Guided Reading? See https://multilit.com/programs/reinforced-reading-program. I am not a teacher so I’m not sure I’m the best person to create a template for teachers, plus I have about a thousand other things on my to-do list at present. All things I neglected today in order to write this blog post! All the best, Alison
Hi Alison,
I just love your break down of running records. The assessments you reference though are for speechies. I wonder if teachers are able to use them also? We use the Read Write Inc assessments here in the NT and it is proving to be really successful.
I do have one bone to pick though…. when you say, “I’ve never met a school-aged child without a severe intellectual disability who doesn’t do these things correctly, so perhaps this is one of those educational hooray-everyone-gets-a-tick-for-something things” I must strongly disagree.
I work in some of the most remote schools in the NT. Some of these kids come to school never owning or seeing an English text. These kids can speak up to four languages before school and only hear English for the first time when they are at school. Often they only hear English at school and have no other use for it. So I guess what I’m saying is the print concept skills also need to be taught to many kids. It’s not just white middle class kids we are trying to teach to read, but our most marginalised kids of Australia too.
Good luck and I look forward to your guided reading blog- hint hint-
Jasmine, hi, lovely to hear from you again. You’re right, the assessments I use are mostly for clinicians and special ed teachers, but the MOTIF ones are accessible to all teachers, aren’t they? And teachers can use the PAST, I wish they would, though having an Aussie version would really help with that. I don’t know the RWI assessments but I’m sure they are good ones too. I wish I knew all the good assessments that teachers can use, but maybe I will find out about more of them from the responses to this blog, so I can update my list.
Thanks for your feedback regarding the indigenous kids you work with who arrive at school speaking four languages but no English. I met kids like that when my sister worked for the Northern Lands Council and I went with her on a trip to Victoria River Downs Station, so I know they exist, and I know they don’t have many books, but are you sure they arrive at school not knowing which way up to hold a book or how to turn the pages? The kids I met certainly knew how to answer the phone in the phone box up next to the big house with its lawn and swimming pool (whereas they lived in windowless tin shacks in red dust, and swam in the creek with the crocs) when my sister rang to talk to the local elder about their land claim. The kids liked telling my sister to “puck off”, and then hanging up, giggling. I just got back from a holiday at Uluru and Alice Springs where I bought a whole lot of great indigenous picture books for my waiting room, and I know Mandy Nayton has been involved in producing teaching materials in indigenous languages, and I have the Honey Ant Readers on my list of things to buy, so I hope literacy-teaching is now more culturally inclusive of indigenous kids. It certainly wasn’t in 1972 when I made friends with an indigenous kid at school (we’re still friends). I’m sure there are some kids who start school without ever having seen a book, but I’ve personally never met one. I’ve also worked with some very disadvantaged kids, I had a part-time job for 15 years at a secondary school next to the Flemington hi rise, where many of the students arrived almost straight from refugee camps, had never been to school till they came to Australia, and lived in public housing with parents or other relatives (if their parents were dead, which was not uncommon) who spoke no English, and the household had few or no books. I am currently working a couple of hours a week seeing 14 kids at Fitzroy PS, and they mostly live in the hi rise, and are almost all of refugee, migrant or indigenous background. I genuinely haven’t yet met a child with reasonable cognitive skills who didn’t know which way to hold a book or how to turn the pages, even the Arabic-speaking ones knew books in English have pages that turn towards the left. I’m sure you have, and I’m sure you’re doing great work with them, but I’m not just talking about white middle class kids. The disadvantaged kids I’ve known have done well with systematic phonics, even some kids who officially had IQ scores that suggested they probably wouldn’t be able to learn to sound out words. I’m sure yours are too.
I think my Guided Reading blog goes like this: “Guided Reading sucks, why don’t you try Reinforced Reading? https://multilit.com/programs/reinforced-reading-program.” Would that work for you?
All the very best, Alison.
Yes, “guided reading sucks” works really well for me actually!
Thanks for your reply, your experiences are not that different to ours here. But yes, sadly, we get kids come to school for the first time not knowing which way is up on a book. Remember, they have an oral culture, and the only print these kids may possibly have seen before school is at the clinic or brands of food at the shop. There are no books in homes and even some communities don’t have street names, just big painted numbers on their houses.
Come for a visit sometime. It will take your breath away. I’m off to jiklmingan, ngukarr, minyerri this week. Google them.
Keep up the good fight! I don’t know how you do it. I lose sleep over one whole language proponent’s tweet these days, I’m completely hopeless.
J
I’d love to visit Jiklmingan, Ngukarr and Minyerri, and I have far too much to do here right now but if you’re still there next year, maybe. I’m old and these days I don’t care too much what people think of me. And if I can’t stick my head above the parapet and say “the emperor of running records has no clothes” then how can class teachers?
Thank you for this excellent blog. I have shared it on facebook in the hope that some of my fellow educators will read it. It is particularly pertinent to me at the moment as my school is insisting on only using RRs and has abandoned its DIBELs assessments and MiniLit intervention. I have been given the Learning Support /EAL/D role as I have been deemed too big of a risk to have my own class this year (I previously taught Yr 1 using a Structured Literacy approach which my principal vehemently disapproves of – she is Balanced Literacy all the way). So far in this role I have been used to give the Yr 1 teachers release in order for them to perform RRs on their students. I have asked to do DIBELs at least on the children who were identified last year so that I can know where their learning deficits are and so I can monitor their progress. Because I knew the answer was likely a no, I went back to an earlier blog of yours which suggested MOTIF assessments. I signed up and was approved the next day. Unfortunately I have been told explicitly not to use any of these assessments, I am to use the RRs in order to decide how I’ll group the children for their Tier 2 intervention and I am to do Guided Reading with them. It turns out that phonological and phonemic assessments are not part of my principal’s “vision”. I am at a total loss of how exactly to deal with this situation.
OH, LUISA, THAT’S AWFUL, I’m so sorry to hear it. I guess since you’re in the system, you need to do as you’re told, despite what you know about the evidence. I just don’t know what to make of Whole Language diehards who currently occupy positions of authority in our system, and don’t accept the scientific evidence. They are education’s answer to climate sceptics, and like climate sceptics they put children’s futures at risk. I’m so happy to work for myself and be able to follow the evidence not someone else’s instructions. If you live near the school, or other schools with a similar approach, maybe you could you set yourself up as an LDA tutor (www.ldaustralia.org/tutor-referral-services)? You’d have plenty of customers, and the freedom to decide for yourself how to teach. All the best, Alison
WOW! You created a firestorm on the Fountas and Pinnell page on Facebook. It’s all about what the research proves. It’s not about what we think we know from years of experience or what we like or who we are loyal to. We all have inherent blind spots to our practice, that’s why we need to look at the research.
Great! I hope it’s set a lot of people thinking. There is a great new video on the Reading League website that unpacks the “gold standard” research on Reading Recovery and Leveled Literacy intervention, I put it on my facebook today and I must add it to my blog posts about RR, LLI and as a footnote to this post. Here’s the link:
https://youtu.be/zS7ice-_mwE.
Steve Dykstra is brilliant. LLI is discussed from minute 52.33 on the video clock, and RR is discussed from minute 1:03:43.
What are your thoughts on the Burt Word Reading Test (sorry if this is somewhere and I haven’t seen it).
Thanks
Hi Diane, I think the Burt test is OK, though I haven’t looked at it in detail, and I’m not sure where or when it was normed or whether the norms are now considered current for Australia. It only tests real words, not pseudowords, so it can’t tell you a lot about decoding/word attack, and it doesn’t have a timed section so it can’t tell you much about fluency/orthographic mapping. But it’s a lot more robust and objective than Running Records! Alison
https://ies.ed.gov/ncee/wwc/EvidenceSnapshot/420 . Reading Recovery is the most successful of early reading program based on the what works site There is a link there to download the full intervention report. Aware that you can find contrary reports about RR HOWEVER the what works clearinghouse has given it very high marks for YEARS based on HIGH STANDARDS OF RESEARCH. RR makes use of Running Records. Hard to square that with what your saying Please consider ALL the research.
Sorry, Sam, but there are serious problems with both the Reading Recovery research and the way the What Works Clearinghouse operates. Please watch this YouTube video: https://youtu.be/zS7ice-_mwE, preferably all the way through, but the section specifically about RR starts at minute 1:03:43 on the video clock. Please also look at the CESE report http://www.cese.nsw.gov.au/publications-filter/reading-recovery-evaluation. Dr David Kilpatrick, who wrote a great book called Essentials of Assessing, Preventing and Overcoming Reading Problems, has been through the RR research and says it typically yields about three Standard Score points of improvement, i.e. barely discernible improvement. The best programs, which target phonemic awareness and proficiency as well as phonics and reading connected text, can give a Standard Deviation (15 points) or more of reading improvement on robust, standardised tests. I often see this in my own clients, but not because of anything special about me, I just read the relevant research, got the best programs and resources I could find, and use them. I looked at the Reading Recovery research, and thought about the multi-cueing model of reading which underpins it, and that’s why I would never use it. Running Records are subjective not objective, and robust, standardised, valid, reliable, objective tests are simply a better way to assess reading skills.
I’ve been a teacher for many years and understand that the whole language approach doesn’t work for all students.
I’ve read a number of your articles and blogs and I want to let you know that I find the way you present the divide between whole language and the phonological approach riddled with emotive language that demeans your argument.
If you want to encourage teachers to look at a phonological approach you need to use your emotional intelligence and use language that is respectful.
The running record was never designed to do those things you say it is deficit in.
I find it concerning that speech therapists are telling teachers what to do- driving their agenda. We don’t tell you what to do!! It’s just rude!
Dear Cathy, sorry to hear you think my emotive language is not appropriate. Perhaps if you sat across the table day in, day out, from children and teenagers who have been let down by the education system, and parents who cry when they talk about how miserable they are as a result, you might have a better understanding of why I feel emotional – and dare I say angry – about this issue.
We had a National Inquiry into the teaching of literacy in 2005, and there were two similar inquiries in the UK and US, and all of them emphasised the importance of systematic and explicit phonics instruction, yet in Victorian schools this still seems to be the exception not the rule. Kids are still being told to look at the picture and guess, not sound out words. They are still being sent home with lists of high-frequency words to visually memorise. They are still being taught “the sounds of letters”. It’s still a wait-to-fail system not one focussed on prevention.
New graduate teachers still tell me they have been taught little or nothing about phonology, orthography and morphology at university, but plenty about Vygotsky, who lived in Stalin’s Russia, died in 1934 and never saw an Australian classroom.
I first worked in education in 1989, so I am not a newcomer, and being in the happy position of self-employment I am able to say things that many others inside the education system cannot. You have no idea how many teachers tell me they agree with me and would say the same things if they didn’t think they’d be ostracised and/or penalised at work, and possibly lose their jobs. So I feel an obligation to speak clearly and frankly about the change that needs to happen.
If you don’t like what I write then please don’t read my blog. Since mid-2012 my website has had 3,857,659 views, so lots of people ARE reading it, and many tell me they find it helpful, and they’re the people with whom I’m interested in communicating. Lots of teachers I respect tell me they wish they didn’t have to use running records, as they find them an uninformative waste of time, and having studied research design and statistics twice at university level, I find it baffling that anyone would want to use subjective assessments when there are plenty of objective ones available. Teachers are professionals and they should have professional tools, and proper training in what science has learnt in the last 40 years about how best to teach children to read. I am going to keep saying that, unapologetically, because it’s true.
Yes. It is clearly the children you are advocating for. Otherwise they don’t have a voice. The emotion in your post is because you care, you are passionate and you know your stuff. I’m listening and others are too. Sometimes the only way to be listened to is to say it how it is!
And our job as teachers is to teach all children not just the children whom our favourite (or easiest, or funnest) teaching approach happens to work for.
I completely agree. Teachers who have quality training and experience can utilize so many assessment tools and gain important information to differentiate the reading process for each student. There is no reason to discredit the use of meaning and structure as a tool for students, as long as you are providing quality phonics instruction. Over my twenty plus years in the classroom and as a reading specialist, I have never encountered a child that learns to read the exact same way. All learners are different and the more tools and strategies we can give them, the better readers they will be. Remember that quality instruction derived from individual assessments will drive your differentiated instruction. Additionally, this conversation you are having does not need to discredit other reading programs with such negativity. Experienced teachers know what works and your passion to dissuade is a real turn-off and puts your knowledge as a teacher of reading in question.
Dear Jennifer, with respect, the scientific evidence does not support your view that every child learns to read differently. Exactly the opposite is true. You can read the book “Reading in the Brain” by Stanislas Dehaene for the full explanation, but essentially all children beginning to learn to read must create the same circuit in the same location in the brain linking the visual areas in the occipital lobe with the language areas in the parietal and temporal lobes. They do this by developing awareness of the relevant phonological segments (phonemes for alphabetic languages, syllables for syllabaries etc) and linking these to the relevant orthographic segments, in our case graphemes and morphemes. Known words’ phonological representations are already connected to their semantic representations, so the link between phonology and orthography facilitates links from orthography to semantics. Some children can figure this stuff out from the guessing-and-memorising teaching which is still mainstream, but I sit across the table day in, day out from kids who are let down by MSV, Running Records and other subjective and discredited approaches and assessments, and who are miserable as a result. Schools need to use valid and reliable assessments and evidence-based teaching approaches so all the children who are capable of learning to read actually learn, and all the strugglers get the extra help they need. There is absolutely no proper scientific evidence for multicueing/MSV or Running Records, and it really upsets me that my taxes are being used to pay for their use.
[…] the professional have objective measures to determine my child’s […]
Hi, just wondering what your thoughts on PROBE reading assessments are? I found your comments on Running records to be very thought provoking.
Hi Sue, sorry to take ages to get back to you, I was stupidly busy and then at a conference. I don’t use the PROBE myself and I’m not much of a fan of assessments of reading comprehension because they conflate word identification and language comprehension, and when you’re scoring them up you can’t tell whether the weak comprehension is due to poor word identification or poor language comprehension or both. I would rather assess these two aspects separately, and then if the child has poor listening comprehension work on their oral language, and if they have poor decoding or poor phonemic awareness and thus are not mapping words into long term memory, then target these things in intervention. An assessment of reading comprehension doesn’t tell me what to do about the problem, it only tells me a problem exists.
HI Alison, I am wondering what assessments you would use for reading comprehension if PROBE is off the table? Thanks, Jess
Hi Jess, I think reading comprehension is two things intertwined, as per the Reading Rope or the Simple view: word identification and language comprehension. I like to assess them separately so I can see whether just one of them is the main source of the problem. I’m a Speech Pathologist so I use the CELF-5 to assess listening comprehension/receptive language, and I generally use the TOWRE to assess word identification. I do have tests like the YARC but I don’t use them these days, I find that they leave me with more questions than answers. Hope that makes sense and sorry for slow reply. Alison
Hi Alison, I appreciate all your expertise and as a class teacher, I can see we share many of the same views on reading and assessment of reading.
I’m interested, beyond the obvious limitations and flaws of running records, if you can still see a place for the utilisation of the actual running record component (not the MSV component), but the reading script that you can use as a teacher to highlight fluency components of a student’s reading? As an early years teacher who uses MoTif suite assessments and has used the PSC effectively, I still find value in the running record instrument for the fluency-related observations. Do you propose or have an alternative instrument you would use for this?
Thanks for your ongoing engagement in this space.
Hi Edward, thanks for the nice feedback and interesting question. My understanding of fluency comes from David Kilpatrick, who says that almost all of the variance in measures of fluency can be accounted for by the number of words a learner can instantly recognise without having to sound them out (their “sight vocabulary” as defined by researchers, though that term is problematic because “sight word” means too many different things to different people). Whether you assess a child’s fluency when they are reading connected text or wordlists, the kids’ scores will be very closely correlated on both measures, because most of what makes you a fluent reader is how fast you can read the words. Yes, there are some prosodic and HOTS things involved in reading complex text aloud, but the main engine of fluent reading in learners without speech-language difficulties is orthographically mapping words into long-term memory.
I have bought a copy of the Observation Survey so I can get my head around it, but I am never going to use it because it is simply far too subjective, though of course if you’ve used it a lot with a range of kids, you can get a sense of what kids should be able to do at different ages. A test like the WARP (https://multilit.com/programs/warp) would be much better from my point of view, as it’s more objective, but if David Kilpatrick is right (and I am not aware of evidence he is not) the WARL would be just as good (https://multilit.com/programs/warl) and of course the TOWRE is the test Kilpatrick recommends and uses, and I’ve found it to be very useful too, and so quick! I hope that’s helpful, thanks for making me think about the question, and all the best, Alison
Hi Alison, You said that in the article that David Kilpatrick explains at 1:07:45 on the clock in the video here (https://www.spelfabet.com.au/2018/04/the-nature-of-reading-development-and-difficulties/), “the lion’s share of your fluency is determined by your sight vocabulary”.
When I go to 1:07:45 David Kilpatrick does not say those words. Does he say them elsewhere? Apologies in advance if I have made an error in looking!
Hi Georgina, the video in which David talks about fluency is embedded in this blog post: https://www.spelfabet.com.au/2018/03/our-goal-is-to-develop-phoneme-proficiency-in-kids, you’re looking at a different one. I thought I had the wrong link in my post about Running Records but it seems to be correct, so not sure what happened there, internet gremlins maybe.
Hi Alison, You said that in the article that David Kilpatrick explains at 1:07:45 on the clock in the video here (https://www.spelfabet.com.au/2018/04/the-nature-of-reading-development-and-difficulties/), “the lion’s share of your fluency is determined by your sight vocabulary”.
When I go to 1:07:45 David Kilpatrick does not say those words. Does he say them elsewhere? Apologies in advance if I have made an error in looking!
Also, do you have any links to peer-reviewed journal articles (of possible not just from Australia but the US too) about the lack of credibility of 3-cueing and running records. I can’t find any and it would be hard to convince my principal without the scientific evidence.
Hi Alison–you’re amazing! Keep up the dissemination of truth, with emotion or not.
I’d like to know your opinion on whether the Running Records could be used for “Reading Comprehension” purposes, after Kilpatrick’s strategies have been fully implemented and decodable text has been “mastered” by students. I suspect some school boards who are adopting a science of reading approach are having a hard time discarding all the leveled books. Your thoughts?
Hi Anthea, Thanks for the nice feedback, much appreciated. I’m not sure Running Records are really the best tool for assessing reading comprehension, because they are very subjective. However, every reading comprehension test measures something different, they aren’t like word-level reading tests which give you much the same result no matter which one you use. I understand the best book about reading comprehension is this one: https://www.routledge.com/Understanding-and-Teaching-Reading-Comprehension-A-handbook/Oakhill-Cain-Elbro/p/book/9780415698313. I’ve bought it, but I haven’t read it yet, sorry. I couldn’t find Running Records mentioned in the index so I imagine they aren’t recommended as a high-quality assessment strategy.
Thank you, Alison. I look forward to your review of the book when you’ve had a chance to read it 😉
[…] Spelfabet – Running Records are an Uninformative Waste of Teacher Time […]