EdSource published my commentary earlier this month showing the rise in ELA and slide in math achievement. When we follow the same classes of students over time, we see big changes in the number of students scoring at proficient and above. We do ourselves a disservice by looking solely at who’s above or below a proficiency cut score. What’s the trend for all students at all levels of achievement?
Here’s the longitudinal trend for each elementary class, presented in stacked bar charts. The percentage of students scoring at Level 1 is in dark grey; the percentage at Level 2 is in light grey. The percentage at Levels 3 and 4 are in light and dark blue. In an ideal world, the size of the blue rises over time and the grey bars shrink. That’s the case for the Class of 2026 in English Language Arts for the Class of 2026. 36,400 more 5th grade students are meeting or exceeding standards than they were as 3rd graders.
But in math, we see the opposite trend. The number of students at Level 1 is increasing over time. 39,700 fewer 5thgraders are meeting standards than they were as 3rd graders.
Friends of mine argue correctly that “growth matters most”. But what if students aren’t growing as they should? What do we do then? More thoughts on these questions to come. You can reach me via email here.
On Saturday August 10th, the Delaware State Newspublished my commentary that the state needs to look at achievement data in a new way. The current method obscures both success and academic struggles.
Rather than comparing this year’s 3rdgraders to last year’s 3rdgraders, follow the same groups of students over time. That is, the state, districts, schools, teachers and parents need to ask: how did this year’sfourth graders compare to how they performed as third graders last year? When we begin to look at the data by when a cohort of students will eventually graduate from high school, we generate data that can radically alter the conversations we have.
Unfortunately, the State News published the commentary without any of the graphs of cohorts. The visuals help make a more persuasive case. Four cohorts of students have at least two years of test score data. Each cohort of third graders improves their English Language Arts achievement by an average of five percent as they move into fourth and then fifth grade.
An even more powerful view of the Classes of 2024, 2025, and 2026 shows an upward trend as they progress from 3rdto 4thand 4thto 5thgrade.
Two thousand more students are meeting and exceeding ELA standards (shown in light and dark gray). The number of students scoring at the lowest levels (shown in light blue and purple) is shrinking. These are steady successes that need to be celebrated and more importantly, understood so that others can build on them.
In math, each cohort of students is experiencing a steady decline. The students’ slide is concerning because future math success depends on having a solid foundation in arithmetic. Gaps in students’ understanding become much harder to fill over time.
The graph for the combined Classes of 2024, 205, and 2026, as they progress from 3rdto 4thand 4thto 5thgrade raises even greater concern.
3,500 fewer students are meeting math standards as 5thgraders than they were as 3rdgraders. The number of students scoring at the lowest level (shown in purple) is on the rise, by approximately 800 students in each class.
For reasons that are too complex to show here, these trends are not a result of the Smarter Balanced tests being too easy or too hard. A 2016 external review by HumRROfound that Smarter Balanced assessments rate mostly as excellent in their match to the content and depth required for college and career readiness.
All of the Smarter Balanced states are experiencing these successes and problems, though Delaware’s struggles are greatest. It’s the only state in the consortium whose cohort reading scores start to decline in 6thgrade. A few states—California, Connecticut and South Dakota—experience a bounce up in 6thgrade math achievement.
Delaware should consider convening expert teachers in both content areas to more fully understand these trends. Begin to generate hypotheses about what might be contributing to the success in ELA and failure in math. Translate the data into information and knowledge that truly improves learning.
Cole Nussbaumer Knaflic, author of Storytelling with Data, ran a design competition last month on slopegraphs, which is a fancy term for a line graph focusing on change over time. This seemed like a good opportunity to rethink how we see PISA (Programme on International Student Assessment) data.
Since 2010, Andreas Schleicher of the OECD has been briefing American policymakers and education leaders on PISA scores. The news on our 15-year-olds tends to stay the same—we do much worse than about three dozen countries and are about average in science and reading. We have begun to tune out the news, which is a shame since many of the best performing nations have pulled further ahead over time.
There are a variety of reasons we have done this, but the way the data is presented may be a contributing factor. Business Insider’s December 2016 story included a typical scoreboard:
The problem with this data view is we don’t see how the groups change over time. Presenting the data in a slopegraph might help.
Using color helps focus our attention. Students in Singapore, shown in purple, began in 2009 about a quarter of a school year ahead of the U.S. in math, and they are now two and a half years ahead. Students in Canada, shown in black, have seen scores decline but they are still more than a full year ahead of the U.S.
We needn’t include all 65 cities and countries that participate in PISA. Israel is notable because its’ students began a full year behind the U.S. and it has pulled even. Seeing the data in a fresh way doesn’t guarantee that policymakers and the public will act any more urgently, but it certainly can’t hurt.
June 2018 (Cross-Posted from CenterPoint Education Solutions: The Point)
For the first time, many states have adopted standards that place equal emphasis on reading and writing. Most states are now deeply assessing students’ abilities to produce narrative, argumentative, and expository writing, and reading is being tested through writing across each grade.
However, a recent analysis of the strategic plans of the nation’s ten largest districts found only one—Fairfax County, Virginia—explicitly mentioning and setting goals for improving writing. Many school districts aspire to improve student literacy but are focusing too narrowly on reading and neglecting writing.
Good communication skills– speaking, listening, and writing– are vital to college and career success. Teachers who hope to develop their students’ writing to mastery levels have a remarkable asset available to support this need. The PARCC consortiumand states such as Massachusettshave released writing prompts along with samples of student responses and commentary that can serve as a powerful teaching tool.
Under the standards, students are taught writing skills that help convey thoughts and opinions, describe ideas and events, and analyze information. The work samples from real students help both teachers and students explicitly see what types of writing performances are required to meet or exceed the standards. Too often, these expectations remain opaque to students. When taught alongside commentaries about whythe writing does or does not meet elements of the standards, students can answer the question: “How good is good enough?”.
Let’s explore how to put this into practice. For example, take this fourth-grade informational writing prompt released by PARCCfrom its 2016 test. Students read several passages about Great White Sharks and then were asked:
“Using details and images in the passages from ‘Great White Shark’ and Face to Face with Sharks, write an essay that describes the characteristics of great white sharks.
A good diagnostic lesson might involve students reviewing two or more of the released samples: one that meets the standard for informational writing expression and another that falls below it. In reviewing the first example, the teacher may provide the commentary.
The fourth-grade informative writing standards represent a big jump in what’s demanded of students. They are asked to include concrete details, “use precise language and domain-specific vocabulary” and to provide a concluding statement, elements which are not expected in third-grade writing.
By engaging students to think critically about the content and ideas presented in the writing, the performance standards begin to come alive. As students review the second example below, they can begin to provide their own commentary about what another real student has done well and what’s missing, compared to the first piece of writing. Students can see that part of what separates the two pieces of writing is how they logically order information.
Example 2. (Score Point 2 “Partially Met Expectations”)
In Embedded Formative Assessment, Dylan Wiliam recommends this comparative approach and, over time, introducing more examples of genuine student work to deepen the conversation about what high quality writing looks like. Throughout the year, teachers across a grade can then give formative assessment prompts and carry out several cycles of this type of analysis.
Every state and large school district now has college and career readiness as central to its vision of what it means to be well educated. While it is clear that writing is a key component of English language arts curriculum, good writing skills are also key to success across content areas and grades. In a world of rapidly advancing technology, high school is not a good end point for anyone’s learning.
Our students have great aspirations to be surgeons, architects, auto repair technicians, journalists, and much more. Rather than expecting them to figure out the criteria for success in these fields on their own, we can explicitly create lessons around performance standards that show students “how good is good enough?”
Submitted to Thomas B. Fordham Foundation’s 2018 Wonakthon Competition:What standards should students meet to graduate from high school?
Imagine Rip Van Winkle earned a diploma from a New York high school in 1998 and worked as a printer before falling asleep. Waking up twenty years later, he wouldn’t have the skills to do his old job. In a world of rapidly advancing technologies changing most job requirements, high school is not a good end point for anyone’s learning.
The question of what standards students should meet to graduate from high school has a straightforward answer: by default, they should earn a college and career ready diploma. Indiana has been a pioneer in this space for more than a decade.
Influenced by Achieve’s American Diploma Project and NGA’s Honor States Program, in 2007, Indiana developed the Core 40. To graduate from high school, students must complete 4 years of English, 3 years of mathematics (including Algebra II), 3 years of science and social studies. Students can meet additional requirements (e.g., foreign languages, Advanced Placement courses, and programs of study) to earn the Core 40 Academic or Technical Honors diploma. Parents can opt out their students out of these pathways and earn a general diploma.
88% of high school graduates have earned the Core 40 diploma (with 34% earning Honors). Indiana also aligned the incentives across systems. Since 2011, four-year public colleges in Indiana have requiredthe Core 40 diploma. College remediation rates are on the decline and on-time degree completion rates are on the rise. The State Board of Education recently tweaked these requirements starting with the class of 2023, eliminating some tests and adding project-and work-based learning experiences.
In spite of these successes, less than half of states have followed Indiana’s lead. A 2017 studyby Achieve found diploma requirements continuing to differ greatly from state to state. Only 20 states report data on whether or not students have completed a college and career ready course of study. As their transcripts show, too many students are “meandering towards graduation.” And in the case of some urban high schools such as Ballou, students are graduating without showing up.
One reason why college and career ready diplomas aren’t the default option is that many states have an earlier problem they struggle to solve. It’s hard to require three years of college prep math when many students haven’t been successful in Algebra I.
In Florida, Maryland and New Jersey, overall pass rates on the Algebra I end-of-course exams hover around 35%. But unpacking this data in the graph below shows an even deeper problem. A third of the students who take Algebra I are 8thgraders but they are a majority of students meeting and exceeding standards.
States and districts could begin solving this earlier problem by filling in two building blocks often missing in schools: truly use diagnostic data and develop standards-based report cards.
Use Diagnostic Data to Close Gaps
At a typical urban high school in these states, the incoming grade levels of an incoming 9thgrade class might look like this:
These schools have inherited problems not of their own making. Nearly half of the new high school students are 3 years or more below grade level. An Algebra I teacher has a Herculean task trying to get everyone up to standard. What if school systems could tighten this distribution and prevent failure?
Great advancements have been made in the development and teachers’ use of daily or weekly diagnostic data to inform student learning. By diagnostic, we mean data that identifies students’ strengths and weaknesses to help the learner improve performance. As Dylan William argues, to be effective, data feedback must provide a “recipe for future action.”
Teachers in the schools using Teach to One Math are taking diagnostic data to mapfrom 5thto 9thgrade the math clusters needed to eventually be successful in Algebra. A quick look at this map shows that being able to multiply and divide fractions is an essential pivot point for students. If they move on from 6thgrade without that understanding, Algebra will be a steep, uphill climb. If we can ensure they have that mathematical understanding intact, they’re much more likely to be successful.
In Ontario Canada, researchers foundthat using diagnostic data had more a positive impact on teaching and learning more than traditional standardized tests. Any school system hoping to replicate this success also has to build the infrastructure, culture, and analytic capacity supporting data use. Over time, the typical urban high school, linked to middle schools also using diagnostic data, ought to have an incoming 9thgrade class that looks like this:
Develop Standards-Based Report Cards
My friend Patrick Riccards suggests in another entry that high schools focus on mastery and move away from seat time. One way to do this would be to establish standards-based report cards.
While most states have moved to true college-ready standards and upgraded their tests, schools continue to use letter grade report cards. When a student earns a B+ in Algebra I, it’s unclear what that local grade represents, especially if he struggles to pass the end-of-course exam. How can we expect large numbers of students to be successful when these systems aren’t closely aligned?
Report cards need to evolve beyond their late 19thcentury origins so that letter grades reflect the specific knowledge and skills a student has learned in that class. They need to be longitudinal across grade bands (K-2, 3-5, 6-8, 9-12) so a 3rdgrader knows how she’s meeting present and future expectations. This would also allow for parents and students to see growth over time. The grain size needs to be at the cluster level, grouping at clusters of standards, such as “equivalent fractions.” Rolling up to domains is too broad; detailing each standard is too specific.
These are initial ideas for what would be a very complex undertaking at scale. A few suburban districts, such as nearby Montgomery County, Marylandhave tried it and shown it’s very easy to get this work wrong and hard to get it right. TheInternational Baccalaureateschools suggest one way forward. Across their large network in more than 100 countries, an IB diploma has consistent meaning because teachers calibrate and moderate scores of external exams and teacher assessments.
With better data in the hands of teachers and a fuller picture of whether or not students’ day-to-day work is meeting college-ready standards, high school students won’t be meandering on roads to nowhere but solidly on paths to upward mobility and lives of using one’s mind well.
The U.S. Department of Education released last week a summary of achievement trends in schools receiving School Improvement Grants (SIG). This is the main federal strategy to turn around the nation’s lowest performing schools, which has directed $7 billion to 1,500 schools.
The reaction from the field is that once again, this is the same old news of mixed results. “Only a little more than half of the schools improved, while the other half saw stagnant student achievement, or actually slid backward,” wrote Alyson Klein in Education Week. Andy Smarick of Bellwether wrote that SIG could use a turnaround itself. Among the very real problems: almost half of the grant recipients were excluded from the federal summary because of changes in how state tests are given.
But isn’t the look at whether or not SIG has effects “on average” missing the point?
Imagine that the $7 billion had been spent by a biotech company seeking a cure for stomach cancer. After completing the clinical trials for “SIG-2”, they find it doubles a patient’s life expectancy. The five-year survival rate for those with stomach cancer jumps from 24 to 48%.
Would you say that “SIG-2” fails a majority of the time? NO!!! Tens of thousands people would live at least another year and the world would be overjoyed at these results.
Schools receiving SIG funds are chronically failing. While they might have some talented individuals, as organizations, they lack the “know-how” that helps a school accomplish the basics. Even getting the bells to ring on time can be difficult, to say nothing of ambitious instruction. More money may or may not solve their capacity problems. Even when making double-digit achievement gains, their move out of “crisis” is more likely into a stage of mediocrity rather than excellence.
Given these challenges, it makes sense for analysts to stop looking exclusively at average changes. Even if positive effects are seen in only 20 percent of the schools, future analyses should examine under what conditions, and why, SIG gets the results it does.