Susan Pendergrass (00:10.7)
Well, it feels like the NAEP scores were released about a month and a half ago at this point, given the current news cycle. Nonetheless, we're excited to talk to Nat Malkus once again of AEI because some pretty important information came out about how our schools are doing. And if you could just give us a quick overview, what is NAEP and why should we care?

Yeah, NAEP is the National Association of Educational Progress. And it's a large test that is done every two years in reading and math for fourth and eighth graders. And look, it gives us a great sort of finger on the pulse of where students are doing, and we can compare over time. And the sampling plan is just about as robust as you can get.

So we get a great national sample because we have 50 representative state samples. And on top of that, we have a bunch of sort of sub-stampels in some urban districts, the Tudor districts, as it were. So it gives us a great sense of how students are progressing, even when they're not progressing so great.

So what did they tell us? What are the biggest takeaways from the release of the 2024 numbers?

Is that right? Yeah, 2024 numbers. And look, this round was pretty important given that we had one post-pandemic round in 2022. And we saw lots of deep cuts during the pandemic, which probably didn't come as a surprise. But one could look at a number of things, particularly just the bounce back from the pandemic, and hope, well, students are probably getting back on their feet, so we should probably see some improvements. That's not quite right.

Nat Malkus (01:53.454)
We didn't see great improvements. In math, we had some improvement in fourth grade. And in eighth grade, scores on average were flat. There's more underneath there, but they were flat. And in reading, we really saw some continued declines. We would hope maybe that we have seen some rebound post pandemic. We didn't. We saw further declines.

And everybody should be concerned about that.

this sort of canary in the coal mine, harbinger of fourth graders. They were probably in kindergarten in 2020 when the schools closed. They probably took the brunt of the pandemic. They were little, little kids. And if you don't learn to read pretty early, first, second or third grade, you might be on a glide path towards a lack of success in your education. And what we found in

Missouri is no exception. So 42 % of Missouri fourth graders below basic and whatever the definition is currently economically disadvantaged, I want you to actually tell me what it is. In Missouri, 11 % were on grade level, like one in 10. Nine in 10 Missouri fourth graders not on grade level and a troubling number who are essentially illiterate. They're 40 % below the basic line. They don't have fluency, vocabulary, comprehension.

And I think the national numbers were similar. What does that portend for you?

Nat Malkus (03:28.758)
I mean, look, I think it's important to note the numbers that you're talking about, about the below basic kids. There's sort of four categories, below basic, basic, proficient, and advanced. And look, there's been some talk over the years, well, is proficient actually proficient? And some people will say, well, they can't even read on grade level and so forth. And a lot of those things need to be skeptically looked at because

NAEP comes up with categories for what they think is advanced and proficient. you know, while I have a lot of sympathy for those categories, they don't really get to define that for the nation. Nonetheless, it's a useful yardstick. I will say that the below basic students are below any standard of where we would want them to be. It is bad. We do not want a lot of kids in those levels. And on reading,

the proportion of fourth graders, and I don't have these in front of me, but if memory serves, was lower than at any, or it was higher than at any point since 1992. And I think it was about a third, if I'm correct. And then in eighth grade, it was more than that. And it was a higher proportion than we've ever seen before. look, you know,

When we think about these scores and we think, well, what does that actually mean? I'm here to tell you that below basic is below anything that we would find acceptable. And we do have a large share of students that are in that zone and that cadre is growing. So when people say that, you know, this is a five alarm fire and that sort of thing, for those students, that's where my concern lies.

It is larger for disadvantaged groups. These things have been, that's been the case for a long time. However, I don't think it's the disadvantaged groups that are driving that lower level. think that it's sort of a population-wide increase in our lower performers. And it's hard to figure out what explains it.

Susan Pendergrass (05:41.09)
Yeah, what explains it and what to do about it? I mean, I heard in the last two days, don't get rid of the Department of Education. We need them more than ever now that scores are low. Or you gotta get rid of the Department of Education because they've been in charge and look, the scores are low. As though somehow the US Department of Education has had an impact on that. But the reality is it's 50 systems. And some systems, some states, like Mississippi, scores actually improved and they've been on a steady improvement. Some states like Missouri, they just continue to go down. They went down more steeply just between

2019 and 2023, but they're still going down. we have 50 systems at large nationally. Yes, the trends are flattened declining, but it's not the case across every state. Is that right?

You know, that's true. Alabama, Tennessee had some marked sort of outstanding performance in this last round. I'm sorry, Louisiana. Tennessee and Mississippi, I think, have done sort of particularly well over a longer period. I don't think that it's wrong that states don't have influence on these things. I think they can. They can also fail to have influence that is potential.

Louisiana.

Nat Malkus (06:51.67)
When we talk about, well, you know, these NAEP scores are evidence that the US Department of Education is not working, that is just wrong. The US Department of Education does not run schools. They don't have much control over schools. They're sort of a supplementary factor. They're a big fat target. And look, I got lots of gripes about the US Department of Education. I'm not saying that they are an over-performing institution. I'm not saying they're fantastic. But if you're complaining about NAEP scores and putting the...

the blame on the U.S. Department of Education, you need to kind of do your homework.

Yeah, I think that this, mean, in my opinion, because I've been doing this long time, there was this feeling back around the beginning of No Child Left Behind that if we do this intensive federal intervention and we say that every state has to get every kid to proficient or higher, that somehow that would move NAEP scores. It didn't really move NAEP scores.

Well, know, No Child Left Behind had some positive impacts that are sort of, we were able to show. And there was a good run after 2002 in a lot of places. And if you look at the trajectory of scores in the decade following No Child Left Behind, and I'm not saying this is because of No Child Left Behind, but there's some evidence that that helped.

Look, it's very difficult to sort of pin down, well, why are the numbers moving in the direction they're moving? That's a whole separate game plan from identifying where they're moving. But, you know, I came out with this paper just before the NAIP scores came out to look a little bit at preparing folks for, well, here's how you understand these things. And there's a couple of things to understand that are key to this. I think one of them is the slide that we see

Nat Malkus (08:42.158)
that we saw started somewhere in the early 2010s. So on NAEP, it's sort of 2013, but it depends on the test. I looked at 22 different assessments and subjects, and they were all somewhere in the 2012 to 2015 range. So that suggests that that really was the high point in recent history, and probably ever, on these kinds of achievement tests. And I can go into more about those things, but

really a lot of times people are thinking, well, that was the pandemic, right? No, no, this started before the pandemic. The pandemic was like a wave on top of a wave. I'm not saying it wasn't important, but it's important to not sort of scapegoat all the declines we're seeing for the pandemic, because that is not an accurate portrayal of the timeframe or where we're seeing the effect.

What's your hypothesis? Missouri also topped out around 2019-20, between 2019 and 2013, depending on the test. Why do you think it's also when we topped out on the number of students we have, why do you think that is?

You know, I'm very reticent to point to a particular thing. And for one, it's not one thing, but I do think that on NAEP, we see a couple of things population-wide that are very, very important. We've sort of alluded to them already, but when I looked at all those test scores, common across basically all of them is that since that sort of peak period in the 2018, the early 2018s,

Scores have tended to drop and not always statistically significantly, but always driven by the bottom half of the distribution. So the upper scores are kind of treading water. Sometimes they go up a little bit. Sometimes they go down a little bit. But by comparison, the low performers at the 10th percentile and the 25th percentile, they're dropping like rocks. And that's what's driving our scores down. So if you look beyond the average,

Nat Malkus (10:48.95)
It's our low performers that are really tanking. And we have some international tests. And when you look at the international tests, find, hold on a second, there's achievement gap growth. And then I'm talking absolute achievement gaps, not necessarily race-based achievement gaps or whatever you have, just the absolute gap between the upper scores and the lower scores. The US is number one across the nations that are measured on these tests. So that's another thing.

to be concerned about. And then the last thing that I'm really puzzled is that PIAAC scores recently came out. Folks are probably not that familiar with PIAAC, but I think it's important to note PIAAC is a test of adults, right? And look, there's problems with this test. There's problems with the sample, but it tested adults 16 to 65 in numeracy and literacy. And the top end scores,

are pretty much staying flat. And between 2017 and 2023, the bottom scores on adults, they just absolutely dropped by an unbelievable amount. Now, I'm not quite sure that PIAAC actually captures that as well as we'd like. I know lots of people are concerned about it, but it rhymes so well with all the student test scores that you gotta think, well, our schools can do something about this.

but they might not be the only thing that's going on with these test scores.

Well, except unless those kids, those adults went through our school system as that achievement gap was growing, right? What is causing, I'm just wondering out loud, school systems to either ignore the bottom or to not focus on the bottom. And yet, you know, we know high school graduation rates are about the same. So these kids are getting high school diplomas. And if they're in the bottom 10th, their performance is going down. Essentially, we're letting kids leave with even fewer skills than what we used to let them leave with. Is it?

Susan Pendergrass (12:49.29)
Is it the systems of accountability in place in the states?

You know, there's lots of different arguments and I'm gonna keep not getting on the hook. look, there's lots of things going on here. there's some folks that say, well, we had the standards era that was kind of gutted when ESSA was put in place around the same times that we saw that high points. And then we sort of...

Not letting

Nat Malkus (13:15.596)
backed off of accountability and so scores slid. And it may not be surprising that they slid for the lowest performers because our accountability system may not be doing a ton for those at the top, but it tries to bring up the floor. I think there's something to that. I don't necessarily know that it's powerful enough to explain these things that we're not only seeing nationally. I mean, we see them, I think Chad Alderman did the work and he saw these patterns in...

I don't know, almost all the states. There's something like 48, 49 of 51 states. So I do think there's something there. As far as the adults, yes, there's some influence of the education system, but it's happening at the same time. So it's not really a cascade. And that adult population is 16 to 65. So that is a huge adult population where they're dropping. So I think we also have to consider the things that...

might be having population-wide that might be influencing, particularly folks that are in the lower part of the scoring distribution, phones and screens seem to fit the timeline. They're not perfect, because again, why in the US and not in other nations? But I think that that's a big part of what we're seeing.

Okay, so just to clarify, recapping, we had this No Child Left Behind era and all the states had to put together these massive accountability systems and people said that we were teaching to the test and drill and kill and everyone got mad about it. And then we redid the law, the No Child Left Behind law to the Every Student Succeeds Act. And we gave most states just a waiver and said, don't have do that anymore.

You gotta do something, you don't have to do that. We're not gonna look at your scores and think about your money, relation to the scores. And maybe that did, I I rail all the time on Missouri's, we have an accountability system where 515 out of 520 districts are fully accredited. So we don't really have one, right? Everyone got an A. And then there's a five A minuses, maybe Bs. There's no appetite for letter grades or for really highlighting.

Susan Pendergrass (15:29.038)
some of our lowest performers and some of our lowest performers like St. Louis Public Schools are, you know, single digits rates proficiency, really low performance. Still, that's not even talked about like a crisis. And I think if we do that, if we don't look at it, then it's easier to let it just fall under their radar. I mean, it's just, you know, I guess those kids, if your overall test scores can look okay and a little bit flat, then you don't have to focus. I mean, that was one thing I liked about No Child Left Behind is the subgroup.

in Missouri, we take all subgroups and put them into one. We call them a, I don't know what call them, but we basically equate a hearing impaired child with a Hispanic child, which is all lumped in there. And so we don't really know. And I think that's a loss from the no child left behind era is that we're not looking at.

Yeah, I think that's true. So some of this, I think, can be blamed on the actual policy design, so forth, and sort of the rollback of that, Inessa. But I think it's worth asking, what's behind that rollback? So in the No Child Left Behind era, we did have this really sort of uncommon sort of right-left.

compact, where you had a bunch of governors from across the board saying, we really need to shore up the rigor of what's going on. And it wasn't a particularly polarized time, certainly not by today's standards. So you had a lot of folks singing according to the same tune. And so there were a lot of things that had a lot of policy impact, but they also had all this sort of soft power behind them, where people were actually pushing in the same direction.

I think that coalition has sort of fallen apart, certainly on the national stage and in a lot of states. And so I think it's worth asking, who are the guardians of rigor for our schools? And if you don't have them in place, then your rigor is gonna slide. I think that it's hard to believe that we're seeing these scores go down and that

Nat Malkus (17:41.696)
our schools are doing a good job at being the guardians of rigor. And so I think that what we need to do is take this slide seriously. It is not, I mean, we see some states, Louisiana is a point that suggests, this is not sort of written in the skies. This is something that we can actually influence. And I think we need to seriously get after doing that.

because what we're seeing is low performers whose opportunity to flourish later in life is withering on the.

Yeah. What do you think about the impact, potential impact on say test scores of shifting accountability to parents? Now we have 13 states that are universal school choice states. you expect to it'll be a little tough to see how that shows up in NAEP, but do you expect to see any?

Yeah, I mean, it's a concern. It's a concern. You know, I am a big proponent of school choice in theory. I think oftentimes in practice, the school choice that we actually get over the finish line doesn't always have the design and structure to deliver. I mean, I'm very concerned that in too many places where we achieve an ESA that allows choice for lots of parents that we're doing it on a shoestring budget.

And you know, I mean, if you're giving out four or $5,000 to families, I'm just not sure how well that works. We need to find ways and stay with these school choice programs until we make sure that they are actually functional. If you are a school choice believer, you're convinced, know, we need to poor folks or folks in failing schools more than one option.

Nat Malkus (19:41.248)
and private schools can be part of that, then we need to make sure that the options that we give them aren't just options in name, but that they're real options. And then I think that if we have quality alternatives that people can afford, I think that's great. And until we get there, we have more work to do.

Yeah. Did the Tudor scores come out?

They have and I'll tell you in the days since I have not been able to examine them. So I believe they're out, I haven't examined them yet. Boy, there's a lot of information in NAEP. There's so much to get

The data explorer was basically, you know, paralyzed for the first three days.

Well, you know, they have a great website, theoretically great. It's not super great. But then every time NAIP scores come out, it just gets absolutely overloaded and you're sidelined for three days.

Susan Pendergrass (20:39.182)
Do you know, just out of curiosity, anything about 12th grade scores? Are we ever going to see those again?

Yeah, 12th grade scores are on a four year cycle. So I don't know when the next schedule is for for for grade 12. But the fourth and eighth grade reading are on a two year cycle. And so the grade 12 scores are not necessarily on the same calendar. But I

think we're due for some soon, and I'm really interested to see. This is one of these questions, and I really think that we should be a little bit more concerned with sort of culture-wide things, because what we're not necessarily seeing is an event that happens, some instructional event, and such that our fourth graders scores tank, and then four years later, our eighth grade scores tank, and then four years later, that's sort of like a cohort effect. I'm not.

seeing that kind of thing. Now, we're only gonna get every four-year data for 12th graders, but it will be interesting to see whether it reflects some sort of cohort effect or what it looks to me like is that whatever is doing damage to our students' scores is doing it sort of at the same time across the board.

Hmm. Well, the smartphone thing is intriguing. mean, we just had somebody out talking about getting smartphones out of schools policies, and that is intriguing in terms of if you don't need to learn how to read a map, which you don't, or do math, which you really don't anymore. You know, maybe, maybe we, our brains are just using those areas for other things. I don't know. But well, it's very interesting and I do appreciate it. I worry a lot because

Susan Pendergrass (22:24.014)
The press release from Missouri's Department of Elementary Secondary Education was like, no problem, we are just right in line with the country. And look at us, we exactly matched. And they were reporting on the basic and above scores. So they seemed to just gone through and picked out some things that seemed to be okay to them and put up a small press release and let it go. Certainly no one's focusing on it. I feel like the last round, several governors or, know.

superintendents of schools at the state level were like, no, our NAEP scores are terrible. We gotta do something. I'm not hearing that this time. I'm not hearing the call to action in the way I have before. I certainly haven't heard it in our state. So I hope the more folks pay attention. One last thing that just came to mind. I do wanna ask you about, did you study the gap between NAEP and state accountability systems? Those gaps are getting bigger.

No, Yes, so what they talk about in terms of this is the honesty gap. So under NoJod Left Behind, right, every state gets to design their own test systems and they set their own levels for what is proficient. And so you're going to get some variation. And if NAEP is the standard you're supposed to meet, which is not set in stone, but is certainly implied because we have this one yard stack, then we can at least measure all these state proficiency rates.

against the NAEP proficiency rate and see how far the difference is. And that's what some folks will call the honesty gap. look, post pandemic, we did have some states that just flat out lowered the score. mean, talk about dropping expectations. Some states, they didn't say, well, we're reformulating our tests and so we have a new score. They said, well, we have the same test, but now you just have to get eight.

less questions right to be proficient. And then there's lots of things that I think are going on that are less deliberate or less overt than that. But again, this kind of comes back to who's going to defend the rigor. And I think that it's important for us to take that really seriously. one other thing, Susan, the pandemic gave you a lot of excuses as a school leader to say, yeah, now we need to march back because this is

Nat Malkus (24:44.562)
You know, this was something that was, you know, an exogenous shock, something that came from the outside that really harmed students, and now it's time to build back. Well, now you're sort of like, well, wow, we were building back. I guess we need to build back more. Well, we do need to build back more, but it's not just the pandemic, because if it were, we would have seen some improvement and we wouldn't have seen a greater slide in reading. So I think that

is tremendously important for us to focus in on and fight against.

Well, when the 26 scores come out or maybe the 12th grade scores, we'd love it if you came back and explain those to us. Awesome. Thank you so much, Nat. I appreciate it.

Happy to!