Pam  0:00  
Hey, fellow mathers! Welcome to the podcast where Math is Figure-Out-Able. I'm Pam Harris, a former mimicker turned mather.

Kim  0:08  
And I'm Kim Montague, a reasoner who now knows how to share her thinking with others. At Math is Figure-Out-Able, we are on a mission to improve math teaching.

Pam  0:16  
We know

that algorithms are super cool historic achievements, but they're terrible teaching tools because mimicking step-by-step procedures actually traps students into using less sophisticated reasoning than the problems are intended to develop. 

Kim  0:29  
In

this podcast, we help you teach mathing, building relationships with your students, and grappling with mathematical relationships. 

Pam  0:35  
We

invite you to join us to make math more figure-out-able. Hey, you know, I really like that part about building relationships with your students.

Kim  0:43  
Mmhm.

Pam  0:43  
And mathematical relationships. That's nice. Yeah.

Kim  0:46  
I

like it. 

Pam  0:47  
Hey, Kim.

Kim  0:48  
Hi. So...

Pam  0:51  
Yes? 

Kim  0:52  
I say this all the time because I do like chatting with you, and we have some fun conversations. But this episode is like a hot topic for me.

Pam  1:02  
Whoo! 

Kim  1:02  
I get excited talking about it. So...

Pam  1:03  
Are you saying you're hot

today? 

Kim  1:05  
I just... I have strong feelings about some things that I won't let go, and other things I'm like, "Well, you know, whatever. It's, you know. That's not a big deal." But when I feel strongly about something.

Pam  1:17  
I mean. I don't know.

Kim  1:18  
I get passionate.

Pam  1:19  
Do you and I feel strongly about anything?

Kim  1:21  
Yeah, mmhm.

Pam  1:22  
Yeah, haha. 

Kim  1:23  
So, recently... Well, like a month or two ago. You were somewhere, and then you came back and said, "Hey, I want to talk to you about something. And so, why don't you tell listeners where you were, and what happened, and...

Pam  1:37  
You betcha. So, I was in the frigid north of Alberta. It actually wasn't that cold while I was there. And it was beautiful. And wonderful people. I had such a good time. I had dinner with a team Edmonton Public Schools. Is that what they call? Edmonton? I think so. Edmonton Public. Yeah. A group of math consultants. Went out to dinner with them. And we were having a wonderful chat. Very, very thoughtful educators. We just had a great time. And one of them asked a super important question that I've been kind of grappling with. We've talked a little bit about it on the team, but, man, I hadn't really heard what you thought about it. So, one of the things that this particular person said is they want to be able to show that what they're doing is working, and so kind of flat out said, "How do we assess reasoning? Do you have some sort of screener or assessment tool that could help all of us kind of prove that students' reasoning is getting more sophisticated and the professional learning that we're doing is working." And I really appreciate the need for leaders. It's super important to show that what we're doing is working, so that then we continue. And if it's not working, then we don't. We pivot. Like, what we don't want to do is just start implementing some... You know, pick your thing. Some sort of bandwagon thing. And then, I don't know, kind of get a feel. You know, a couple people complain, so then we just toss it. And, oh, we better do something completely new. Or, well, we did this thing last year, and whether it worked or not, now we got to do a new thing this year. Can't tell you the number of times that I work with leaders who are kind of on this bandwagon approach to education, where it's like, "Well, you know, we did this kind of..." I don't even know what to call it. "...bandwagon approach. This new thing. And we had the person come in. They talked about it. We've done that. We're done with that. So, now, we've got to do a new thing. What's the new thing this year? Who are we going to bring in? What's the new book? What's the new thing?" The new... I don't know. Can I just say "thing"? The new thing that (unclear). 

Kim  3:36  
Lots of things.

Pam  3:37  
Yeah, all the things. And we think it's important, and so I honor the fact that this leader was being very thoughtful to say, you know like, "We want to have some data to say this is working, and so we're going to keep going. Or this isn't working, and so we're going to pivot."

Kim  3:49  
Yeah, so smart. 

Pam  3:50  
Yeah, it's super important. So, I came back and I said, "Hey, like, Kim, what do you think about... You know like, what do you think about screeners? And (unclear).

Kim  4:00  
I think I might have laughed and said, "Oh, I have all the words for this." And we were about to have a coaching session for our Journey 

Pam and Kim  4:08  
coaching group. Mmhm. 

Kim  4:09  
Yeah. And you said, "Do you mind? Like, I don't know what you think about this. Do you mind if we just open it up and we just talk about it?" And, you know, we had some really great questions that month, and so, you know, we really didn't get to it for the last bit, and we just kind of ran out of time. But so we thought we would have the open conversation here. 

Pam  4:24  
Bam, alright.

Kim  4:26  
I told them at, you know, the last few minutes that we had that if there were a continuum that you had to place yourself on for screener or no screener. Like a universal screener for, you know, putting kids wherever. Then I am on the far side of no screener. 

Pam  4:44  
And I raised my eyebrows. 

Kim  4:45  
You did. 

Pam  4:46  
Yep.

Kim  4:46  
And I think really because there's just way too many issues that... We can talk about some of them, but I think there's just way too many issues with the screeners that we use.

Pam  4:58  
So, let's get on the same page about screeners.

Kim  5:02  
Okay, alright. 

Pam  5:02  
So, you just said screener.

Kim  5:03  
Mmhm. 

Pam  5:03  
What is a screener? What do you mean when you say screener, a universal screener, a screener? What does that mean to you?

Kim  5:10  
So, let me describe my perspective because I used in the classroom several different kinds. And I know there are many more now since I've left the classroom. But for screener, for me when I was teaching, was, "Hey, I got handed this thing." We used phonic screeners that had words on a page. There were hundreds of words. And I would call a student back, and they'd read words on the page, and I would mark, you know, the ones I did or didn't know, or...

Pam  5:34  
Like a story

or just random words?

Kim  5:36  
Just random

words. 

We also... 

Pam  5:37  
Oh, fun.

Kim  5:37  
Yeah, yeah, yeah. Some of them were incredibly long if they were well read or, you know, could word call very well. I also used another reading screener that they would read stories. We'd have conversation. Basically I was to judge what level they were. The math screeners....

Pam  5:55  
Hang on. Sorry, sorry. You said they were incredibly long. Something. Is that because if a kid couldn't read, then you stopped? 

Kim  6:02  
Yes. Okay, so... 

Pam  6:03  
And if they could keep reading, then you had to do them all. 

Kim  6:06  
Yeah,

I figured math people didn't care too much about those. But (unclear).

Pam  6:09  
I'm just trying to figure out what you're talking about here because yeah. 

Kim  6:11  
You didn't do phonic screeners.

Pam  6:12  
No. 

Kim  6:12  
So, phonic screener is a page, and the words were broken up into like vowel, consonant, or...

Pam  6:20  
Well, don't get too deep into it.

Kim  6:21  
Well, I mean, so there's sections. There's sections. And you start at the top, and it'd be like, Cat, Hat, Bat at the top. And then at the end of the page were like three and four syllable words. And you would just start at the top and they'd read, read, read, read. Like, word call all the way down. And you had to have these specific markings that you would do. So, it took some time if your kid, you know, could say a lot of words. 

Pam  6:41  
And you had to do this individually with every kid.

Kim  6:43  
Right. And so, I'll skip the one where they read a story, but that took even longer.

Pam  6:48  
Okay, alright. 

Kim  6:48  
And so, the math screeners, there were a couple of different kinds that I encountered. The ones that kind of burned their way into my brain for the longest were you could do whole class, but it was, there was one that was all about computation. And so, there were like just problems on a page. And so, you would give them seven minutes, or eight, or whatever. And everybody could do it at the same

time.

Pam  7:12  
Meaning, instead of an individual interview.

Kim  7:15  
Right.

Pam  7:15  
You're handing them something, and they're like, "Go." But they're doing it individually. They're not doing it as a class.

Kim  7:20  
It's timed, yeah. I give the instructions, and then time, then you do the thing. But then I scored them, obviously, individually. And the scoring for one in particular was based on digits in the problem. And then there was another one...

Pam  7:34  
Wait, wait, wait. 

Kim  7:35  
We'll talk about that later.

Pam  7:36  
Oh, sorry, sorry, okay. Sorry. Trying

to keep up. 

Kim  7:40  
So, then, there was another kind that was concept based. And so, it wasn't just computation problems. There were more about different ideas. So, it might have something geometry or algebraic reasoning or whatever, but in any case, there were two different kinds of tests. And so, we'd give them to the class. Everybody would take it. And then we'd score their information, and then we put the scores into whatever the platform or whatever the system was in order to rank kids. Either like Red, Yellow, Green. Or 1, 2, 3. Or whatever. So, it was a way to give out some sort of test in order to basically rate kids, to judge kids.

Pam  8:20  
Sort kids.

Kim  8:20  
Mmhm. To sort them. 

Pam  8:21  
Okay.

Kim  8:21  
Yeah. So, that's... When you say "universal screener," or "screener", that is one way. There are other screeners that people might have in mind where it's interview based. And so you would call a kid over. There's a set amount of things that you would ask. There are predetermined problems. You do the interview. You score the interview. You calculate the data. And then it's a ranking or it's a sorting system. That's what I mean when we're talking about screeners today.

Pam  8:50  
Okay. And so, maybe one reason why I have interestingly... Hmm. How do I even say what I'm thinking? I didn't have to do that. So, as a high school teacher.

Kim  9:02  
Ah, yeah. 

Pam  9:03  
As a high school teacher, I never interviewed students. That wasn't a thing. I often gave them tests and quizzes. There were... I don't... I could give them like pre-tests and stuff. I didn't because I didn't think they were very useful. But I, especially at one school, I had very specific common assessments that my students had to take. And I didn't write them. So, there were some things like that. But not what you're. Okay, so you're on the continuum of screener to no screener. You're on the side of no screener.

Kim  9:34  
Right. 

Pam  9:34  
Let's talk why.

Kim  9:35  
Yeah.

Okay, so there's at least a handful of problems. One of the major problems is I'm going to use the word "validity". I know that you're going to poke on my use of the word.

Pam  9:46  
Okay. 

Kim  9:47  
But I don't buy that these tests are valid judges of students. For a couple of ways, couple of reasons. Part of it is that the value that the students and teachers see in them affects how the students do on those screeners. I have a personal student who lives in my home who feels completely over tested, can now, as he's gotten older, say all the words about being over tested, and he just doesn't give a rip, to be honest with you, and doesn't really put in a lot of effort. He's got some teachers or has had some teachers in his lifetime that also don't really find them all that valuable. So, because they don't value the test, what they go into, their mindset, their effort, their how they facilitate it.

Pam  10:41  
Willingness. 

Kim  10:41  
Yeah.

Pam  10:42  
Yeah. 

Kim  10:42  
Doesn't paint an accurate picture of academic knowledge. So, I really struggle with kind of validity and value wrapped up together because so much of these kinds of screeners depends on the test taker following what is designed for that test and the facilitator following the expectations, and the rules, and the readings, and whatever that is designed for that test, so that you can create a fair system of ranking kids. And all across schools, there are teachers who in their classroom are facilitating these kinds of things differently than the classroom next door in how long they give for them. Do you let the kid finish the problem they're on if the timer goes off? Do you hard stop? Do you let them have side paper? Do you make them do it without extra paper? There are so many little, tiny give and takes that happen that by the nature of we have humans facilitating these and all kinds of like different teachers in every classroom, it cannot be as valid as people think

that they are. 

Pam  12:04  
And you just mentioned a kid who might like have an attitude of, you know, I don't care, and so (unclear).

Kim  12:10  
Yeah. 

Pam  12:11  
But you also might have a kid who's, you know, got stuff going on. Kids come with baggage. 

Kim  12:17  
Absolutely.

Pam  12:18  
And maybe it's even just on the day that they take the screener. It's not going to be an accurate representation of what they know and can do because of where they are sort of emotionally.

Kim  12:30  
Well,

yeah. So, two of the tests that I gave were timed. One was a seven minute or eight minute. I can't remember. The other was a 20 minute. And if you are the kind of kid who feels pressure under time, there's no way to factor that anxiousness, that nervousness, that pressure that affects some kids, that don't affect other kids. There's no way to factor for that. 

Pam  12:53  
Yeah.

Kim  12:54  
Yeah, so there's...

Pam  12:56  
So, now we have a societal measure that says if you can handle the pressure, then we're going to be able to measure what you actually know and can do in mathematics. If you can't handle the pressure, then instantly what we think you know and can do in mathematics takes a hit.

Kim  13:12  
Mmhm.

Pam  13:12  
Like we just really don't know because your brain was flooded with chemicals, and so you're actually not being able to show us accurately what you actually can do. So, there is something to... You know, there's a colleague I know that runs a whole school. A whole school. That has a program, whatever, to help kids learn to deal with time pressure under tests.

Kim  13:34  
Mmhm. 

Pam  13:35  
I think that's interesting. I think I get the the reason that you would do that because we've created these situations where if you're going to do well, then you've got to be able to handle that time pressure.

Kim  13:50  
Mmhm. 

Pam  13:50  
So, I kind of get that. I question the premise itself that putting kids under time pressure is going to give us an accurate representation of what all kids can do. I think it does give us an accurate representation of what kids can do who can handle time pressure.

Kim  14:07  
Mmhm.

Pam  14:07  
But I don't know that it does other than that. Is that part of your 

beef? 

Kim  14:10  
Yeah, I just actually saw a principal, a woman who was a principal, at an elementary school, and we were talking about screeners, and she was telling me that what she did in her school for a portion of time is that they... The whole school. They would shut the school down. And so, it was, you know, say, Monday at 8:00 or whatever. The teachers would pass out a portion of their screener. And then they would read over the announcements on the intercom. They would set the timer. And as an attempt to make things... 

Pam  14:44  
Standardized.

Kim  14:45  
Standardized, they would do that as a school wide thing. Mmhm. Another thing that people sometimes do to standardize it is you have the same screener, the same person screen for an entire grade level. So, you know, there are some ways people are attempting to make the results...

Pam  15:07  
(unclear) if

you were doing an interview, then ideally you'd have the same person interview everybody?

Kim  15:11  
Mmhm, yeah.

Pam  15:12  
Okay, okay. 

Kim  15:12  
One of the other ladies at my dinner was a preschool screener. Which don't get me started on preschool screening. But she would go in. She was a former teacher who would go into a school that would hire her, and she would facilitate in order to take, you know, away from the teacher's time having to do it. But in order to make it more standardized, she was the screener for all of the kids, so that it was more consistent. So, I think people are trying to address.

Pam  15:42  
At least one of the inconsistencies

Kim  15:43  
One of the inconsistencies. 

Pam  15:44  
Yeah, yeah. 

Kim  15:45  
Yeah. 

Pam  15:46  
That's interesting.

Kim  15:46  
So,

so the value. And listen, here's the thing about my kid, right? He doesn't care about... 

Pam  15:52  
He's awesome, by the way. 

Kim  15:54  
He's feisty. I don't know he gets it. So, he is the kind of kid who part of the thing is he's figured it out. And so, he will do these screeners. He doesn't really care about them because of the other issues I have with them, and so he won't really play the game. But then he shows up for the one state testing that he feels like he's going to get judged on at the end.

Pam  16:19  
The

one that's going to matter. He's like, "This one's going to matter, so I'm showing up for this one." 

Kim  16:24  
Yeah,

yeah. And so, it gives a false impression of him. And he's not so concerned about time. (unclear).

Pam  16:31  
Well, wait. The screeners. The screeners give a false impression. 

Kim  16:35  
Right. 

Pam  16:35  
But the one he shows up for, is it more accurate?

Kim  16:38  
Yeah. Yeah, yeah.

Pam  16:39  
Yeah, yeah, yeah. Yeah.

Kim  16:40  
Yeah, so then we have all kinds of issues. So, you know, there's a lot of kids, and we create. We have these situations where we're screening kids and we think we know something about them, and then we may not know what we think we know. It might be inaccurate. Anyway. So, value that people see in them, the way the tests are given are kind of hand in hand, but that is one issue that I have. 

Pam  17:06  
Okay.

Kim  17:07  
(unclear) on to the next one?

Pam  17:08  
Yes, I think so. Though, I don't want to forget that at some point you said something about ranking kids. So, I don't know where that comes in our conversation. 

Kim  17:16  
Oh, well, let's go there next.

Pam  17:18  
Okay. Because yeah, tell me. Like, the way you said kind of could be interpreted that people give screeners in order to rank kids. Would you agree that sometimes get people give screeners... Like, we like the guys at Forefront, their Universal Screener for Number Sense.

Kim  17:33  
Mmhm. 

Pam  17:33  
I don't know that they're about ranking kids. I think they're much more about finding out what kids know, so they can build on

that knowledge. 

Kim  17:39  
Yeah. Well, so the next two things I want to talk about are what...

Pam  17:43  
You have a list. Alright.

Kim  17:44  
I do. I do. 

Pam  17:45  
Bring

it on. 

Kim  17:46  
So, you tell me which one you're talking about. One is what we screen and why we screen. 

Pam  17:52  
Let's go...

Kim  17:52  
Tell me where you want to go next.

Pam  17:53  
Like, in the order you gave it. What we screen. (unclear).

Kim  17:56  
Okay, so

what we screen speaks to what we value. And sometimes as teachers...  You know, I know there's some teachers right now who are screaming, "But I get told too!" And you're spot on. Sometimes we get handed a screener and say, "This is what we do at our school," and so you have to. And that was the case where I was teaching. It was what we did. And what we screened had nothing to do necessarily with...

Pam  18:24  
You just took a huge

deep breath. I can tell. How did you feel about that, Kim?

Kim  18:29  
Listen, I love my principles, and it was a district mandate. And so, you know, it was what we did. It was we need to know about kids, and so we screened them. But the problem is what we screened on this company made screener didn't reflect necessarily what was happening in my classroom. We were judging kids based on the experiences that were not happening for them. My kids weren't getting timed things, and so to screen them on a time situation felt unfair to me. And the answer could be, "Well, give them timed stuff." No, I'm not bending. So, what one of the screeners that we used was all algorithm based and, like I mentioned earlier, the scoring of it was based on how many digits you got correct. So, let's say you were given a problem like 49 plus 28 and the answer is 87. If you got what did I just say? It's 77. 49 and 28. Good heavens. So, if you got one of the sevens correct, you got a point. If you got both sevens correct, you got two points. And it didn't matter which seven you got correct. So, you could get the ones place digit correct, and you could say 87, like I said, and get half credit. But if you got 88, or sorry, 78, or whatever, something close by, you got no points. So...

Pam  19:57  
You

could have gotten a closer answer.

Kim  19:59  
Correct.

Pam  19:59  
That's really better in your lifetime.

Kim  20:01  
Right.

Pam  20:01  
But if neither digit was correct, you got no credit.

Kim  20:04  
Right. 

Pam  20:05  
Interesting.

Kim  20:06  
It

also didn't matter which. So, if it was a three or four digit number, it didn't matter which digit you got correct. So, you could get the hundreds place, but no other digit correct, and you got some points.

Pam  20:17  
Or even worse. The ones digit correct.

Kim  20:20  
Right.

Pam  20:20  
And the hundreds was way off.

Kim  20:22  
Mmhm. 

Pam  20:22  
So, the answer kid could be completely way off, but you could still get some credit. But a kid could get really close and get no credit if the digits were all a bit

off. 

Kim  20:31  
So, here's where I, you know, open and honest, say yeah, I didn't necessarily care to follow the scoring procedure

because...

Pam  20:40  
Are you telling us right now you didn't? 

Kim  20:42  
One hundred percent there

were times where I was like, "That's so dumb." So, anyway, the what we screen mattered. It was created by somebody who wasn't looking at our standards. It was created by somebody who just was making a national test, and in Texas, we did not necessarily have the same standards that were made on this particular test.

Pam  21:09  
Oh, so... 

Kim  21:09  
How weird is that? 

Pam  21:10  
...not only was it sort of algorithm based, but it wasn't even testing the right algorithms because it was not for the particular state? 

Kim  21:17  
Well,

so the algorithms probably would have been, but if it the portion of the test that was on concepts, there were plenty of concepts that were not in, say, third grade, when my kids were taking a third grade one. Which is so... 

Pam  21:30  
Because the standards were different.

Kim  21:31  
Because it was a universal screener.

Pam and Kim  21:32  
Yeah.

Pam  21:33  
Huh. 

Kim  21:34  
So, what we screen, I think, matters a lot. And the person who knows that is the person who's teaching. 

Pam  21:43  
So,

Kim, would would you say that was similar to when my students in high school had to take this common assessment. And I didn't write it. I didn't have any chance to have any input in writing it. And they would have to do this super esoteric factoring of polynomials that didn't matter to me because I was much busier thinking about how polynomials were created from adding terms that were other polynomials, and what those graphs look like, and how that affected their end behavior and the short run behavior. And like there was a lot of kind of like... I was going to say math. Math that I cared about. That I spent less time on these esoteric factoring by grouping with crazy different exponents and stuff instead of doing that, and so when that  question would show up on that test, I just would be like whatever. I don't care. I don't care that they're going to miss it because it's not important to me. Is that similar? 

Kim  22:36  
Yeah.

Pam  22:37  
Okay. 

Kim  22:38  
Yeah.

Pam  22:38  
Okay.

Kim  22:39  
There's also a challenge with what we screen because we don't have control over, like you said, what's on it or how many times something's on it. So, let's say we've got a student who is a fantastic mathematician who has a difficulty in one area. If that one area is on the test multiple times, does that make them a lesser mather than somebody who's only got that one type of question down?

Pam  23:05  
Hmm. That's

interesting. You just made me shudder a little bit because I could kind of get away with what I was just saying because there was usually only one or two of those stupid esoteric questions, and the rest of them were good enough that then I wanted my students to do well on the rest of it. And so, you know, since I taught well, they did. But yeah, if the test was all that, that would have made it harder. 

Kim  23:29  
Yeah.

Part of the other problem that I have is, you know, a lot of times we screen. We would screen beginning of the year, middle of the year, end of the year. I mean, at the time, that felt like a lot to me. But now comparatively, I mean, kids are always testing. And often at the beginning of the year, we would test on... Like, third grade or fourth grade, whatever I was teaching. We would cover. We would test material that was going to be taught in that year. How wild? We would we would screen kids based on things that they were going to use that year. So...

Pam  24:05  
Is that similar to when my kid would come home and say, "Today, we just took last year's release test?" So, like when he was starting say fifth grade, fourth grade, we took the end of the year fourth grade test at the beginning of fourth grade. And he would say, "I aced it. I got 100. Why am I even taking math this year?" Is that kind of a similar? Like, it's not even.

Well, I love that for your kid, but yeah.

Well, no.

My point is, it was useless. Like, either the kids bombed it, and then felt horrible about themselves, and now they have this like, "Oh, goody. This year is going to be horrible." Or they ace it, and then they're like, "What am I going to do the rest of this year? I know all this stuff."

Kim  24:45  
Yeah, I think that's a hard place for a kid to be, for sure. 

Pam  24:49  
Yeah. So, like how does it help? 

Kim  24:51  
Yeah. 

Pam  24:52  
I don't...

Kim  24:53  
Well,

and so what I don't love about the situation that I was in was that we were... You know, I think it's safe to say that in those situations, most of the kids did not do well, right? It's end of the year stuff.

Pam  25:03  
Yeah. 

Kim  25:03  
But the bigger issue is, then it's like sets up in your mind like which kids might do well this year because they got a third of it already. Or which kids, oh, am I really going to have to like work with a ton because they didn't get any of them? It's such a... 

Pam  25:20  
Well,

not only in your mind, but in the kids minds, right? Like, everybody's minds are already kind of preset stuff. Yay. 

Kim  25:26  
Yeah. Yeah.

Pam  25:26  
It's great we start the year. Yeah. Kim, it feels like we have a lot to talk about. I don't think we got to your second why we screen? 

Kim  25:34  
Yeah. 

Pam  25:35  
Ya'll, what do you guys think about universal screeners? What do you think about the idea of like why do we give them? What are we screening? What kind of control do you have over it? How often do you have to screen? And how about well let's have another episode and talk more about why we screen. And then, Kim, you're right. I am going to poke a little bit on the word "valid".

Kim  25:54  
Mmhm. 

Pam  25:55  
And I'm also going to bring up the word "reliable".

Kim  25:57  
Mmhm. 

Pam  25:58  
And yeah, let's...

Kim  25:59  
And maybe what we would actually suggest? 

Pam  26:02  
Yeah.

What can we do? Because, again, we started this episode with these wonderful, very thoughtful educators saying we need some data to prove that we're making... This is working. We're making some ground. Yeah. Alright, Kim, let's do it. Ya'll, thank you for tuning in and teaching more and more real math. To find out more about the Math is Figure-Out-Able movement, visit mathisfigureoutable.com. And keep spreading the word that Math is Figure-Out-Able!