Chris Colley 0:06

Okay, here we are. Another episode of Shift Ed Podcast coming to you. Um, as school is begun now, we're past that oh my god, it's begun phase into this is really happening. Um I have Tom and Natalie from the Canadian Assessment Center coming in from BC. Um, and they are experts in assessment, all things assessment. Um, we were just talking about Tom coming in visiting us in October. Um, so we will get glean more expertise on assessment here in Quebec. I want to thank you two for carving a small amount of time out to come and share some of your thoughts. And uh like I said, I'm really looking forward to the workshop coming in October. Yeah. Thanks for the invitation. We're happy to be here.

SPEAKER_00 0:52

Yeah, absolutely.

Chris Colley 0:54

So we'll start big and that's we'll whittle down a little bit. I mean, assessment has been part of the educational practice since I mean, I I found it like centuries ago, a part of always this assessment, assessment, assessment. I guess my first question is, and I'll throw this to you, Tom. What is the purpose of assessment?

SPEAKER_01 1:16

I know it's a big well, I mean, yeah, it's it it's it's it's interesting, you know, because I would say that that is both a simple question and a complex question. The sim the simple part of it is that assessment just means to gather information or evidence about student learning. So anytime that we assess, we are simply taking inventory on where a student is, you know, within their learning journey. And of course, we can start getting a little bit more granular around the formative purpose versus the summative purpose. But ultimately, the purpose of assessment is to gather information about student learning. And primarily the purpose of assessment is to drive learning forward. And of course, there are times where we use assessment evidence to mark, to grade, to report, to do all of the things that we do as far as verifying the degree to which a student has met the outcomes. But most of the reason we gather evidence disproportionately should be to figure out what's strong, what needs strengthening, and what are the next steps for the learner. So I would submit to you that that the primary purpose of assessment is to drive learning forward. Interesting.

Chris Colley 2:17

And Natalie, we here in Quebec operate on a competency based, right? We have typically three competencies per subject. You know, and we're supposed to measure, as Tom said, the growth of, you know, am I getting closer to the objective of being competent in whatever that competency is? Things kind of shifted a little bit because competency is so hard to understand for parents in particular. Yeah. And the next levels up, you know, like what does it mean a five? You know, what's a four? What am I missing out on? Can you kind of fill in that blank a little bit as to what is competency and why don't we uh adhere to that more? Because it does seem like more of a logical human way to evaluate somebody based on if they're competent or not, rather than, you know, if they got the right answers or not.

SPEAKER_00 3:10

Yeah, exactly. Well, I think it would be really helpful for the audience if we differentiated, because when we say competency, depending on which province you're listening from, it can mean many different things. Like, for instance, here in BC, when we talk about the competencies, we're talking about those core learner attributes that exist across all the subject areas. So like collaboration, critical thinking, creative thinking, et cetera. And in the instance you just used, it's like competency-based, which some provinces would say is proficiency-based. The idea of using levels to qualitatively measure learning. So I just want to make sure we're on the same page. Are we talking about using levels to qualitatively measure learning?

SPEAKER_03 3:52

Yes.

SPEAKER_00 3:54

Okay. Yeah. So, well, at the core of that, there's a lot of things in the assessment space that have a robust body of research behind them. It's probably one of the most researched areas in all of education. But if there was one thing in particular that has the most depth behind it as to why we do this, it is using fewer, more clearly distinguishable levels to communicate and measure learning. And the one of the first studies I can think of off the top of my head that goes back all the way to 1913, like we're talking a century's worth of studies. It was Starch and Elliot, I believe, are the two last names, but they were looking at when you take a group of English teachers, that was the initial study, and they gave them sections, samples of writing. They said, okay, here's the writing. Use all your professional judgment using the hundred-point scale, which we would traditionally call, you know, the percentage scale. Where do you think these samples of writing are at? And there was a vast spread of scores that emerged because it's a very discrete scale. There's a lot of potential levels to land on. Now, as soon as they shrunk the scale and they described what each of those levels were, which is what we're doing with a proficiency-based or competency-based scale, the margin of error was almost nothing. Like it was, we were right aligned. I have done this as a coach. I was working with a team, actually in the States. It was in Michigan, large high school English team, 25 teachers. We came in, they all brought samples of student work. They had a rubric, this huge, you know, analytic rubric with all the boxes across it that they were working on. And they were fighting over this because they were trying to quantify it. They were trying to add up all the boxes and get a number and get a score. And their scores ended up being like wide. And then it's, of course, Brenda, you don't take writing seriously because all you care about is grammar. No, Dolores, you only care about the soft stuff like voice, right? And everyone's yelling at each other. So I just said, everyone stop. Tell me, is this writing mostly exemplary, proficient, developing, or novice? So I took all of the numbers they had spread it across and I said, just tell me, is it one of these four levels? They were all on the same page. We we closed that margin of error. So it is that is why we ultimately do it. It's to make school a more fair and equitable experience for students so that their grade isn't dependent on who their teacher is.

Chris Colley 6:21

Right, right. That makes a lot of sense. And Tom, like if we look at modern day assessment that's going on, like are there things that we just still have these misconceptions about about assessment, like that we're kind of not hitting the mark properly? Like, can you maybe like explore a couple of maybe those misconceptions that just seem to be pervasive and hard to um you know solve, I guess.

SPEAKER_01 6:50

Yeah, there there's there's definitely a let me just preface my comments by saying there's a lot we're getting right, and there's a lot that we've gotten right over the last number of decades. And I know that sort of sounds like the idea. I'm gonna say a couple of positive things before I start grilling assessment on the on the back end of it. But it is true that the evolution, I mean, I've been in education for 35 years, this is my 36th year. I have seen firsthand the evolution of assessment from 1991 until 2026. So there's a lot we're getting right, but there are a couple of things that immediately come to mind when I think of what we're still getting wrong. Some of it relates to what Natalie was talking about, which is the percentage scale. This is something that has almost a cult-like following across Canada, where we just can't even imagine scoring without a percentage-based grade. And yet the assertion alone that there are 101 distinct levels of performance, if you include the zero, is kind of absurd. And it's an impossible research that Natalie was just quoting really speaks to not a critique of teachers. It speaks to the impossible ask of trying to ask teachers to distinguish between all of those levels. But the idea that, you know, there's there's quality and then there's counting, right? So we can count right and wrong. Another example that Natalie and I often use in our workshops and trainings is we'll say, imagine two students score 15 out of 20 on an assessment. Now, on the surface, it looks like both of them are 75%. And yet you could have one student who had no idea how to answer five questions. So they got 15 out of 20, but they left five questions blank. Whereas another student answered all 20 questions, knew exactly what to do in all 20 problems, made five simple errors, like four times three becomes seven, but they carried the math through. And even though the answer was wrong, the thinking was right. It's clear that those two students are not in the same place as learners. And yet across this country, in every province, people would say, well, they're both a 75%. So that shows us the imprecision, the margin of error that Natalie was talking about, plus this example illustrates how um imprecise, if you will, the percentage system is, even though the percentage symbol makes us feel very scientific and very precise. Never mind the decimal places. So that's one of them is this idea that, and I just want to speak to what a lot of people use as kind of the reasonslash excuse as to why we can't reconsider the percentages, because they'll say things like, well, the universities want this. Our universities across Canada have students from all over the world. Not every country uses percentages. Uh, not every student in every school who's applying to our Canadian university, some of them are in the IB program. The IB grades on seven levels. You know, you have advanced placement in some Canadian schools and advanced placement scores on five levels. So this is not a foreign concept to universities. And the idea that our, especially our public universities would not accept students because we were not marking on a percentage scale is just absurd because they're publicly funded. So I mean it's a bit of fear-mongering that goes on. That's one of them. The other place, and I know hang in there with me, listeners. I'm I'm I'll I'll try to be succinct. Because one of my true talents is making a short answer long. But the second, the second way is we we still have, and I believe there's two sides of this coin. One is we've made a lot of progress where this is concerned, and there are still too many classrooms across this country where grades and marks are weaponized against kids to coerce behavioral obedience, that we use grades as kind of a threat or even a promise for behavioral obedience or disobedience, right? So if you are a nice kid who tries hard and you and you work, you have that sort of you work hard and all of those things, we're gonna give you some extra, some bonus. And if you are disobedient, we're gonna dock your score. It really comes down to this, and this is one very simple question that we have to ask ourselves, which is either we want to accurately report learning or we don't. And if if we don't, then you can do whatever you want. Like put whatever you want in your markbook, like have at it, funnel everything you want in there, hair color, shoe size, height, put it all in the markbook. You can manipulate the grade how you want to. But if accurately reporting learning matters to us, and of course, 99.9% of the teachers listening or educators would say, yeah, it does matter. Well, then there are some fundamental rules we'd have to follow in order to ensure that accuracy. And that is something we're going to talk about in the upcoming workshop is how do we ensure that what we report accurately reflects the learning. There are other places where we have a ways to go, but I would say that the percentages and the weaponizing of marks and grades against kids in terms of behavioral disobedience is probably the two places where I would say that there's still issues that we have to have to resolve. Great, great.

Chris Colley 11:32

And getting assessment to where it, you know, it fits within all of the different learners. Every, you know, classrooms are full of different learners, and we always try to kind of like you know, funnel them down into the same, you know, assessment. So uh a question for you is how do we bring more uh humanity back into assessment where we look at the learning and the understanding rather than if a student is really good at memorizing or not? Like how and I mean I'm kind of like pit pointing at formative assessment versus kind of more, you know, summative assessment where we look at process rather than an end game. Um like what where can we start to open those doors so that we start understanding learning and understanding rather than if if I know how to complete a test or not?

SPEAKER_00 12:28

Totally. Something Tom and I explore in depth in our book, Rehumanizing Assessment, is this concept of storytelling as a really great vehicle for assessment. And there's really two stories that we're always telling in the school space. The story of what I'm learning, which we're all very accustomed to. That's where the test usually comes into play. It's I have these specific learning outcomes, I need to get evidence of them to assess that the student has achieved these learning outcomes. Cool. But there's a critically important story right alongside, which is the story of how I'm learning, which is what you're speaking to with process. And now this isn't just some nebulous, like, oh, you know, everyone can only describe what they're learning. Sure, yeah. For for instance, there is a lot of variability, but when we're looking for specific things, what we're actually looking for is called self-regulated learning. And this is, again, a robust body of research that we can draw from that there are very discrete, observable things that we can look for. Like how do they how do they regulate emotion? Like that's a big part of learning. How do they manage their environment? How do they manage their relationships? What are their help-seeking behaviors? I could name tons of things, but there's lots of different literature out there. One of one model that Tom and I often use when we're coaching teachers in this area is it's really simple, which is why we like it. It's just a three-step process. And it's really looking at, you know, what am I doing before the learning? So, how am I making sense of it? How am I estimating, you know, what the task is going to require? How am I setting goals for myself? Like that goal setting part is critical. How am I monitoring during the learning? So, how am I course correcting when my goal is not being achieved? How am I not just continuing to do the same thing and hoping different results, which is like the definition of insanity? And then the final piece, which is huge, is reflection. So, how do I reflect on that experience to pull it all together? Those are all teachable, learnable skills that we can gather evidence of. And if we choose to respond to it instructionally, hence the formative purpose, we can get, we can help students improve. And so when we talk about, you know, humanizing, I know people could, I'm putting air quotes around it if because they can't see me. A lot of people think we're being fluffy or cute or like, oh, it's it's a nice thing to say. But a, I'm gonna give you the most practical, simple way you can do this. Most teachers are familiar with an exit ticket, right? We do them at the end of a lesson. It's like, here's a question, I'm gonna mark if it's right or wrong to see if you learned what you're supposed to today. So very much the story of what I'm learning. If you were trying to better gather the story of how I'm learning, what if your exit ticket was instead focused on conflict? Every great story has a character who encounters a conflict, who endeavors to solve the conflict, has a climax, and then eventually reaches a resolution, right? So that's that's all we're talking about with story. So your exit ticket could say, what is the conflict you you um had in class today? Now there's typically four, right? It could be learner versus learner, could be interpersonal, could be learner versus self. So there was something internally you were struggling with. It could be learner versus environment, maybe that was too loud, maybe it was too cold, whatever. Or it could be, you know, learner versus, I guess you could say like society or like what's going on with the education system. And they could check it off and then they'll tell you what there's a box to say, what is the conflict? Tell me more about it. And then did you solve it? Yes or no? And if you solved it, like how did you do it? And if you didn't, like what do you think you're gonna try differently? So do you see how I'm basically hitting a lot of that monitoring part of it? Now, the magic of formative assessment is I can look at those. I'm not gonna grade them because that like we're not putting a grade on this, it's about responding to it. I'm gonna look for a pattern. If most of my learners had interpersonal conflict that day, and there were a couple students who had really thoughtful ways to handle those conflicts, I'm gonna get them to teach about it at the start of class the next day. We're all gonna pick one strategy to try and we're gonna see if we can adjust that. So we can take the things we're already doing and just adjust it to look at a different story of learning that's more focused on the process and not the product.

Chris Colley 16:27

Absolutely. I love that. You guys are like fire in here. I love it. Tom, please chime in. Yeah.

SPEAKER_01 16:35

Yeah. I think that, you know, one of the powerful things is the the that to pick up on what Natalie was saying is just student, I think one of the most underrated assessment strategies we have is having students think about how they're learning. And really that metacognitive piece and it might be the most important thing as an outcome of any school system is that level of self-awareness that students come out of school with, the ability to understand myself as a learner so that when I head to university, I can advocate for myself. I understand what works for me, what doesn't work for me. So that metacognitive reflection on how I learn, you know, what do I do when I get stuck? You know, what happens when, what helps accelerate learning for me? What are the strategies I can use that allow me to be more prepared for a test? Or what are the, what are the what are the ways I plan to set myself up to do an extended writing piece? So the story of how I'm learning is, I think when Natalie and I first started writing the book, we kind of saw these as equal and and even the story of what I'm learning is maybe something that would be more emphasized. But as we wrote the book and thought more about it, we thought kind of the unsung hero of this is really looking at how students learn and understanding where I am as a learner. So putting myself, as Natalie talked about, kind of the hero's journey or that conflict, right? Put myself in that place and recognize how I can tell the story of my evolution as a learner. What I used to think, now I think this way. And here's what happened to me as a learner as I move through the assessment space. So one of the things that the research is pretty clear on is that uh the self-regulation of learning and assessment, they engage in kind of a reciprocal relationship, which means each serves as an input for the other, right? So if teachers out there are listening saying, Oh, I would love to help my students develop self-regulation strategies, but I don't know how to do it, well, you can use the assessment cycle to do so, right? So the assessment cycle being at its most fundamental is where am I going? Where am I now? How do I close the gap? Those are three questions that Royce Sadler developed all the way back in 1989 to identify what students need to think about if they're going to self-self-assess or self-regulate. So this assessment cycle of where am I going, that aligns perfectly with what many call the forethought phase in self-regulation, right? So setting goals and where am I going and what are my expectancies around outcome? Then the where am I now phase aligns certainly with the performance phase. So how am I learning? Am I staying on task? Am I distracted? What can I do to bring myself back to regulation? And then the reflection phase is how do I close the gap? So now I reflect on was I happy with the results? Did I actually stay focused during the extended period of time? Right. So I can use assessment as an input to teach kids how to be self-regulated learners. But what the research also tells us is that because I'm focused on teaching kids to be self-regulated learners, their assessment results increase. So self-regulated learners, to put it in plain English, self-regulated learners do better. And so if we can use assessment as an input, and then assessment becomes the output when we're teaching kids to be self-regulated. So that whole piece is how we get students invested in their learning as opposed to being passive recipients to the assessment experience.

Chris Colley 20:03

Yeah. I mean, I love hearing this because we often overlook the skill development of becoming a learner. Um, it takes practice, it's not natural to some people. And oftentimes we don't even support the student in that journey where we give them opportunities to figure out what's inside them and how do I mini, you know, how do I improve these things so that once I leave school, I have a capacity to learn. Um, I know how to do it, I know where to look, I know, you know, I have certain um, you know, skills that I've developed. Why can't we focus more on the the internal of the learner rather than this external stuff that always seems to be a reason why we can't look at the inside? You know, like it it just seems so confusing at times. Like I I totally agree with you on that development. I encourage teachers all the time to get those still skills developed, but then there's that flip side of, well, I have this assessment coming down the road. Yeah, I just don't have time to do that. You know, I've got to get through all this stuff. Natalie, can you take a swing at that? And like I know it's again, these are like really hard um topics and and conversations to have, but like, how can we get closer to developing the inside of the learner?

SPEAKER_00 21:24

Tom and I often say that what often looks like resistance is just a lack of clarity and understanding because the irony here, you said the keyword, which is time. That's what people put as the barriers to why this can't happen. And the way to accelerate how fast your students learn is teaching them how to be better learners. So the solution is right there. But I think what's hard is we actually don't do a lot of really great professional development in schools about learning, about how learning works, about how competencies are developed, about how you reflect on learning, about Who you are as a learner. Like I think sometimes teachers are just so caught up with being busy in the business of school. They've gone to school their whole life. They went to university. They got a teaching job right out of university. They're so busy. They're busy, busy, busy. They haven't even taken the time to sit and reflect on who am I as a learner? In what conditions do I become the most creative? Where are my critical thinking skills strong or where are they not as strong? How do I know? What does it mean to be a critical thinker? Not having the time and space, I think, themselves to reflect on all those things, then just becomes this trickle down because then there's the assumption that the only way to accelerate the learning is to get through more lessons. You know, so we're kind of in this snake eating its tail moment where people, I think, are operating off of the knowledge that they have. And that's why it appears that they're resistant to what the rest of us who have spent more time maybe thinking and reading and contemplating, it's so obvious. Like it's painful. It's like you can literally, you can walk yourself right out of the maze. Like you're in this maze, and the door is literally right there. You just got to open it. And when people do, this is why we love the work we do so much. When we get to work with schools and districts and we see those breakthrough moments, people never go back. Like there's a reason the assessment community is a bunch of super nerds. We love this stuff. Like it's not sexy. People aren't like, I want to learn about assessment. But once you have that moment of it gives me everything I've ever wanted as a teacher, you become like evangelical about it.

Chris Colley 23:33

Yeah. Oh, that's true. I yeah, yeah. You slowly start to build this kind of cult of assessment nerds, I guess. Totally. In a way to to, you know, and I love that you say it's right there in front of us. Um, it's not we just need a little bit of time to think about it and reflect upon. And I listened to your last podcast kind of like the start of the school year, and you gave five assessment decisions you can do quite easily. But you launched this challenge at the end of it, which I loved. And I would like to reiterate that on this podcast as well, where you said one assessment decision that you are gonna change from last year. Can you can you add a little meat to the bone on that challenge that we can throw out there for our Quebec, well, our Canadian teachers will just say in general?

SPEAKER_01 24:22

Yeah, I think there's I mean, the idea of of changing, uh it isn't change for the sake of change. It is being analytical about where maybe the information was not as clear or where you were let down by an assessment that you provided, right? And I think sometimes we forget. I we know we know we have to assess our students, but sometimes we forget to ask the question, why? Why am I assessing? What is the purpose of this assessment? And that, of course, will speak to the formative and summative purposes, but even on a more granular level, which would be what evidence am I really looking for? So it isn't just change for the sake of change, it's change for the sake of enhancement. How do I refine my practices so I can elicit better information and more accurate information about learners so I can then turn that around into next steps for them that are more specific and allows a student to see more clearly the pathway forward to confidence or proficiency, right? How do I make that pathway clearer and you know less obstructed and help them understand what's next for them as a learner? So the idea, you know, really comes from that each year, we had a year-end episode last year and said, hey, let's kind of reflect on our practices and let's ask ourselves maybe some tough questions about our assessment practices. And we know that most people do a pretty good job in assessment. We're not submitting that there's there's dire situations in every classroom, but we'd also on the flip side say there's probably a lot we can improve upon and a lot we can refine. And so where are those places of refinement? Where could I get a little bit more specific? Or maybe it's not about what I'm gathering, but it's when I'm gathering. Could I get more real-time information? Right. So could I engineer some conversations in classroom? Honestly, one of the most underutilized yet most powerful formative assessment strategies is to simply get kids talking to each other about their learning and the teacher goes and listens to them talk. You're going to hear the understandings, the misunderstandings, the correct terminology, the misunderstanding of terminology. You're going to hear everything in real time. So maybe my refinement is actually just a better question during classroom discussion. So when listeners, when you hear the word change my assessment practice, don't think that change always has to be so epic. Sometimes it's simply a refinement that enhances what you gather, which will lead to you giving better guidance as to what's next for the learner going forward. So, you know, it's one of those things where we just always looking to grow and always looking to improve and where we can refine our practices. To me, that would be the right way to approach it and the right balance between, you know, the sky is falling and everything's totally fine, right? There's a there's a midpoint that tells us, hey, we're doing a pretty good job and we can probably find places to refine.

Chris Colley 27:09

Absolutely. Nile, would you like to add on to that?

SPEAKER_00 27:12

Yeah. I think this is another similar answer to the idea of we're creating the working conditions that we're collectively shaking our fist at and going, ah, the system, nothing we can do about it. I would invite people listening to stop grading a couple assignments per week. Because right now, the status quo is there's this myth that's pervasive in the field that there's a policy, I often say it in air quotes, because there often isn't one, that we have to grade everything. We have to put everything in the grade book, no matter what. It's an expectation. And yet, teachers are also complaining, I have no time, I have to go home, I have so much marking. I have to mark, if you're an English teacher, especially like I was, I have to mark all weekend so much marking. Oh, and then I have to give feedback. Oh my gosh, I got to explain why they lost all these points. If you just embrace this idea that, oh no, this is formative. So I don't need to put a grade on it. I don't need to summarize. I just need to read through enough samples tonight to find one thing to teach tomorrow. It's faster, takes back your evenings and weekends, and it's more effective. And I feel like there's so many of those little moments in assessment where people are really scared of the change because, like Tom said, I think they imagine it's gonna make more work when in fact a lot of the shift is less work. It's just more intentional work.

Chris Colley 28:38

Absolutely. I love these ideas. And I often say that to teachers, we're not changing, you know, we're not tearing the house down here. We're tweaking things a little bit because you're actually doing the things that you might not be aware of already, but they're embedded in your practice. So sometimes it's small tweaks like that. I really appreciate the time, guys. This has been a really great conversation. You've made us all a lot smarter. Tom, I look forward to meeting you in October. Yeah, it'll be great. And uh Natalie will we'll run into each other again one day, I'm sure.

SPEAKER_00 29:12

Yeah, definitely. I hope so.

Chris Colley 29:14

Really appreciate you guys and uh check out the podcast, guys, that they have, and uh I'll put all of that in the description. Thanks again, guys.

SPEAKER_01 29:22

Yeah, thank you. Thank you.