Chris Colley: 0:13

All right. So here we are, another episode of ShiftED Podcast, and I have a real treat for our listeners across the province, across the world I am reaching literally across the world. I'm pulling in John Hattie over in Melbourne and he's morning, I'm day, and hey, we had a great little intro chat and I was just so excited to have the opportunity to have John on the podcast to talk about not only his research but where it all kind of started for him. So we're going to kind of do a little a smallish deep dive within the time that we have. So, john, thanks so much for taking some time and joining us.

John Hattie: 0:49

It's a pleasure. Chris Love talking about this stuff.

Chris Colley: 0:53

Oh, it's so fascinating. So before I start, I'd love to just kind of build a little foundation on which we can have our conversation, and I kind of mentioned that at the beginning of this that you had gotten your PhD just across our next province over in Ontario from University of Toronto and you did your first degree in statistics.

John Hattie: 1:16

Is that correct? Psychometrics, yes. Statistics, yes. That was the basis.

Chris Colley: 1:20

Right, and what led you to that? Where did that interest begin? That you liked numbers and researching and finding stuff out? Where did all that interest bubble up from?

John Hattie: 1:30

Well, I have to give credit to Mr Tomlinson, my math teacher in my final year of high school, who turned me on to the subject and made me realize that it was worth doing and it was fun. But I also lived in a small country town and the only way I could get out of that town, as I discovered, is they paid you to be a teacher. I trained to be a teacher and so I had the chance of. Another Canadian. Tom McGuire from Alberta, came out to New Zealand, where I was at the time, and he said, yeah, why don't you continue in the area? And he put me on to Oise as the place to go. And so I went there and had a really exciting, wonderful experience. Said, yeah, why don't you continue in the area? And he put me onto Oise as the place to go. And so I went there and had a really exciting, wonderful experience.

John Hattie: 2:11

Got used to shoveling, snow and skiing and ice hockey and all the things you're supposed to not do while you're doing a student, and my specialization was in psychometrics and statistics, so that's where it started. But the funny thing about that, chris, is that and I'm sure many of your listeners will appreciate this we kind of tolerate it in faculties of education. I kind of think, yeah, we need someone in that area. But and so when I first took my first job on, everyone told me that if I was going to be successful, I had to study, ironically, what they were studying, because it's going to make the difference whether it be communication or technology or curriculum, and it kind of bemused me that everybody knew the answer. It was what they were doing.

John Hattie: 2:56

And then I went out into schools as part of teacher education and I met many, many teachers with the same philosophy. All you need to do, john, is watch me, because how I teach is wonderful and I've never met yet I don't know if you have, chris, I've never met a teacher who said they're below average and it didn't make sense because, I'm sorry, I was a kid. I know teachers varied in terms of their impact on me, and so it kind of was the bringing together, because my whole career until the last 10 years has been in psychometrics, it's been in measurement, it's not been visible learning at all, and so, about 2009, I decided to finish all I'd been looking at on that area of bringing together the measurement, to ask to change the question from what works, because everything seemed to to what works best and could we come up with a relative notion? And that's kind of where it started and it took me about 30, 40 years to collect the data, um, and kind of took over well, and john, how did that visible learning begin for you?

Chris Colley: 4:01

like, what was it? Was it a conversation you had or was it a part of a study that you were doing?

John Hattie: 4:07

like, where did the nucleus work begin no, it was a straight hobby, a side project. The whole concept of matter analysis started in 1976, in the same year that I started my career, and so as a measurement person, I thought the best way to find out what this new thing is to do one. So I did one, then I did another one, and it was many years later. And I remember the moment and I was at the University of Washington in Seattle and I was sitting there thinking maybe if I did a synthesis of meta-analysis, I'd have the data to change the question, to ask about the relativity of effects. And that's where it started.

John Hattie: 4:43

And, as I commented before, in the first few years the data were patchy. There just wasn't enough meta-analyses. I had about 150, which was very, very small. Now I'm up to almost 3,000. And so, collecting the data, and then I've just collected the data, collected the data, and I've had, every now and then I've gone back and re-looked at it and re-looked at it. And, as I say, it was only when I moved to New Zealand in the year 2000, I thought here's a chance to sit down and see if I can make sense out of this whole thing. And that's where it came. So it was a very gradual thing. It was very much a hobby. It was not part of my day life, but it was fun because of that.

Chris Colley: 5:26

I had no pressure and you cannot believe, chris, the number of times I got it wrong, right, right. That's really interesting. And so, as this hobby evolved, what was the data like? We talked earlier before we hopped on about how data can tell these really amazing cool stories, like there's story behind those numbers. What were some of those early stories that really that that the numbers were showing you that was what was interesting, or shocking, or wow. I never thought of that. Like, did you have any of those stories you could tell us?

John Hattie: 5:50

Oh, totally Like. My whole measurement career has been about the interpretation of numbers, whereas often data people talk about the numbers. They come up with fancy graphs and pictures and then blame you if you don't understand them. I've been quite against that. So you're right, it was the story. What's the story here? And I really did expect to see a normal distribution of effects with a kind of a mean around the zero effect, and that's just not true.

John Hattie: 6:19

95% of things that we do in education enhances a kid's learning. That shocked me until I realized that enhancing learning means just improvement, whereas there are many teachers surprisingly, when you use the average of all the effects, 50% to 60% of teachers have a stunning effect on kids stunning effect they raise learning. More than a year's growth for a year's input. That's really impressive. And so probably what shocked me most of all is that the whole distribution of effects is incredibly positive, incredibly above the zero point. I did not expect that.

John Hattie: 6:58

Some of the effects like one of the earliest ones in fact, the very, very first meta-analysis was on class size. The effect is positive, but it's tiny, and trying to make sense of why such an obvious influence should be so tiny, it occupied my brain for many years. The other one teacher subject matter knowledge 0.09. Why is it so low? And part of my interest over the years was trying to look at those low effects, to come up with a story, why, like I'm a data person, I accepted the evidence. The effect size of subject matter knowledge is very, very small. And then what is the evidence? And that's what led me to many, many openings, many different ways of thinking about the world, which has really helped, and talking to people like Lee Shulman about this. I talked to him about this and talked to others about the class size and many of the other of those effects that are very, very small. And it still surprises me about some of those small ones.

John Hattie: 7:55

But the other side of the equation, trying to understand the big picture, like here's the problem, chris, with 300 million kids, you could get swamped in data. In fact, I was very, very proud of my final edition of Visible Learning in 2008. It was full of data 500 pages resplendent with graphs that had three-mode factor analysis. You name it. I had it in it and my biggest critic read it. My wife and she said and what two people in the world have you written this for? So I threw it away best decision I ever made and said focus on the story, focus on the story, and that's what I think is what really matters in all data.

Chris Colley: 8:39

Right and visible learning came out, as you mentioned in what was it 2008,. Right. Visible Learning came out, as you mentioned in what was it 2008,. Right, yeah. What was the initial reaction to your analysis and your research from scholars or researchers? What was that first impression once you put that book out?

John Hattie: 8:57

Well, it was my 10th book, so I expected the same as the previous nine. No one would notice, and a lot of academic publishing is vanity. You do it, you're happy, and I seriously wrote the book to finish it. To get all that done, I spent 40 years doing it. I might as well put it together. I never expected it to take off. In fact, the week it was published, the Times Education Supplement did a story on it which generated a lot of interest.

John Hattie: 9:25

Weirdly, in New Zealand, where I was living at the time, the front page of the major newspaper above the fold had a feature on it In January, which here is the time where all the parents are thinking about going to school. It created a storm and so I realized that something was happening here. But it was a reasonably slow takeoff. But it took off and I still look back and do not understand why it took off. A book like that you've seen it it's not what you call the easiest novel to read for bedtime reading. It's sold over a million. That's weird in our business. So I wished I had the secret, but obviously it hit at the right moment.

Chris Colley: 10:07

Well, I think too, a lot of our thoughts started to think about that information and bring it into our practice a little bit more. What were some of your great stories that you heard that people took the information that you had gathered and synthesized and that had success with it. In those earlier, like once the book first came out, those first couple of years where people were starting to take that information in, what were some of the good surprises that you discovered through the work that you had put out there?

John Hattie: 10:42

Well, what you said then, Chris, is that 20 years ago it would be very abnormal for a principal to go to the research literature. That's not true anymore, and I think many others people like Dylan, william and others have been popularizing the notion of using evidence in schools, and that's really happened. It would be tough now for any minister, any school, to not look at the evidence. But they're becoming much wiser interpreters of it, much better at critiquing it. And in the early days I had no interest of working in schools on a visible learning program. It was the team I was working at with the moment at the time which was developing the New Zealand assessment scheme. We'd finished the project and they said could they switch to have a go at it? And I said happy to do it. But two rules. One is I don't want to be involved. I know that I can go into a school, I can give a talk, I think it goes down pretty well, and by the time I've gone out to the car park nothing happens. So I don't want to do that. And the second thing is I want it scalable. I don't want it depending on one or two of you, because the biggest problem in education is we don't know how to scale. We're very good at fixing problems. We're pretty hopeless at scaling success.

John Hattie: 11:58

And so we started and we learned a lot. We learned a lot about what to do, and one of the things we learned very quickly is don't tell teachers how to teach, which is what a lot of professional learning is. And one of the big themes that we developed is not what you do that matters, it's how you think about what you do that matters. And the same as you said about data it's not the data that matters, it's how you think about the data that matters. And sometimes we focus on the data and we don't focus on the person doing the thinking. And so we've had some pretty good successes on the early days.

John Hattie: 12:32

And then we kept refining and refining. It got bigger and bigger. In fact, it got so large that it's now been taken over by companies in the US sort of oversight the thing for the world. I retain the role of quality control. I get the data. The other thing that happened was I realised that most professional learning focused on the teacher. Not surprising. I'm going to focus on the kid. Yes, if the professional learning doesn't change what the kid does, then what was the point of it and like one of my fascinations, for instance coaching, the effect size of 0.48 on teacher practice but the effect size on kids 0.08. It doesn't translate to kids, it doesn't keep its weight down.

John Hattie: 13:15

No, because we stop at the classroom door and we don't go inside, and one of my fixations at the moment is that changing the conversation about how we teach to how we learn. Now I'm greedy, chris, I want both. It's not either or it's both, and so I think that's one of the things I've learned over the years is how to reinvent the story so that you're trying to get at those big picture things. And so, yeah, it grew and grew and grew, and now we work in many, many tens of thousands of schools around the world have implemented, and so it's taken off.

Chris Colley: 13:49

I love, too, what you just said before, because I got that quote. I think it was from a talk you were giving how we think about what we do rather than what we do, what are we going to do, like today? No, think of it. Like that reflective practice I find in education sometimes gets overlooked and I think that that's one of the huge parts of being a teacher is that reflective practice that we're always kind of going back to think about how did that work? Could I tweak that? What's your ideas on that?

John Hattie: 14:17

Well, I struggle with the notion of reflective practice, chris. I really do, and the analogy I want to use is Alice in Wonderland. Like Alice didn't look in the mirror and see herself. And most reflective practice is that you look at what you do and you reflect on it. Like Graham Nuttall showed, 80% of what happens in a class you don't see or hear, so why would you want to reflect in the mirror? What I want you to do is, like Alice, go onto the other side of the mirror and see how other people see you. And this is why things like collective efficacy are so powerful but so hard to implement. Because so often we've said to teachers Chris, like you're a teacher, you have the autonomy to teach as you wish, you have the autonomy to interpret. When you come and talk to me, you say, oh, but you were there, you don't understand my kids, you don't understand my teaching. We've got all these protective mechanisms that allow us to look at that 20% in the mirror. So, yes, I like the notion of reflective, but I don't like the notion that it's been used as a mirror kind of notion. I want others, like in terms of others, I want others to come in your classroom and see the impact you're having on kids. I want to talk to the kids.

John Hattie: 15:28

One of the mistakes I made it wasn't a mistake in one sense, but it was latter was that league table of all those influences. So what I was finding in the early days is teachers were saying I'm doing this, I'm not doing these things at the bottom. And so that's when we switched and changed the message to know thy impact. It's about your impact. It begs the question what do you mean by impact? It begs the question do the kids understand and like achievement test scores? They're the outcomes. They're not the inputs, they're the outcomes. And so what is the classroom climate? Is it safe to make a mistake in this classroom? Is it okay to have a sense that you feel invited to come and learn? Is the concept of what it means to be successful transparent in the eyes of the kids? Do the kids actually see errors as opportunities, not as embarrassing? All these things about interpretation. That's the kind of reflection I want. So help me here, chris. I want another word, because the minute you say reflective practice, we say, oh yeah, that's us talking about how we think.

Chris Colley: 16:30

We get caught up in these, like we have all these terms right in education all the time and the hottest one of the day or the month or the year, and oh, we're going to be doing this whole language now. The hottest one of the day or the month or the year, and oh, we're going to be doing this, so whole language now, and I understand those that it does become a burden on the actual practice of because it's all in your head until you actually look at the kids and the effect that it's having on your students. And I guess I would tweak that. My reflective is that reflect on how the kids are reacting to what you are doing, kind of coming back to what you quoted as think about what you're doing. Rather, you know, like you got to tweak it a little bit to get at the meat of change or affecting success in your classroom.

John Hattie: 17:12

Like a byline, is when teachers see learning through the eyes of students. It's exactly that. And the word visible learning, like my critics say, it's absurd because learning is not visible, it's all in the head. And I'm saying but you missed the point. I want to make that in the head more visible to the kid and to the teacher. Not easy, but it's the right stuff.

Chris Colley: 17:32

That's it Really cool, Like I mean, just this small chat, John. Things are starting to spark already, and another thing that you had said was stop talking and start listening, and I think, again, that's that same line that we're kind of, you know, bouncing around on that. If we're not listening to what's going on in our class, we're just going through the motions.

John Hattie: 17:55

Oh, and we're good at going through the motions. We're great performers. Chris, you were a teacher. We have 20,000 hours of classroom transcripts that we've used. We know exactly to the decimal point, on average, what percentage of time did you talk, chris?

Chris Colley: 18:10

I tried to talk as little as possible actually.

John Hattie: 18:17

I was a big advocate on getting kids doing stuff and getting their hands active.

Chris Colley: 18:19

Do you know what the?

John Hattie: 18:20

average is no 89% Now this is now your point.

John Hattie: 18:25

How could you possibly hear your impact if you're talking 89%? Now, let me be fair, I'm an academic. It's 100%. Now, my fellow Canadian, lynn Sheridan.

John Hattie: 18:38

I have just finished a book on learning to listen listen to learn Because I pointed out to her a couple of years ago I said isn't it fascinating? We've got thousands of books on teacher talk, lots of books on student questions, student talk and voice and agency, but no one talks about listening. And I think it's a really important skill. I was trained many years ago as a Rogerian counselor, which is about listening. It's about demonstrating back to you, chris, that I have not only heard you, I have understood you. It doesn't mean to say I agree with you and I just want to see. And it's really interesting when you ask kids like Pisa asked 31 million kids, 40% of them said the teacher does not understand how I learn, the teacher does not keep teaching me till I learn. And you think this is scary. Those kids want you to understand what they're thinking because they want to know.

John Hattie: 19:31

How did you do that, chris? Like I know, you're a smart guy. How did you do it? Now, here's the problem when you tell me how you did it. You edit that. You went over here, found it was wrong because you're good at error management. You came back, you went over somewhere else and you finally got the answer and I say how do you do it? And you say, oh, it was obvious. I went from there. It's not obvious. Learning's messy Learning's all over the place and there's a visible part, right, john, that you were talking about.

Chris Colley: 19:59

Like you see it, it's there Exactly.

John Hattie: 20:02

Oh well, like getting a teacher to make a mistake and talk about it doesn't happen. It needs to. Getting teachers to put up a problem and say here's a problem with the wrong answer, how did the kid get the wrong answer? Because I can guarantee you that half the kids in the class would have got that wrong answer and been told they're wrong. They want to know how. Now let me be quick and upfront, because my critics will jump in here and say learning is always about something. They're right, but remember, I'm greedy, I want curriculum and I want how to think. So I just think we need a lot more listening.

Chris Colley: 20:36

I totally agree with you. So, john, you just had a new book that came out, the the sequel. Right to visible. Learning the sequel. What were, what were the biggest differences between, you know, the original release to the sequel, what what's changed like? If you could paint that story with, with the numbers and the data that you found, um in between two, what's the latest story on how we're doing here?

John Hattie: 21:05

Well, the good news is that not a lot. The biggest difference was that in 2008, I had a lot of data and I was looking for a theory. In 2023, I have a great theory and I'm looking for data to disconfirm it, and so every time a new meta-analysis comes out, I could be wrong, and that's how science progresses and that's what we should be looking at. And so I think in 2023, I got I hope I got a lot smarter about focusing on the story, like, for example, I took most of the data out of the book and put it on a free website. Not only it's reduced the size of the book by 500 pages, it meant that I had to focus on the story more. The other difference is that in 2023, I had 10 years of data from implementing it in schools, and so I was able to hopefully be able to refine the message to be a lot more direct. I'm not pretending refine the message to be a lot more direct. I'm not pretending it's straightforward, but a lot more direct. One of my frustrations in 2009 is I didn't really understand the effect of teaching methods that well and I didn't focus on it. I think it's kind of an irony, chris, that in 2008,. I published a book on visible learning and did not have a chapter on learning. Ouch, I've made up for that.

John Hattie: 22:23

Introduce this notion of intentional alignment, following John Biggs' work on constructive alignment. What makes the difference in teaching is whether you're focusing on the facts and the knowledge or the problem and the teaching methods. Effectiveness differ. What we often do in education, though, is we say oh, we're a knowledge-rich curriculum or we're a problem-based, and we polarize it, and the problem is that problem-based learning is pretty hopeless for knowledge-rich development, and direct instruction isn't that very good for problem posing. It's when you use it at the right time for the right focus, and that unraveled and made a lot more sense out of the different teaching methods, and here's an observation that stuns me. When anyone wanting a master's or a PhD, it's a perfect topic.

John Hattie: 23:13

When I look at the 400 and odd different teaching methods that Bob Mazzano identified, only one, maybe two. Only one covers both the content and the problem-based, the jigsaw method. Why isn't someone inventing teaching methods that deliberately cover both, like direct instruction and reciprocal teaching, like the acronym DIRT? We're going to do DIRT. No one does that, and I just think this is such an obvious gap in our whole literature. Getting away from this notion about this is how I teach to. Given the content we're teaching today, this is how we do it. I want two assignments one about the content, one about the problem-based. I want two different activities, I want two different teaching methods and that's what helped. So that was a big difference between the two books is try to better understand the teaching. As you can see, if you compare, the two books spend a lot more time on the notion of learning. Like when I ask teachers about their theory of teaching, quite frankly it's a very rich, long conversation. When I ask them about their theory of learning, it's silence. We must have both.

Chris Colley: 24:20

I like that idea. It brings me to the Kenny Rogers model that you had coined. You know, knowing when, when do you hold? When do you fold?

John Hattie: 24:32

Well, thanks, kenny. We slightly changed his words. You've got to know when to hold them. You've got to know when to play them. Again the point you were making right at the beginning. People would go away and say intentional alignment. What jargon is that? Kenny Rogers? Simple, got to know when to hold him. Got to know when to play him.

Chris Colley: 24:50

I think all great lessons come from songs. There's so many great songs out there that have so many great lessons.

John Hattie: 24:55

Well, you talked about my PhD thesis. I hope everybody doing a thesis has a song. My song was Another, and another one bites the dust Every page. And another one bites the dust. Thanks.

Chris Colley: 25:08

Queen. So, john, to kind of like bring things to a close again. Thanks so much. This has been so fun. I've just really enjoyed talking with you. We have AI, right, artificial intelligence. It's the talk of education now. It's going to transform everything. The paranoia, the excitement, I mean it's a plethora of feelings that we have towards this. What do you? What do you? How do you see this artificial intelligence in our system of education? Just from you know, a professional standpoint.

John Hattie: 25:40

Well, three things. Firstly is it's the biggest change in my lifetime. The second is don't underestimate the skills of schools and education systems to resist using it. We've resisted using televisions, calculators, social media, iphones. We ban them, all that kind of stuff. We're very good and when you look at the 300 meta-analyses on technology since 1975, the effect size has been very small. We're very good at resisting. But the third thing is the way through this.

John Hattie: 26:12

What I'm trying to do now and I invite listeners to think about this is what are the skills we need to teach teachers and kids to use the AI Like? Take, for example, what I think one of the most critical skills. We have to learn how to ask the right questions, private questions. Now we know teachers ask two to 300 questions a day that require less than three word answers. We know that a class of kids not a kid, a class of kids asks two to three questions at most per day about their work. They don't understand. We are not very good at teaching kids how to ask questions. The problem with AI is, if you ask it the wrong question, it gives you an answer. We have to be smarter about how we probe. The second thing is quality control. Kids think and teachers tell kids that it's the teacher's role to do quality control. It's not. We need to teach kids when's good, good enough. We need to teach them. Is it right or wrong? How do you fact check? How?

John Hattie: 27:08

do you triangulate and, I think, focusing on what are these skills? We need to teach kids how to work with each other in teams, to critique and to improve, and so I'm adopting the adage like take lesson planning. I don't understand why a teacher ever plans a lesson. I want the 80-20 rule. Get AI to write your lesson for you and then you use 20% of the time on quality control. Improve it. Now, think about that, chris. 99% of lessons a teacher constructs, administers to the kids. It's too late for quality control. It's over.

John Hattie: 27:42

I think AI has incredible opportunities. Yeah, it's got massive downsides. I know that We've actually written a paper on the downsides as well as the upsides, but I also worry that it's going to be one of the slowest interventions and schools are going to be the last place. It happens by teachers, not by kids, and we already know kids are using it, so we see that as a problem, that cheating. Well, we just said the same about TV. I remember this dates me in the 1950s, when we went from a pencil to a fountain pen. We weren't allowed to use barrows because they were going to kill writing. Now you've heard all those stories. Come on, teachers, let's get smart. How do we look at the skills we need, how do we develop those skills ourselves, and I just think it's an incredible opportunity. It could dramatically transform and, adopting the line that many others have taken is AI will not put a teacher out of a job. A teacher using AI will put a teacher out of a job. Yeah, well said, john, well said.

Chris Colley: 28:38

AI will put a teacher out of a job. Yeah, well said, john. Well said Well. And that idea of developing skills within our students. I find that we need more of that, and AI is this opportunity for us to start really thinking about. How do we make kids more critical, how do we make them look and dig deeper and question things, rather than just accepting what the teacher said.

John Hattie: 28:58

But I also want them to know knowledge, and that's where AI can be good. I use it probably every day. I don't know something. In fact I've used it twice this morning. There's a new tool out that actually writes the essay for me, and I've constructed one. Read it and the first one I thought ah. Then I realized I was the fault. I asked it. Such a generalized question. It gave me a generalized answer. Such a generalized question that gave me a generalized answer. So hone, hone, focus quality control.

Chris Colley: 29:25

That's the skill we need to teach our kids, and I'd even push that into our pre-service teachers. Asking good questions should be a course in itself that you learn, because, I mean, that's what one of our jobs.

John Hattie: 29:33

I feel. Anyways, that's half the course, chris. The other half of the course is getting kids to ask good questions.

Chris Colley: 29:39

Exactly, first you have to understand what makes a good question and then that development can chris, the other half of the course is getting kids to ask good questions. Yes, exactly, first you have to understand what makes a good question and then that development can happen and start to share it.

John Hattie: 29:47

but they have to have opportunities and, and to be specific and jargony, it's probative questioning. It's how to probe right a very specific kind of question and if you go and watch in any classroom, you'll never see those questions from the students.

Chris Colley: 30:02

I need them. Very true, you're dropping it's an exciting world we live in a lot of interesting thoughts, john, for sure I I would love eventually if we could have you back um I mean I have I've asked you two questions and then we've been chatting, but there's so much more that we could talk about and this just been a real pleasure. I really, really appreciate your taking a bit of time out. Thanks, chris, take care, and we'll hopefully talk soon. John, happy 2025.

John Hattie: 30:32

Thank you, Thank you listeners.