Traditional standardized assessments are easy to use because they expect every student to take the exact same test. It’s easy to grade and inexpensive to administer. But these kinds of tests don’t work. They expect mastery at a granular level and don’t account for students who are well above or below the bell curve.

Technology now allows us to administer adaptive assessments, where students are taken on a journey and the test is customized to find out where they excel and where they might need more help.

One person leading this charge with adaptive assessments is Nathan Thompson of Assessment Systems. In this week’s episode, Nathan shares more about the importance of adaptive technology, how we can do better as educators, and other use cases for this technology.

About Nathan Thompson:

Nathan Thompson is CEO and Co-Founder of Assessment Systems, a company driven to improve assessment by making modern psychometrics accessible to more organizations, from adaptive testing to item response theory to automated essay scoring. Nate earned his PhD in Psychometrics from the University of Minnesota, with a focus on AI algorithms in adaptive testing.

Jump in the Conversation:

[1:42] – Where Nathan’s story of impacting education began
[3:11] – What is computerized adaptive technology
[5:20] – Adaptive technology is beneficial for making assessment the right size for every learners
[6:14] – Where we are as a country in adaptive technology
[8:07] – Lots of room for improvement on state assessments
[9:09] – How to make adaptive tests more realistic for teachers
[10:18] – Difference between formative and summative assessments
[11:24] – How to do better as adults
[13:31] – How it works in the real world
[14:20] – How to get started in creating better assessments
[16:00] – Assessments can help people evaluate where they should go in their careers
[18:23] – Creating stackable credentials
[21:28] – What’s next for this mission
[22:42] – How educators can use smarter and more accurate assessment tools
[24:48] – What about for educators that struggle with statistics and numbers
[26:56] – Turbo Time
[28:06] – What people need to know about computerized adaptive testing
[29:40] – What Nathan brings to assessment work
[32:07] – Nathan’s Magic Wand
[33:32] – Maureen’s Takeaways

Links & Resources

• Assessment Systems
• Connect with Nathan on LinkedIn
• Follow Nathan on Twitter
• Agilities
• Email Maureen
• Maureen’s TEDx: Changing My Mind to Change Our Schools
• The Education Evolution
• Facebook: Follow Education Evolution
• Twitter: Follow Education Evolution
• LinkedIn: Follow Education Evolution
• EdActive Collective
• Maureen’s book: Creating Micro-Schools for Colorful Mismatched Kids
• Micro-school feature on Good Morning America
• The Micro-School Coalition
• Facebook: The Micro-School Coalition
• LEADPrep

Transcript:

Maureen O’Shaughnessy 0:03
Hello fellow parents and educators. Thank you for joining me at Education Evolution, where we are disrupting the status quo in today’s learning models. We talk about present day education, what’s broken, who’s fixing it, and how. I’m Dr. Maureen O’Shaughnessy, your host and founder of education, evolution, micro school coalition, and co founder of active, I consult and train with schools and leaders who are fiercely committed to changing the narrative, reimagining the education landscape, and creating learning that serves all children and prepares them to thrive. If you are new, welcome to the podcast, please subscribe on our website to get it delivered to your inbox weekly. If you’ve been around a while, have you left a review?

Maureen O’Shaughnessy 1:08
Hi, Nate, it is so good to have you on education evolution.

Nathan Thompson 1:12
Thank you, Maureen. I’m glad to be here.

Maureen O’Shaughnessy 1:14
And listeners. today I’m chatting with Dr. Nathan Thompson. He’s the co founder and CEO of assessment systems, and Nate with your extensive experience in psychometrics and test development. In so many capacities, you have a really unique perspective into the assessment industry and the best opportunities for improvement. I think assessment is kind of make or break in how we even treat learners. So I can’t wait to dive in. And first thing I always like to ask is just about the evolution. We know schools have to evolve to sort of learners. Reliable assessments are needed to make learning relevant. I’m curious, where did your story of impacting education begin?

Nathan Thompson 1:56
Oh, thank you. Well, uh, both of my parents are teachers, actually. So I grew up in the education field, that’s for sure. And I always wanted to be a teacher growing up. And I was ended up being a psychology major in college. And one of my advisors there said, Hey, I think you should look into the quantitative side of things, psychometrics measurement, that sort of thing. And that’s how I got interested into it. And the thing that I like about assessment is that, you know, I fundamentally believe that assessment provides useful information about people, and that’s why it exists. And that information can be used to help instruction, it can be used to help organizations hire the right people, it can be used to help ensure that people have the right job skills to go into career that they want to go into. But there’s a lot of not so good assessments out there as everyone is well. So I’m all about providing services and software that makes it easier to make good assessments.

Maureen O’Shaughnessy 2:56
I love that because there’s so much information we want to get as educators or as employers. And if we aren’t asking the right questions, we’re not going to get the answers we’re looking for. So what you’re doing is very important. Before we start into assessment, can you explain what Computerized Adaptive Technology is?

Unknown Speaker 3:18
Sure. So I do a lot of work, like you said, with Computerized adaptive technologies and assessment. And what that is, is design and assessments built around a machine learning algorithm, that’s called item response theory. And those of you who are math teachers, or have exposure to math, you remember the standard normal bell curve, where you’ve got a average of zero and the standard deviation of one day? Well, it’s pretty easy to put people under that bell curve. You know, and you think about students in fifth grade math, where do they stand that sort of thing? By item response theory takes questions on an assessment and puts them onto that bell curve to so that we know if a student is that plus one, you know, above average, we can give them a question above average. So we use this to make assessments that adapt to people in terms of their difficulty, by giving them the exact level of difficulty that’s appropriate for them.

Maureen O’Shaughnessy 4:13
That makes a lot of sense. And I wish I had known that I took students a couple of years ago to a college placement test. And one of my students kept taking more time and taking more time and I thought, Is he asleep? Is he frustrated? What’s going on? And I was able to talk to him at university and they said, Oh, well, the more you get, right, the further it goes on, because it can adapt. And it’s like, so many assessments are black and white. So you’re telling me that the the what you like to work with, can adapt to the responses that they’re receiving and personalize the questions based on if the student has comprehension or doesn’t?

Unknown Speaker 4:52
Yes, exactly. And it will get tougher and tougher and conversely it will get easier and easier for the lower level students. So you know if you’ve Add a fifth grader who’s not quite at a fifth grade level, the test will figure that out and give them easier questions, which not only allows you to get them a more accurate score, you know, a picture of where their skills lie, it provides a better experience for the examinee. In that case, as well, they don’t get as frustrated.

Maureen O’Shaughnessy 5:16
Absolutely, you know, I was just at a conference on inclusion. And we were talking about the bell curve. And we do a fairly good job and education on that middle chunk. But the kids that are out whether it’s gifted or challenged, they can get super frustrated, because it’s too difficult or too easy. So it would be really nice if we could include all learners. And adaptive technology sounds like a great way to make it the right size for every learner.

Unknown Speaker 5:44
That’s exactly the case. You know, one of the examples I first learned about it years ago in grad school was that the traditional way of building an exam was to take a bunch of middle difficulty items and deliver them to everybody. Right? Right, you get 100 questions, everybody gets the same 100. They’re all average difficulty. But that really does a disservice to the top students and to lower students. And that’s why adaptive testing is useful.

Maureen O’Shaughnessy 6:07
Absolutely. So let’s take this concept and put it into educational assessment. Where are we, as a country? I guess? And where could we be when it comes to educational assessment?

Unknown Speaker 6:21
Oh, there are a lot of assessments that are adaptive, that people don’t necessarily realize like that it like you said, you encountered that one that was a placement test. That’s one of the most common ones, I believe, the LSAT exam that a lot of people are taking when they graduate high school that is moving to an adaptive digital version this year and 2023. Wonderful. Yep, the GRE, which is the test you take to get into graduate school after four year university that’s been adaptive for like 20 years, is actually was adaptive when I took it in 1999, the GMAT exam to get into a business school or MBA that’s adaptive. And there are a lot actually, they take one big pool of math questions, you know, from K to 12. And they can adapt anybody along that line there. Because if you’re a fifth grade, while you might actually be operating at a seventh grade level, or third grade level, and they build the algorithm to adapt to that.

Maureen O’Shaughnessy 7:14
That makes so much sense that I can even see students that have maybe moved schools, they’ve had exposure to exponents, and they know one area above grade level, but maybe they really don’t have the metric system down well. So it’s nice that we don’t have to say this is fifth grade level that we can see who’s above or below on each topic, and make the instruction really match where they are.

Nathan Thompson 7:40
Yes, absolutely.

Maureen O’Shaughnessy 7:41
I don’t think that the mandatory state tests that students take to graduate, I don’t think they have the capacity to be adaptive. And it makes me kind of think that a lot of our assessment is still, how much can you memorize, and put back out to get the right answer? Do you have a sense that a lot of our assessment maybe isn’t as top notch as it could be?

Unknown Speaker 8:06
Yeah, there’s still absolutely more room for improvement there. Some state exams are adaptive, I do work with Indiana and Minnesota, and both of them include adaptive benchmark exams. And I know other states do as well. But there, there’s still a lot of work that can be done with, for example, more high fidelity question types, so that it’s not just so much memorization and calculation. But getting into deeper thinking, problem solving, making sense of data, you know, deep reading comprehension, things like that. And there was a lot of work that was done by the park consortium over the last 10 years, that’s not going on as much as it is now. But they’re trying to develop new item types that we’re trying to get into a more deeper evaluation of student learning.

Maureen O’Shaughnessy 8:53
So it seems to be that would be really hard for a teacher to do. Unless you wanted to correct every question and do it individually. It seems like a lot of times we want answers that are black and white, so we can tell if it’s right or not. How can we make this more realistic? In an everyday sense for teachers?

Unknown Speaker 9:16
Yeah, that’s definitely a tough one. Because it’s not easy to write good test questions, as anybody who’s done it knows very well. It’s, it’s easy to fall in the trap of making simple questions with black or right, black and white answers, right. And it’s harder to get into more complex questions like even just making sense of data. You don’t just want somebody to be able to come calculates 30 plus 30, then got to read some graph, understand some growth rate and some factory or some economy, turn it into a question of 30 plus 30. Because that’s the kind of work that we do in real life and trying to train students to be prepared for that type of thinking. So a lot of it, I think, is training teachers to think that way, you know, what are some of the best practices in item development? What are some of that? New item types that are out there that would be able to better evaluate students.

Maureen O’Shaughnessy 10:04
absolutely think that we have a lot of room for training and to stay current on things like this. I wonder You had mentioned to me before we hopped on this, you were talking about formative assessment. Can you explain to our listeners the difference between formative and summative assessment?

Unknown Speaker 10:24
Sure. So formative assessment is assessment that’s happening during the educational process, that to help the instruction itself, you know, find out where the students lie, where we can better direct teacher effort to help them learn more, or the students that are well ahead know that they’re well ahead. Whereas summative assessment is an exam that takes place at this summation of some educational process. It could be like a benchmark exam at the end of eighth grade, let’s say, but a lot of the assessments that we take, as adults are really summative assessments to you know, you go to medical school, and you’ve got to take an exam at the end of it to become licensed as a doctor, right? Summative assessment

Maureen O’Shaughnessy 11:07
Got it. So what have you created with assessment systems to help us do better in assessing to get the data we want? And to prepare kids to do more than answer right and wrong kinds of questions, and to be able to be good thinkers as adults?

Unknown Speaker 11:24
Well, I like to make software that allows organizations to implement psychometrics like item response theory or adaptive testing on their own. Yes, there’s a certain level of expertise that’s required to do that. But if you go back, you know, 2030 years ago, and the only organizations that were doing this level of technology, were those big exams, like the GRE and the sap, and so on, right. I’m trying to empower, let’s say, even a university that wants to make their own English Placement Test for incoming international students. To make that adaptive, I have two universities that I work with who are doing just exactly that.

Maureen O’Shaughnessy 12:05
That seems really important, because it’s not like different organizations are going to necessarily want to buy the same test. They have unique populations, unique Look Fors. And so your software can help them use best practices and current research to create the assessments that they need for their populations.

Unknown Speaker 12:30
Oh, that’s exactly right. That’s one of the ways that I describe what I do is working with organizations where off the shelf exams are not going to meet their needs, you know, I do a lot of work internationally. Because you go to the Ministry of Education in the country, they couldn’t care less about a prebuilt examining Common Core standards, right? Because they’ve got their own educational system, their own educational curriculum. They just need a software platform and expertise for them to help build their own item make to their curriculum. And that’s the kind of work that I enjoy doing.

Maureen O’Shaughnessy 13:00
That is something that sounds like it could be valuable everywhere. And as somebody that’s worked overseas, yeah, the Ministry of Education in Kuwait is looking for some certain things and making sure there aren’t some other things. So they definitely don’t want to be off the shelf. Tell me how it’s working. When you when you’re doing this, do people go? This is too hard, or Wow, this is transformative? What’s their take on this? Because this is a very different way of looking at assessment. And it’s it’s more work, but it’s also more valuable.

Unknown Speaker 13:31
Yeah, it certainly is transformative. And it’s it varies by you know, what country organization I’m working with, because some certainly have more resources than others. My largest project that I work with is the United Arab Emirates Ministry of Education. And they, of course, it’s a well developed country, they’ve got money from oil and tourism. And they’re putting that into education. And they’ve got a team of experts, a team of psychometricians that really, really know what they’re doing. On the other hand, I’ve gone to places like Kazakhstan or Botswana, where the level of development in the world of psychometrics is much, much less now than it is in places like the US, UK or UAE. And those cases, it’s about trying to build capacity to understand modern assessments and how it fits into the educational system.

Maureen O’Shaughnessy 14:22
So how would you start? If somebody’s like, Hey, I hear this country is doing this and so we’re checking it out. But how would you start if they’re pretty satisfied? With a my assessment I get I see if they know the answers, or they don’t, why would I need something more? Where do you even start to build that capacity if there’s no awareness, and perhaps even resistance?

Unknown Speaker 14:45
Yeah, that’s certainly a good question. It’s hard to do that. In most cases, they’re coming to me because someone in the organization has realized that there’s room for improvement, whether it’s to because they need to move from paper to online or because they want to move to a day Up to where they want to use to more modern item types. Or maybe they’ve got some high level policy situation in the education ministry that says, hey, we need to get more of these types of assessments because we need to improve our math or English or something like that. Because it because you come down to a lot of it is like countries trying to compete on the world stage now, you know, they’re trying to improve their math and English skills, because they want to compete on the world economy, because it’s, you know, we have digital nomads and everything else these days. So, in a lot of cases, they are coming to me in that situation. And then it’s a question of, you know, practically, how do we get there from here, you know, if they are still delivering paper exams, and they want to get to delivery nationwide adaptive tests online within 10 years? What is it going to take to get there? And what kind of resources do they need?

Maureen O’Shaughnessy 15:50
I like that, that you can start where they are, and build capacity or empower, depending on that. I want to pivot just a little bit. I feel like a lot of times we tell our high schoolers, hey, go off and go to the four year college that your cousin went to a four year degree and your life is set. And my 20 Something daughters are like, that was a myth. And why didn’t they tell me what I needed? So I could get a job and get paid? And I think a lot of four year universities are about getting degrees, not necessarily about where this can be applied? Are you seeing assessments helping us figure out what skills somebody really needs to be successful in a career? And evaluating that?

Unknown Speaker 16:36
Yeah, absolutely. I think that’s one of the green fields for the world of assessment and not just assessing, there’s only one component about that. But it’s all about shifting to, you know, a knowledge economy and being able to measure what people can do, not just what college they attended. My four years of college, were some of the awesomest times in my life, that’s not true for everybody. And for sure, nowadays, I mean, I could learn everything that I learned then off of YouTube, if I just knew where to go. So there are certainly initiatives for dealing with that change of paradigm. In fact, there’s one called the credential as you go Initiative, where organizations are working together to try to change the way they’re thinking about job training and job credentials. Because that four year university is not necessarily the only way to go. I mean, maybe you get a two year degree, take some bootcamp courses in Java development, or data science or something else didn’t get a great job like that.

Maureen O’Shaughnessy 17:33
I like that. Because I’m sometimes concerned that we put so much into a four year degree with somebody that has no life experience, they have just left home for the first time or maybe are still at home, and then say, now go off and have your career. And maybe you’ll have a little professional development here or there. But it makes more sense to have like a two year base. And then as you need different things to be able to take boot camps, get certifications, get credentials, or as you learn more, hmm, I like being in banking, I wonder what it would take to be involved in the loan piece of it. To get the training, as you gain more experience seems so much better than front loading it with 18 year olds that really are trying to figure out who they are as adults. And so this seems super important. Is it? Is it getting traction?

Unknown Speaker 18:19
It is there’s even a term for what you described. And it’s stackable credentials.

Maureen O’Shaughnessy 18:23
Tell me more.

Unknown Speaker 18:26
Yeah, so they realize that credentials shouldn’t just be a simple summative assessment, like, you know, taking a big exam at the end of medical school, right? There are so many different skills that you can gather along the way, especially in certain fields that are still finding themselves like digital marketing. You can think of all the different skills you can have in digital marketing, you know, trenches have to go to university for four years and take one big exam to go into that field, you can get maybe a credential in WordPress, you can get a credential in Google AdWords, you can get a credential in SEO, whatever it is that you need those in your stackable credentials within the field of digital marketing.

Maureen O’Shaughnessy 19:05
That makes sense. I was just talking on a LinkedIn live with somebody were talking about public speaking. I didn’t know until well into my career because I can get in front of people and read things and share things. I didn’t realize I’m pretty nervous about doing public speaking without anything in front of me. And so it wasn’t an official credential. But Toastmasters was a godsend. And I was in Toastmasters for a couple of years and really gained confidence without having my script. So I just see people as they go along in their careers, it’s really important that they don’t have to stop and go back, get a master’s or do something big, but they can just focus on that one area that they want. This, to me sounds like a really important wave of the future.

Unknown Speaker 19:48
Yeah, I think so too. It’s going to change how people train for careers and greatly facilitate how they can change careers.

Maureen O’Shaughnessy 19:57
So is this something they’d find out through to Google or Microsoft or a university, how would somebody even start to figure out how they could do something other than the two year degree or four year degree?

Unknown Speaker 20:08
Yeah, that’s something I’m not sure with. Because you know, I’m not in high school right now. And my kids are too young to be in high school. I don’t know what they’re being exposed to. But there are certainly lots of places out there that get into this. There are boot camps for software development and data science, you know, product managers, so many roles like that. You can take a lot of courses now on places like edX, or Udemy, or Coursera. They’re not necessarily governed in quality. And some of those, you know, more so than some others, like, you know, Coursera, and edX, of course, are tied to universities. So you get a certificate because you’ve completed, let’s say, 10 credit hours of work from Northwestern University, or something very legit like that. That’s absolutely a high quality, stackable credential in those cases. But I’m not sure how kids these days are being exposed to something like that, that you don’t necessarily have to go to Northwestern University for four years. Do you know that Coursera already has a six month certificate on digital marketing from Northwestern University that you can take?

Maureen O’Shaughnessy 21:12
Wow, yeah, I’m not sure either. Now, it’s making me curious. And I want to look into that, because I like for our kids to have a ton of options. Because one size never fits all? No. So you’re doing really important work and helping organizations look at assessment in a way that’s going to be much more valuable and relevant. What do you see as next for you and your mission?

Unknown Speaker 21:37
I’m interested in applying principle, machine learning and artificial intelligence, because those continue to grow. Because those are becoming much more democratized as well. We all heard about chat GPT within the last couple of months, yes, but there are opportunities there as well to like, making chat GPT detectors is is something that’s going to be important. And there are things that have been, you know, again, functionality that was only available to billion dollar companies or nonprofits 1015 years ago, is now available to everybody for free. And that’s going to change things. So much automated essay scoring is another one. And 20 years ago, you know, there was code for automated essay scoring, but it was only done by gigantic corporations. Now, anybody who knows how to use R or Python can build their own automated essay scoring system.

Maureen O’Shaughnessy 22:30
Boy, this is definitely changing how we think of assessment. Yeah. And if you’re looking at where educators are now, and school leaders are, what might be two or three steps you could encourage them to take to use smarter and more accurate assessment tools.

Unknown Speaker 22:50
I think first part is a little bit of assessment literacy. Now understanding the basics of psychometrics is the first place don’t get into that. What does reliability mean? What does precision mean? What are some of the items statistics that comes out out of an assessment? So a very basic level psychometricians will look at questions on assessment, and they’ll analyze what percent of students got it right. And what we call a point by serial correlation, which is looking at how well the question is differentiating between the low students and the top students? Because if it’s a good question, it should differentiate people. But if it’s not a good question, students find it confusing. Those statistics are going to be low. So it’s the it’s a really a data driven field. So it’s important to understand what kind of data is used to build assessments and why, you know, it’s not just okay. Somebody is in a room sits down and writes 50 questions to make a lot of these assessments that are happening out there in the world, there’s a lot of work that goes into it with statistics that are analyzed and other aspects, like studying a cut score on an exam is actually a really, really involved process. You can’t just pick a round number like 70, because that’s what you use in high school. When you get to higher stakes exams, like a placement or you know, university admissions or getting hired for a job, things like that. There’s a lot of work that goes into deciding what the passing point should be.

Maureen O’Shaughnessy 24:16
Wow. So we want to make sure that our educators and school leaders really have some assessment literacy. And, you know, just as you were talking, I had some flashbacks to undergrad and grad statistics classes terrified me. And even my doctoral studies, I had to take something on qualitative and quantitative, and I survived the quantitative but my research was all qualitative. So what can you do for educators that are like yeah, that’s important, but gosh, numbers just mess me up. So somebody else that have to deal with it, how can you help them see the importance and make it not so scary?

Nathan Thompson 24:57
Oh, that’s fantastic. I think a lot of it, of course, is educators that are out there know, the quality of the teacher, you know that it’s, it matters so much actually. I said when I was 17, that I was never going to do math again, because my math teachers in high school were so bad. Oh, yeah. And then I got to college, and I had great math teachers. And I had a math undergrad, major, and then went and got a PhD in statistics. So it’s not like I didn’t like math, it was just the teacher part of it. Yeah, so I think it’s finding a good teacher. You know, there’s a lot of good resources online now, of people that are trying to teach complex math concepts, whether it’s statistics, linear regression, machine learning, all that kind of stuff. There’s a lot of good resources out there that will help you learn that, that kind of thing. And there’s no reason to be scared of the quantitative side of things. It’s something that we have the resources to do now. And you’re not stuck with the one teacher that you get, because there’s so many resources that are out there.

Maureen O’Shaughnessy 25:59
You are so right. That’s part of my college. Trauma, I think was the teachers I got at times. Yes, yeah. Yeah. In grad school, though, to his credit, we all were working on the board and applying the formulas, we didn’t have to memorize any formulas. But we were up there working them with teacher and student input, it was much less stressful, and it felt really valuable. Because I can look at formulas, I don’t need to have that in my head, I need to know what to do with them. So I really appreciated his approach.

Unknown Speaker 26:28
Yeah, but that’s exactly how I think of it to I taught statistics when I was in grad school to the undergrads. And I’d start off by saying, I don’t expect you guys to love statistics and go into it. But you just need to be able to know how enough of it to get by to do machine and paper, you know, understand the research papers you’re reading to make your senior paper. That’s my goal.

Maureen O’Shaughnessy 26:46
I love it. Yes. And then I’d like to pivot and get to know the person behind the content. So may I ask you a few turbo time questions?

Nathan Thompson 26:58
Sure, of course.

Maureen O’Shaughnessy 26:59
Awesome. Okay. What is the last book you read?

Unknown Speaker 27:03
Lost and founder by Rand Fishkin.

Maureen O’Shaughnessy 27:06
Lost and founder.

Unknown Speaker 27:07
Yep.

Maureen O’Shaughnessy 27:08
Cool. And how about two inspirational folks you’d love to meet

Nathan Thompson 27:11
Well geez, I think, Barack Obama, I think, for sure, is that a lot? And she’s so many people on the machine learning and math world, like, certainly love to meet Albert Einstein. You can’t see him anymore. But somebody like that?

Maureen O’Shaughnessy 27:28
Yes. How about a TED Talk that inspires you?

Nathan Thompson 27:35
Certainly, some of the things by Simon Sinek, about leadership. Some of it is kind of cheesy stuff that you know, goes towards selling books in an airport. But there’s a lot of good takeaways that you can get from it.

Maureen O’Shaughnessy 27:49
Yeah. And really getting to the core of the story before you start to talk about is eight. Yeah, I think he’s in his golden circles. And some of his concepts are really powerful. And you’re right, some of them sell books at airports. What’s the biggest thing you wish folks knew about Computerized Adaptive Testing?

Unknown Speaker 28:10
That it’s not about where the algorithm is diving into where somebody that’s just thinking that they’re weak? It’s that it’s a single scale of difficulty, and you’re sliding the scale of difficulty to the appropriate level of the student. Some people think it’s content areas, but it’s really difficulty.

Maureen O’Shaughnessy 28:28
That is such a great definition. Thank you, that makes really good sense. How about a pet peeve of yours?

Unknown Speaker 28:38
Gosh, well, certainly, the previous one is a pet peeve of mine. I’ve certainly heard that question before. Another one is automated essay scoring. I do work in automated essay scoring. And to do automated essay scoring, you need to have data and you need to train a machine learning model on it, and a lot of work goes into it. It’s a pet peeve when I get people sending in sales inquiries onto my website, thinking that, hey, they can just give me a bunch of essays. And I have a magic crystal ball that will read their essays, and also read their minds about what they think is a good essay, and automatically score all the essays.

Maureen O’Shaughnessy 29:14
And you don’t have that magic wand or that crystal ball.

Unknown Speaker 29:17
Yes. Yeah, it’s either one of the most important things of essay scoring is to define rubrics, you know, how are you going to score them argument on voice and these types of things?

Maureen O’Shaughnessy 29:27
And how does that mean?

Nathan Thompson 29:29
Yeah, yeah, they haven’t even thought about what’s a good essay. They just say, hey, score them for me.

Maureen O’Shaughnessy 29:34
Yeah, that doesn’t work. No. What’s the passion that you bring to your assessment work?

Unknown Speaker 29:41
Oh, I love looking for new ways to do things and easier ways to do things and trying to break down complex concepts into ways that everybody can understand. You know, teaching things like adaptive testing without throwing up big long equations on a PowerPoint slide.

Maureen O’Shaughnessy 29:57
That makes all the difference. Nobody wants to feel like they’re back in the university and being lectured. They really want to understand. And when you make it simple, then people don’t get defensive and shut down because this is too hard. So you’re making teaching?

Unknown Speaker 30:12
Yeah, yeah. Just you know, seeing a PowerPoint slide with a gigantic equation on it is not the way to go about it.

Maureen O’Shaughnessy 30:19
Exactly. I mean, even when you say, psychometrics, that’s a big word, lots of syllables, and not one in most people’s everyday vocabulary. So I could feel myself, like tensing up a little bit like, am I going to be smart enough to follow what Nate saying is? So there? Is that sense of scary? And do I try to learn this? Or is it too scary? So I agree with you breaking it down, making it simple, making it user friendly for folks that aren’t math and statistics whizzes? Totally nice? How about your favorite thing or fun fact about Minnesota?

Nathan Thompson 30:53
Oh, guess I like that. It’s God’s three iron ranges. I’m not sure if most people realize but a lot of the iron that comes in the United States industry came from Minnesota, also from northern Wisconsin, and Michigan. And one of my favorite things to do on my free time is mountain biking on an abandoned Iron Range in northern Minnesota, where there’s a bunch of old strip mines, that they put the tailings next to it. And now the tailing piles are wooded mountains in the strip mines are 300 foot deep, clear lakes. Beautiful.

Maureen O’Shaughnessy 31:28
Wow. That’s a really fun thing to know. I love it. Yeah. And how about something most folks don’t know about you?

Nathan Thompson 31:36
Oh, gosh. I guess that’s the small town, Wisconsin. I grew up small town at heart even though now I get to travel the world, it seems so foreign to me. I grew up thinking that, you know, Milwaukee was far away in a big city. And now having gone to places like Dubai and Shanghai, it’s changed my view of the world.

Maureen O’Shaughnessy 31:58
Absolutely, yes. And finally, I’d like to close with a magic wand questions if I hand you that magic wand and don’t tell you to correct essays? Without knowing the rubrics. If I ever hand you that magic wand, what would you wish for in the world of assessment? I mean, if you could just make it something was amazing and relevant, what would you wish for?

Unknown Speaker 32:22
A tool that could help people make good assessments and, you know, ensure that we don’t have assessments that are not accurate or not fair. Because really, you’re talking about what psychometrics is. And really, it is just people that are trying to make assessments fairer and more accurate. Because we don’t want them to be providing wrong information about people. And there’s just so much manual work that has to go into the doing those things, and we’re gonna figure out a way to make it easier. That’s going to not just make people’s lives better, that are doing the assessments like teachers, but it’s gonna make people’s lives better all over the world, in that they’re, the students are going to get the best instruction people are going to be putting into jobs that they like, and so many other places that our assessments are used.

Maureen O’Shaughnessy 33:12
I love it. And I love the outcomes that you’re talking about. Nate, thank you for what you’re doing. And thank you for being a guest on Education Evolution.

Nathan Thompson 33:21
Very welcome. Thank you for inviting me to talk.

Maureen O’Shaughnessy 33:32
Wouldn’t it be great if teachers could get really clear on what matters in their subject, and maybe weave into that what they are passionate about because kids can always tell when a teacher loves what they’re talking about, and then have ways to assess this priority material on a sliding scale. With the level of difficulty adjusting to see what each student has mastered. It seems like that would allow for equity. Once the teacher sees the gaps, especially informative. In the moment assessing adjustments could be made. There’s no reason with the degree of technology we have available for learning to be one size fits all. instruction can be adapted based on the assessments, and students can be challenged at every level, instead of being bored at the top or being lost and shut down at the bottom. Nate and I also talked offline about the importance of students knowing themselves and aligning with a future that appeals to them. We place a lot on our 18 year olds, when we ask them to figure out what they’re going to do for the rest of their lives and spend four years studying for it. And our career planning and placement programs are sadly under supported and underdeveloped at the high school and college level. Nightmare Jind work he had done with you science.com. And I’ve put that link in the show notes. And this helps folks understand themselves.

Maureen O’Shaughnessy 35:09
I also know that the Bruce foundation work recently that I talked about with Leann Taylor night in Episode 136, has developed a wonderful free tool at agilities.org. And they even have a process to map out a student’s goals visually or a worker’s goal. Visually. I look forward to seeing more programs like credentials as you go. We need to be lifelong learners. And wouldn’t it be lovely if we could do a little bit of training and certification, and then go out and get some life experience, then let that inform us of what additional training we might want to get. And we could repeat this cycle throughout our lives. That four year chunk of learning when a student is 18 Seems very outdated. Here’s to technology and dedicated people like Nate, helping us make assessments more accurate, equitable, and adaptive. As always, thank you for being a part of the education evolution.

Maureen O’Shaughnessy 36:24
I know how challenging it is to make changes inside your own school or community. I’ve spent years working with schools around the world on creating learner centered programs. And it always struck me how much schools were able to get done with the right tools and guidance. If you’re ready to make changes like this in your own school, let’s talk and put together an action plan. Visit educationevolution.org/consult for a free 15 minute call. And let’s see if we’re a good fit for more work together. Thanks again for listening. To support the education evolution. Subscribe so it lands in your podcast app and gets out to more decision makers. Then rate and review it. For more information in shownotes go to educationevolution.org. Education Evolution listeners, you are the ones to ensure we create classrooms where each student is seen, heard, valued and thriving. We are in this together and we need you. Let’s go out and reach every student today. Thank you for listening, signing off. I am Maureen O’Shaughnessy, your partner in boldly reimagining education.

Transcribed by https://otter.ai