Ed Thinking Cap

Stethoscopes and Education

2011-12-16T12:48:00.000-06:00

What would happen if a doctor used only a single instrumentto make a diagnosis about a patient? A stethoscope that finds a regularheartbeat and lungs free of pneumonia is incapable of offering up an opinionwhen it comes to diabetes, for which the proper tool would be a glucose test. Ablood pressure cuff can say a great deal about whether or not someone has highblood pressure, but says nothing about whether on not a patient needs anantibiotic to treat an infection.

And so it goes.

Should any one instrument be brought forward and presented,as the only instrument a doctor should use, the results would be catastrophic.Patients that were clearly unhealthy would be deemed as being just fine.Patients identified through the instrument as needing attention would be at themercy of an uninformed opinion as to what to do next, resulting in massiveinefficiency and less than stellar outcomes. Even worse, when an interventionworks it will not be because it was selected as the best intervention given theoverall needs of the patient, but because someone made a lucky guess.

Think of the old adage regarding test taking that when youdon’t know the answer just mark C. The reason is pretty simple: assuming fourchoices if you always chose C odds are 25% of your C responses would in factland on the correct answers, since most tests randomize the response pattern.The point is that if you guess consistently you stand a chance of hitting onthe right answer often enough that you might fool somebody into thinking itssomething other than luck.

Or consider a second adage: a broken clock tells the correcttime twice a day. If the clock was frozen at 6:00 and you happened to glance atit twice that day, once at 6:00 am and once at 6:00 pm, you might indeed thinkthat the clock was accurate and doing it’s job. In fact, you could act on whatyou perceived as being accurate information and due to the fact that youhappened to look at just the right time your actions would be seen assuccessful because of the clock eventhough the clock had nothing to do with it.

A one-instrument system cannot, by definition, work. When itappears to, it is either because we guessed consistently over time and as aresult hit on the right answer at least some of the time, or we got lucky inthat our actions were right even though the instrument was in fact broken.

Some remarkable things happen the instant a secondinstrument can be added to the mix. For example, the result of the firstinstrument can be called into question, as when a stethoscope alone declares aperson healthy while a glucose test suggests just the opposite. Or an outcomecan be shown to be the result of lucky guessing and not a good decision makingprocess. Or the first instrument can be show to be flawed or even broken.

We are trapped in a paradigm in education that accepts andeven celebrates a one-instrument system. We rely on standardized testing asthat single instrument, and while it appears in a number of forms (e.g., end ofyear tests, end of course tests, formative assessment) they are all based upona similar set of assumptions. Our love affair with the standardized test is along one and shows no signs of subsiding.

As a result, consider how far from the goal of a highquality personalized education that places us. Our over-reliance on a singleinstrument means that we lack any real basis for offering much personalization,and are left to come up with generic approaches that we hope fit as many of ourstudents as possible. When/if those approaches work, we are left to presumethat it had to do with something other than luck, and yet we have no way to proveor even know that. As a result, not knowing if a success was the result of ouractions or blind luck, success is in no way scalable.

The policy problem

2011-10-24T09:52:00.000-05:00

What if a government agency asked you to build a building that was guaranteed to fall down, but because it was the law you built it anyway?

Furthermore, what if that same government agency threatened your job if you failed to prop up the mess that got built every time it started to fall down? And yet again, what if that same agency now threatened to find the individual workers who worked on what was a bad design and hold them accountable for succeeding in spite of a design that all but guaranteed failure?

Making rising test scores the goal of education creates just such a system. Test scores were designed and intended to be a check on the system--and that is the limit of their use and their promise. Anything beyond that pretends that a test can magically transform itself into something beyond itself, something as illogical as it is dangerous when it comes to our students.

Article just published

2011-10-03T11:51:00.000-05:00

An article I wrote comparing teaching to the test in schools to studying for an eye exam was just published in the October 2011 edition of The School Administrator. Click here to open the article.

The paradigm of school reform

2011-09-27T08:26:00.000-05:00

For years now "reformers" in the form of educators and policy makers alike have operated under the assumption that they have the ability to fix this thing called education that they and others find profoundly broken. Such a paradigm brings with it a way of thinking about education that has considerable consequences.

For example, in both policy and practice, our actions over the past twenty-five years have taken the form of "anything has to better than this," since something so radically broken can't possibly get any worse and at least this is something. That has led to policies that created massive systems and infrastructure around ideas and tools that were never proven to produce the results dictated in the policies. It meant we stopped trusting educators, turning the reigns of education over to business leaders and other non-educators as if a business model could finally get it right. It meant we entrusted policy makers that have never taught a child with setting policy for that activity. And it now means that all of us have become addicted to test scores and accountability formulas that have almost zero capacity to signal what must be done to make a school better, because they were too short-sighted to consider that there might be a better way.

Make no mistake about it--at no point in the history of education has their been a moment where a careful look at education wouldn't show significant room for improvement (the same is true for any organization). But consider the very different reactions when you compare one approach that condemns American education and demands reform, with one that approaches each and every educator and asks the simple question: "what could be done to improve the quality of education for your students?" The first approach requires a process of condemnation, gathering agreement that the condemnation is justified, putting forth a proposal for change that will then compete with other proposals as the thing that can best fix the problem, and then a process of implementation that will be completed just about the time the next set of solutions are being presented as the new thing that will fix the mess. What a waste of energy that could be put to better use.

The second approach asks that accept what you find in the status quo (so neither good or bad, just as is), and determine a hand full of things you could to make it better. The steps to such a process are: look around, decide what to do, and do it. That doesn't require years to implement, it doesn't require a policy change, and everyone already has the tools to do it.

Imagine if we asked every school principal for the three or four things that they believed could improve their ability to help students achieve at very high levels. Imagine if we also insisted that these things be practical--as in, these things have to work within your budget, you have to work with your current staff, etc. I've had the chance to ask a great many principals that very question insisting on those parameters and every single one of them has been able to offer an immediate answer.

But that answer is almost always followed by the articulation of a set of constraints that are likely to prevent those things from happening, coming almost entirely from the policies and requirements imposed in the name of reform.

If we would stop operating under the paradigm of "reform" for the changes needed in schools we might realize that it is our policies that really deserve the level of energy demanded of the word "reform." If we realized that virtually every teacher and principle has ideas and a professional understanding as to how to make education for their students just a little better, and we held schools accountable for implementing those ideas, we could start tomorrow and see improvements almost immediately. Instead, our reform paradigm insists that teachers and principles really don't know or they would have done it already, that the system remains broken in spite of billions of dollars of investment and lots of hand-wringing by policy makers, and that the broken system has nothing to do with policy makers having spent and legislated stupidly.

Thus the argument to maintain the reform paradigm is really an argument to continue down a very expensive path that to date has accomplished almost nothing in spite of the level of energy and investment. The two great successes that reformists claim occurred on their watch have to do with the increased inclusion of students who historically struggle and increased attention to closing achievement gaps in minority populations, but ironically the argument can be made that this had more to do with broader trends in society. One can, I believe, make the argument that these things would have happened with or without education reform, and that the best the reform movement can claim is to have sped up the process.

What the paradigm of reform has not been able to produce is a great surge in national achievement or a grand improvement in the educational outcomes for children when that was its very promise.

It is high time we thought seriously about that.

Of interest right now

2011-08-25T07:37:00.000-05:00

I watched some YouTube videos recently of a DARPA competition to see whose robot could drive a car on a 60 mile course without outside aid. The cars were equipped with the latest in sensors, teams of some of the smartest scientists in the world oversaw each competitor, and some of the best programmers anywhere in the world were brought in to write the code that would become the brains for each robot. The track and every obstacle had to be extremely well mapped, the maps were programmed in to the massive computers, and away they went.

The winning car completed the course in around four hours, cost a whole bunch of money to develop, and walked away with a big prize for the effort. While the lessons learned are invaluable to DARPA, the effort remains at the prototype stage and will be years from having a commercial purpose. After all, if we're talking about putting human beings into cars that can drive on their own, lives are at stake and we have to be careful. The success experienced by the winner occurred within a closed system where every possible obstacle was known beforehand, which hardly describes what happens when we drive out in the real world.

I wish we would have but the same degree of thought into the data systems that have become so pervasive in schools. Numerous systems have been developed in recent years that gather all of a school's data under one roof, and then offer up the promise of driving instruction such that the result is an increase in achievement.

What disturbs me about that promise is the arrogance in thinking that the answers for something as complex as individual student needs can be programmed into a computer as if learning occurs in some kind of controlled environment. We are just now capable of creating a computer model of driving that is light years behind what a student in driver's ed can accomplish in the real world, and we think that we have the capacity to model individual student learning, something infinitely more complex? DARPA had around a hundred entries for their competition and all fell far short of replicating behavior that most of us do almost as a reflex. We have millions of students, none of whom are entirely alike, each with needs that are uniquely theirs. If we can't yet replicate a single, linear, well-defined task in a controlled environment with that amount of brainpower, why would we think we can model teaching and learning?

Systems, policies, and practices that support teachers in providing for those unique needs stand a better chance of offering an improvement or two for every student than systems that standardize an offering against an algorithm, no matter how complex the algorithm. It is shameful that we seem willing to relegate teachers to automatons within such systems, when the teacher is the only system that stands a chance of actually working.

What if the real standard isn't even in the test?

2011-08-09T12:04:00.000-05:00

Here's an amazing thought given all the efforts behind school reform: what if the actual standard that we have in our minds for what students should accomplish in school isn't even on the test?

Here's the simple reality: it doesn't have to be.

Standardized tests work when they can show the distribution of students within a domain. We currently define the domain via a set of standards, but the only rule a test designer has to follow in terms of the the standards in order to build a standardized test is that the items come from those standards. However, the most important criteria for a standardized test has little to do with the standards. Rather, the most important criteria is that the item contribute to understanding the distribution of students.

Items that contribute to that understanding are items that roughly half (say 40-60%, generally) of the students taking a test will answer incorrectly. A single item can then divide the students into two piles, a second into three, and so on.

But consider what that means for those things that might matter most to us. I certainly want my kids to achieve a high standard, but in order for items to be included only about half the kids can answer an item representing a high standard correctly. But what if it isn't taught so no one gets it right? If that is the case then the item won't contribute to our understanding of differences among students since it showed them all to be the same--they all missed it. In standardized test mode that item would be tossed as being useless for the purpose behind the test.

But lets say that just such an item managed to make it through somehow. To see the impact of such an item, imagine that all the items were lined up from the easiest item on the test to the most difficult. If a student took the items in that order, we could imagine them starting out strong and than at some point beginning to struggle, and eventually reaching a point where they answer the rest of the items incorrectly. Among those answered incorrectly would most likely be the item representing the standard we actually care about.

What makes such an item particularly useless in state testing programs is the fact that schools are judged by how many of their students answer a certain number of items correctly. So long as they do, the school is declared as having done their job, and to the degree they do not they are judged as missing the boat. Remembering that students will typically answer items correctly only up to a point, and then answer the remainder incorrectly, a school can in fact be declared 100% successful when 100% of their students miss the real standard as represented in that item but answer just enough items to get over the established hump.

Can you think of a stranger world? One in which the very goal of reform which was high standards for all may now be deploying an accountability measure that either doesn't contain the actual goal, or if it does, positions it in such a way that it doesn't actually matter?

Uni-dimensional students

2011-08-09T11:20:00.000-05:00

The selection of standardized tests for the purpose of school accountability remains one of the great culprits to schools actually becoming great. Consider just one mismatch that has huge consequences for teaching.

Standardized tests (of which all of our state accountability tests are a type) are designed to be what we call "uni-dimensional." That's because you have to be confident that the test is measuring only a single thing. You could, for example, combine a bunch of math and reading items on a test and obtain a score, but you wouldn't know what it meant.

Even within a single domain the same problem exists. "Reading," for example, is a multi-dimensional activity. It manifests itself differently in children, requires multiple approaches in order to learn it effectively, and over time students develop the habits and skills that make up reading in profoundly different ways. Just consider the ways in which students come to embrace concepts such as metaphor and irony as but two examples.

A standardized test, then, as a uni-dimensional measure, is poorly equipped to do much more than make broad generalizations about the real world multi-dimensionality of reading instruction. And yet what do we do with our accountability tests--and this according to legislative mandate? We return those test scores as quickly as possible for the stated purpose of informing instruction.

What was to have been a uni-dimensional proxy for a multi-dimensional, real world activity, becomes the real world. That narrowing is wrong not just because it narrows reading to tested material, but also because that tested material is selected to measure along only a single dimension, when kids don't in fact learn that way.

The disconnect is that in a world that was supposed to be about getting each child to learn more and do better against their needs, we are left to teach less and avoid the individualization that was at the heart of the reform movement in the first place. All because we put so little thought into selecting the tools for accountability.

Kids versus growups

2009-01-09T06:53:00.000-06:00

In a set of meetings this past week with some fairly innovative and high powered educational folks, the most common theme was that our current generation of students is ready to participate in their own learning in a way not seen before. These kids don't like the notion that an education will simply be delivered to them (or worse, done to them), but rather, they want it to be as participatory as the rest of their lives. This reminded me of how hard it is to get out of our traditional educational paradigms and actually do something that will begin to create a working system.

Educational efforts that work to streamline existing processes and programs (and this is most of them, by the way) do so based on the assumption that if we can get more efficient and more equitable with those old systems then the good things that happened to educate current educators can happen again, only this time for everybody.

But systems designed to educate only a few can't be brought to scale because they operate under the assumption that what worked for a particular segment of the population can be extended to everybody, when that is just nonsense. And systems designed by and for a mode of thinking that no longer prevails (or even, researchers tell us, exists anymore) among our digital native children are just exacerbating the square peg in round hole approach.

The trouble here is that pundits and politicians hear this and claim that if the kids can't read and do basic mathematics they are doomed, and thus advocate for a "basics first" approach. But the basics must--so the argument always seems to go, though often unspoken--be delivered in the old fashioned ways they were to the pundits and reporters, most of whom have no pedagogical training (which frankly renders many of them unqualified to offer up an opinion without at least a little bit of investigative effort). Absolutely right are those who suggest that reading is the gateway everything, but absolutely wrong are those who presume that a generational approach that stems from a print world can work for one that lives digitally.

That's why we have to go the kids and bracket our biases. We grown ups are more in the way than we think.

The big reform questions

2008-12-31T09:27:00.000-06:00

When reform began there were two big questions that educators needed to ask themselves: “what should students know and be able to do?” and “how good is good enough?”

The first, “what should students know and be able to do?” was to be the purview of the standards documents. Early attempts like the New Standards Project took the view that the standards should be primarily about performance, while state efforts created massive documents designed to drive a curriculum. At present all fifty states (and Washington DC) have some version of standards that guides the educational enterprise for their students.

The second question, “what should students know and be able to do?” was to be the purview the tests, at first consisting of tests deemed “authentic,” and later of tests based upon the more traditional standardized tests, often referred to as “criterion-referenced” because the content came from the standards and the judgment was against a cut score rather than a comparison among peers. But the initial question was expressed not as one that was to be exclusively measured by a test score, but rather, as a broader question that had both practical and philosophical consequences for how to determine what it was that students needed in order to have met the foals of education. In other words, it was one thing to demand that a student write, and quite another to determine when they had done so sufficiently.

How we have come to see these two questions implemented in the current reform is a journey that I shall not even attempt to articulate beyond what is necessary to make the point here. Suffice it to say that the standards component took the approach that what failed to make it into the standards would fail to be relevant, and so breadth became the watchword, and the tests that were supposed to be about the depth and richness of educational attainment came to reflect the voluminous breadth of the standards, eventually represented almost exclusively by multiple-choice items. Education has thus become largely about teachers teaching to and students passing overly broad multiple-choice tests. It is interesting to note that this transformation can be seen to be out of sync with the beginning of the movement simply by looking at the etymology of a term now synonymous with “challenging”: the term “rigor.” (I've made this observation in an earlier post as well.) In a strange twist that will surely have etymologists scratching their heads for years to come, a term that was originally invoked by reformers for its secondary meaning: “exactness,” has now morphed into something very different. People now refer to rigorous content, rigorous standards, and rigorous tests, none of which now evoke the original sense of the term. Instead, the term has come to adopt something of a secondary meaning, one not seen in any dictionary with which I am familiar, that of ‘challenging.” However, when one considers that the term is also the basis for the Latin phrase rigor mortis, which references an unyielding stiffness, one can see that the idea of "challenging" is a stretch at best. I think it safe to say that the original standards movement had in mind a certain sort of precision that was missing in education and deployed the term somewhat appropriately, but as standards took on a broader and broader notion the term continued to be applied but against the idea that the standards when deployed in education would be sufficiently challenging.

Indeed, a rigorous education would be one—against any dictionary with which I am familiar—one that is “harshly precise,” a notion perhaps descriptive of certain 19th century pedagogy but hardly appropriate for the goals of education today. If we mean challenging, we should say so, and in what way.

I digress. But with a point.

If educational standards were, to my point, intended to have an element of precision to them, what happened? A colleague of mine—Chris Minnich—pointed out the problem in that really there are three and not just two questions trying to be answered. The first—“what should students know?” represents a question that lends itself to overly broad claims as this is where the kitchen sink and all the other teaching biases—for good or bad—enter into the picture. Precision is simply not possible with the question because it allows opinion to reign and much difficulty exists in determining whose opinion should trump another. The second question, the one the Chris points out has just the opposite sense, is one where a broad consensus around a few points should be possible, or even provable through research and/or evidence: what should students be able to do? By separating it out Chris shows that much of our confusion may have come in that we thought we had to do both at the same time. Instead, if we focus on the few things around which we can generate consensus that students need to do to be considered “educated,” such consensus can be achieved independent of the biases that teachers might bring to their practice of getting students to do those things.

Standards written around what students must be able to do have a chance at consensus and thus can take students to the levels they need in order to find success. Standards written around what they need to know never will.

The problem with "rigor" in education

2008-02-09T07:26:00.000-06:00

A few thoughts. Mostly they are about vocabulary, but the educational environment suffers from imprecise language about the most important elements of our activity and the lack of clarity harms us in subtle but significant ways.

The word “rigor” refers to the quality of being thorough, exhaustive, or accurate. Its secondary meaning is severity or strictness. Only in its noun form (rigors) does it take on the idea of being demanding, but this refers to things like “the rigors of the harsh winter.” The etymology of the word comes from Latin and literally means “stiffness”: think rigor mortis. No where in the history of the word has it meant what we seem to think it means when used today in education. When people talk about standards we should excise the term "rigor" from our vocabulary and be a little more rigorous in our word choice when describing what we want—seriously. Note as well that saying “rigorous education” means an education that is exhaustive, thorough, and strict—a nineteenth century notion that isn’t going to serve 21^st century students well and its misuse in a modern context is embarrassing.

If you want tough standards or standards that are more precise or based on high levels of performance when compared internationally, say it. Don't use a term that fails the test of having an application in today's educational environment.

Another vocabulary issue concerns the term "standard." We use it to mean "a content statement that students must be taught." But then we use it when we refer to the passing score on the state test to determine whether or not a student has or has not met the standard. And then we compare states and suggest that one with a lower passing rate has higher standards than one with a high passing rates. And we confirm that inference about high or low with the standard set by the pass rate on NAEP (the National Assessment of Educational Progress). And seldom do we use it properly: as the threshold of performance we expect from a system or student. So which is it?

We have to get a handle on vocabulary. Standards, content expectations, cut scores, and passing rates are NOT the same thing.

Thus the idea of "higher standards" is interpreted to mean better content expectations, higher cut scores, or increased passing rates. These are fundamentally different from each other, and furthermore, all fail the test of ensuring that success at any of them will result in higher degrees of numerate, literate citizens ready to hit the ground running in the twenty-first century. Thus a term with great power: "standard," has been co-opted to make it mean what it never intended, and it confuses a lot of very smart people. We need to get back to basics on this one if standards are going to have a real role in education.

Precision around "rigor" and "standards" will clarify a great deal.

Education and rocket science

2007-12-12T07:03:00.000-06:00

I've now heard half a dozen people in the last few months use the phrase "it isn't rocket science" when referring to the solutions to our educational woes. But it is rocket science.

Imagine if we'd treated the goal of getting to the moon in the same way as education policy. In the space race we identified a goal and a time table, funded it, put some great minds to the problem and did it. Had we applied our current methodology for fixing schools we would have legislated the rocket building process itself, created an ambiguous goal that makes no sense to the broader audience and is ripe for misinterpretation, mired ourselves in technical conversations about which we know little or nothing, and then fought over definitions and specifications that may or may not have anything to do with rocket building. Odds are that the rocket would never have gotten off the ground, if even it had been built.

Policy makers have reduced educational success to passing a test. This bad solution follows the adage that says "education ain't rocket science." But it is rocket science and could learn some valuable lessons from our success as rocket scientists. Pointing to the moon worked. The moon in education is simply those skills that matter most to preparing a student for the world. Point to the skills, set the goals, and quit thinking that passing an end of year test based on content will get you there. Challenge our teachers in a meaningful way and you'll be surprised at how far they can take our students.

Quality and NCLB

2007-11-15T10:09:00.000-06:00

Three questions matter in educating students if quality for all is to be the goal:
1. How effectively did the system deliver the educational material?
2. How well did students learn what their teachers taught?
3. Was the student able to effectively synthesize and apply their learning in meaningful ways? (Tests don’t count here.)

Note the similarities to the system of apprenticeship, where a master teacher transfers and supports the acquisition of knowledge, but not as an end unto itself. Rather, that knowledge is a necessary precursor to the desired skill or performance. Furthermore, just because someone has the knowledge doesn’t mean they also have the skill.

Now consider the system dictated by NCLB, remembering that the intent was clearly to produce numerate, literate students. It clearly asks and attempts to answer through annual testing questions #1 and #2, but stops there. But the goal of an apprentice is to master the craft—or at least obtain some level of proficiency in it—and without a demonstration at the craft level we cannot be sure that the transfer of the craft has even occurred.

But as much as the apprentice metaphor seems to have application, note that the system in which we ask teachers to function is much more akin to that of a factory. Consider that students walk through a variety of stage gates called “grades,” that each grade brings with it the prescribed activities to be carried out called “content standards,” and that the exit criteria are set to determine what percentage of the raw material (students) came through clean (i.e., having passed a test). The problems with this paradigm or significant: factories must accept some waste or loss at each step, each stage must assume that the previous stage accomplished its work effectively on every part passed on and that it will do the same before passing the work on, and that in the end the goal is a consistent, predictable product.

What seems to be wanted is the craft result of students who can perform important skills beyond the level of an apprentice. What we’ve done to get there I apply a manufacturing model with the goal of consistency as the end result—a consistency that in education is measured through an end of year test of what was taught. Students become widgets whose goal is to test in a pre-established fashion, rather than apprentices trying to become journeymen.

If we believe quality is the goal of an education then we need to make some serious changes.

What does it mean to want higher standards?

2007-11-06T14:23:00.000-06:00

I just read another article by a group demanding higher standards for our students. A day doesn't go by that I don't see something like that, and always in the form of a maxim--after all, who wants lower standards?

What people are really asking for when they talk about higher standards is for more students who leave schools both numerate and literate. No one in their right mind could argue against that. They say things like "we want our students to be able to think and problem solve and write" and so on. Then we teach them to content standards that may or may not be capable of getting students to that point and then ask them to pass tests that may or may not represent success at those sorts of things.

And thus when pundits say "we want higher standards" what they think they are saying is "we want more numerate, literate students," but then they turn to the world of tests and content standards that they know and the acual translation comes out "raise the cut scores on our tests of content."

And it won't work--which should be obvious. If we want more literate, numerate students then we need to measure for literacy and numeracy and hold schools accountable for getting students to that point. If we want students to master a little more content we should raise the passing scores on their state tests and hold schools accountable for delivering that content--but if we do that it will likely be all we get.

Just remember: if pundits succeed in getting states more aligned around the percentage of kids passing, perhaps even against a more and more similar block of content, we'll soon be right back to this conversation about how we need to raise the standards yet again because we still aren't producing numerate, literate students at a rate we find acceptable.

That is what I consider unnacceptable.

End of year tests of content

2007-11-06T06:29:00.000-06:00

A system that identifies the content to be taught and tested during the course of a year assumes a great deal. As one example, it assumes that the vast majority of students beginning a new school year are prepared for the new, more advanced content, even if evidence exists from last year's test of content that they are not. And yet, consider that at the end of the year both the teacher and the school will be judged entirely on the student's ability to grasp the new content.

So where is the greatest incentive: to teach the content by which the teacher will be judged, or to teach the content the student most needs? It seems deplorable to me that we've created a system where the question can even be asked.

Kids learn and develop at different rates. A system that allowed kids to show their content mastery as it occurred, and that set benchmarks in a student's life as to when things should be accomplished, gets rid of that horrible question. It also moves closer to what a real standard is--something important that all kids should meet--because schools begin a year with an understanding of what a student has mastered and focus their efforts on getting kids past the rest of the benchmarks.

Change, change, change

2007-10-22T12:40:00.001-05:00

According to Eli Goldratt in The Goal, only three questions matter to a manager or leader:

What do we change?
What do we change to?
How do we accomplish the change?

Asking these the right way will have huge implications for school leaders.

Content verses performance standards

2007-10-22T11:34:00.000-05:00

The idea of standards in education arose because in a Democracy it just makes sense to define same fundamental levels of citizenry and numeracy and literacy at which a person will be able to make a contribution to their world. That means we have somewhere an anticipation that students should leave schools having demonstrated a number of skills requisite to meeting that qualification.

Note, however, that in virtually every public school in this country demonstration of skill is not even a requirement for our students--in fact, the way in which the current standards movement has evolved in which standards are taught and assessed is almost antithetical to a skills-based education in spite of political expectations that it be otherwise.

Consider writing as a topic for illustrative purposes. A skill-based approach would suggest that no student should be allowed to experience their seventeenth birthday without having written well at least once--whether an extended paper or a story or a report of some kind. For some, this artifact might takes years to construct, while for others but a short while. But regardless, in a skill-based world schools would be accountable for seeing that 100% of their students had demonstrated that skill at least once by an established moment in a student's life. Students in such an environment would leave school knowing that whatever differences exist between their abilities and another's that they had performed well at least once, greatly increasing their chances for doing so yet again.

Now consider a content-based approach say to reading. Rather than identify the skill and an appropriate level, we instead identify a list of standards to teach and create tests against those standards. The content in those standards consists of lots of stuff, and proficiency against those standards is declared when a certain amount of that stuff can be learned. Success in schools is then determined by the percentage of kids getting enough of the stuff, with the measure being an annual test against the stuff. While the actual skill of literacy is the goal, a system based almost exclusively on content risks declaring a school successful without ever producing a literate person.

Remember that tests are proxies

2007-10-22T08:47:00.001-05:00

Let me be clear on something--tests are good. They have a very real and meaningful purpose and when used for those purposes can be extremely useful. But when used outside or beyond those confines watch out--they will always fail if they are placed in a context that extends beyond the measure. Yet the tendency to use tests in such a foolish manner is awfully seductive and very smart people do it every day.

Remember that tests are proxies for a limited piece of reality--in the NCLB terms for reading, mathematics, and science. Yet the proxy is extended beyond those subject areas to the quality of schools as a whole. And while no one would expect those subjects not to be important, nowhere else in the world will you find proxies extending their reach beyond what they represent without serious repercussions for the organization. Think of the valid proxies we use in business (financial statements), or in schools (grades--whether inflated or not is for another entry): both of them limit their interpretive reach to the things represented. For a business its the activities that support the financial statements; for a student its the course represented by their grade. A grade in PE isn't expected to say much about an English course, and blending my company's results with yours and then asking us to manage to the blended results would be nuts.

Yet that is exactly what we do when we ask a school to manage itself as an institution against proxies that represent only a portion of that institution--no matter how important they are. This is a lesson straight out of the business world--that the proxies to which you are accountable must limit their interpretive realm to what they actually represent or the result will be horrible decisions. It is surprising to me that anyone with that understanding ask a school--the most important institution for preparing students for a life in those businesses--to manage themselves in a manner that would destroy even the best of businesses.

Yet that is what we do.

Proxies for institutional success exist and are actually easier to get at than most people think. More on that later.

The first few thoughts

2007-10-22T07:40:00.000-05:00

One of the most troublesome things in education today is that policy makers actually believe you can motivate and inspire people by insisting that they "try harder." This gets masked in accountability terms that rely on proxy measures to tell us the degree to which educators aren't doing their jobs. Think of it this way--would you be motivated by being held accountable to a measure that didn't respond very well to the changes you make? And yet teaching and testing in the world of content standards does just that.

Testing (and this is an area in which some consider me somewhat knowledgeable) for content is a matter of lining up the content from the simplest to the most difficult and then drawing a line in the sand when someone has learned enough to stuff to get over that line. That's what we call "proficient" in this current world of content standards.

Now teach to that or manage a school to that kind of an end. Change (the lifeblood of any organization) is going to be about deciding what content to drop out and what to include--not about how it all ties together to create literate, numerate students ready to tackle the worlds they will inherit. Thus in the name of accountability we have created an anti-motivational environment where good educational managers are unable to ply their trade and bad ones (or those who are told to do so or risk their livelihood) focus on the content most likely to help their kids get enough of the content to pass the test--even though their may be little or no connection between success in the real world and success on the stated content.

There are answers--stay tuned.