Monday, September 28, 2015

Educational Testing and the Elephant in the Room by William Mathis


Editor’s note: This commentary is by William J. Mathis, who is the managing director of the National Educational Policy Center, a member of the Vermont State Board of Education and a former superintendent of schools. The views expressed are his own.
Measuring the effects of education is like the apocryphal group of blind people describing an elephant based on the part they feel. People with assorted predispositions touch different parts of the elephant and shout how the truth of their dearest theory is now confirmed. Alas, for the educational elephant, the proclamations are almost invariably of the “The scores are too low and the costs too high” genre.
The past weeks have provided the oracles of numerology with two totems to brandish: the release of the SAT and the SBAC testing consortium results. The SAT is one of the two major college entrance examinations while the Smarter Balanced Assessment Consortium (SBAC) is one of the tests adopted by a group of states to meet federal requirements. Despite a strong public backlash against too much testing, such “college and career readiness” tests are bally-hooed by the federal government as essential.
Most prominent among the critics are media lamentations about the low student “proficiency” rates on the SBACs and the “10-year low” on the SAT tests. Quickly recognizing a media opportunity, pro-privatization think tanks also seized the initiative. Thirteen years into the No Child Left Behind era, Michael Petrilli of the right-leaning Fordham Institute was quick to double-down on this demonstrably failed strategy, calling for more standards-based reform initiatives in high schools.
But, let’s take a closer look.
It is true that the SAT scores are at a 10-year low. They re-normed the test 10 years ago. (Vermont scores remain basically even with last year.) However, what is missing from the story is that scores were consistently increasing from 1980 to 2000. They are going to re-norm it again next year so, like the state tests, long-term trend comparisons will be impossible. Yet, this is not the important point.
Test-based reforms provide us with the illusion and the excuse that giving tests, publishing scary numbers and talking-tough about “accountability” will close achievement gaps and educational opportunity gaps. They won’t and that’s the elephant in the room.

The SAT is not a required test, it is voluntary and the participation rate varies. Nationally, there are more test-takers than there were before. Also, the composition of test-takers has changed. To encourage economically stressed children and children of color to go on to higher education, many states are paying for all students to take the tests. As more of these students with fewer educational opportunities take the tests, the overall average comes down – even as the average scores of students in that subgroup go up. The irony is that as more students of color graduate and aspire to higher education, the overall effect of these gains is to push SAT scores down.
The SBAC results have also just been released. These tests were built to measure the controversial Common Core. For each of the grade levels (3-8, 11), a “proficiency” cut-score was established, and each student was scored as proficient or non-proficient depending on whether they were above or below the cut-score. For Vermont, the percent of students who scored proficient in math ranged from 37 percent to 52 percent depending on grade level. That’s not very high. For English Language Arts, the proficiency percentages were in the 50s. What this means is that according to the cut-off criteria, almost half of the students were declared to be not “college and career ready.” (Most of the 17 other states that administered the SBAC scored lower than Vermont.)
Are these scary numbers accurate? We really don’t know. The cut-scores have not been validated. The scores of those who were successful in colleges and careers have not been compared with those who were not. To be sure, the tests were subjected to a variety of rigorous examinations by “experts.” Unfortunately, this is just a collection of opinions. Setting the cut-scores was a political decision unconnected to the readiness claims. The effect is that commercial textbook and testing corporations who have a huge financial interest in scary numbers publish a lot of scary numbers.
The National Assessment of Educational Progress (NAEP), which is given to a random sample of students, gives us a steadier long-term benchmark. It tells us that for reading and mathematics, the nation has improved at a relatively consistent rate over the past 30 years. The good news is that lower performing sub-groups have increased at the same pace as higher-performing ones. The bad news is that, they are not catching up. As a nation we continue to systematically underprovide for our neediest children. While we don’t know if the tests measure “college and career ready,” we do know they measure poverty quite well. One in three children in the U.S. lives in poverty and the needy are increasingly clustered together. And as the income gap increases, the achievement gap increases.
Sadly, test-based reforms provide us with the illusion and the excuse that giving tests, publishing scary numbers and talking-tough about “accountability” will close achievement gaps and educational opportunity gaps. They won’t and that’s the elephant in the room.

Thursday, July 9, 2015

Equity in Vermont Public Education? Hardly!

Equity in Vermont Public Education? Hardly!
By Jay Nichols

Do we have equity of opportunity for students in Vermont? Do all students have the same access to high quality teachers? Do all students have the same opportunity of programming? As a state, are we committed to providing an outstanding educational experience for all Vermont children? The evidence, from my perspective, is that the answer to all four questions is an emphatic “no!”

First the good news, in Vermont, those of us who believe all children in our state deserve equal opportunity to a high-quality public education experience applaud the Brigham decision, followed by Act 60 and Act 68. This legislation, while imperfect, helped poorer communities by forcing a statewide property tax system so that all Vermonters (at least those with property) helped pay for all students. In essence, the legislation has led to equality of property value worth to help pay for public education. As important as this is, it only scratches the surface of the problems associated with educational equity in Vermont.

We have three major issues that stand in the way of equalized educational opportunity for all students: the first is related to population density and our archaic public education delivery system; the second to the disparity of wages paid to educators across the state; and the third to how we are addressing the associated student challenges that manifest much more frequently in areas of high poverty.

Small school systems often offer less opportunity for students than larger systems. This is especially true at the high school level. Students in really small high schools have less course offerings than students in larger high schools. How is it equitable that one high school can offer a dozen Advanced Placement classes, for example, while another in close proximity cannot afford to offer more than a couple? In elementary schools, less opportunity most often manifests in less art, library, technical education, foreign language, physical education, and other important offerings that end up being reduced or eliminated as communities struggle to support their school by passing budgets.

Across much of the state, our ridiculous governance system gets in the way. As I write this, I know our legislature is examining our current system. We must have legislation that compels or seriously incentivizes the sharing of resources to increase opportunities for our students. I am not suggesting closing schools – I am suggesting that schools in towns joining each other become school districts so that together they are all responsible for all of the students in their area. We know that this will lead to more opportunities for students, save taxpayers money, and help us sustain our education system moving into the future. So, why not do it? Of course, the answer is always “local control.” Perhaps it is time to redefine “local” to something a little more flexible than an arbitrary town line. Remember, all Vermont children are our future – not just the ones who live in same town as we do.

Much of the research on the relationship between teacher effectiveness and student learning points to the critical import of having the most effective teachers serving the most needy students. Students in high poverty areas are statistically much more likely to need the best teachers possible than students in relatively affluent communities. At worst, it would make sense that children across the state have the same opportunity to have high performing teachers. Our current financial system is a disincentive to teachers working in needy schools. I’ve been an administrator in some of Vermont’s poorest communities and in Vermont’s richest communities. It is increasingly disturbing that teachers who work in poorer schools are paid much less on average than teachers of the same experience working in richer schools. I’ve worked with great teachers in both settings. However, many really good teachers in the poorer districts move to richer districts because they know they will make a lot more money.

 In addition, research demonstrates that it is much easier to teach children from affluent backgrounds than it is students from low-income backgrounds. More money? Easier job? Hard to blame people for making that jump! If we are really serious about ensuring equality and supporting all Vermont children, perhaps we should pay the teachers more who choose to work in high poverty schools – or, at the very least, equalize teacher pay and benefits across the state. State workers are paid the same for the same classification. I believe this also holds true for the Vermont State Police. Why not educators? Do we really believe that some students are worthy of higher paid teachers simply because of where they reside? It is time Vermont started to walk the walk and not just talk the talk around equity issues in public education.

I believe it is time to have a statewide teachers contract. Great teachers are great teachers and they should not be penalized simply because they choose to work in poorer communities that can’t afford to pay as much as more affluent communities. Phase in a system over time. Have cost of living differences, if you must. But again, if Vermont is serious about the equity of quality instruction for ALL students than we need to compensate teachers in an equitable manner across the state. If we don’t have the courage to walk the talk than we might as well stop talking about equity and saying we value all children’s education equally.

Finally, we arrive at the third issue – the tremendous problems with challenging students. Teaching is tough. Our teachers in this state are, for the most part, worth their weight in gold. We have more students living in poverty then ever before in this state and in this nation. We have more students coming to our schools – especially our poorer schools – with greater needs. The trauma that many of these students have been exposed to is tremendous. Students are entering our schools at record numbers having been physically abused, sexually abused, emotionally abused, living in violent-laden homes, living with drug-addicted parents, living with only one parent who is barely getting by, or not living with either biological parent. Many of these students end up with an individual aide – contributing to higher costs for school districts. The truth is often these aides are no help at all in helping the student learn – but are a great help in keeping an emotionally disturbed student safe, and equally important, keeping other students safe from an emotionally disturbed student. 

In too many cases, the only real structure and safety children have in their lives is the school; the only healthy meals they get come from the school; and, the only real hope they have to break the poverty cycle comes from education. I fear we are rapidly moving toward a time when our schools will have to be full service agencies for many of our students. With morning programs, after school programs, and summer programs, we already provide much more services than schools have ever done before in our state. I have no answer for this third problem. Students who have been mistreated need extra support and that costs time, money, and other resources. You either pay at the school, mental health, child services end or you pay at the other end with dropouts, crime, and increased prison numbers.

How do we provide equity in Vermont public schools? Change the delivery system to one that values opportunities for students above adult convenience and adult perception of local control. Pay teachers with equitable experience the same amount regardless of where they teach in the state. Provide schools and families with the social service and mental health supports they need so that children, especially those living in poverty, have a fair chance for a successful life. These steps will make Vermont the education state for ALL of our students.

Jay Nichols is the Superintendent of Schools for Franklin Northeast Supervisory Union and the President of Nichols Education & Leadership Consulting.

Thursday, January 22, 2015

The Iceberg Effect: International School Comparisons that really matter!

New study finds U.S. has the world’s most educated workforce—but students face unparalleled levels of poverty, inequity and violence
Washington, DC. January 20– A new study released today challenges the practice of ranking nations by educational test scores and questions conventional wisdom that the U.S.educational system has fallen badly behind school systems abroad.
In their report, School Performance in Context: The Iceberg Effect, the Horace Mann League (HML) and the National Superintendents Roundtable examined six dimensions related to student performance—equity, social stress, support for families, support for schools, student outcomes, and system outcomes—in the G-7 nations (Canada, France, Germany, Italy, Japan, the United Kingdom and the United States) plus Finland and China. They then examined 24 “indicators” within those dimensions.
Of the nine nations, the United States remains the wealthiest with the most highly educated workforce, based on the number of years of school completed, and the proportion of adults with high school diplomas and bachelor’s degrees.
“Many policymakers and business leaders fret that America has fallen behind Europe and China, but our research does not bear that out,” said James Harvey, executive director of the National Superintendents Roundtable.
Despite high educational levels, the United States also reflects high levels of economic inequity and social stress compared to the other nations. All are related to student performance. Measures included rates of childhood poverty, income inequality and violence. For example, in American public schools today, the rate of childhood poverty is five times greater than it is in Finland. Rates of violent death are 13 times greater than the average for the other nations, with children in some communities reporting they have witnessed shootings, knifings, and beatings as “ordinary, everyday events.”
The study is a unique analysis, which for the first time compares K-12 education internationally on an array of social and economic indicators, not just test scores. The goal was to look at the whole iceberg, not just the tip—and provide a clearer snapshot of each country’s performance, including its wealth, diversity, community safety, and support for families and schools.
Some key findings:
 Economic Equity: The United States and China demonstrate the greatest gaps between rich and poor. The U.S. also contends with remarkably high rates of income inequality and childhood poverty.
 Social Stress: The U.S.reported the highest rates of violent death and teen pregnancy, and came in second for death rates from drug abuse. The also one of the most diverse nations with many immigrant students, suggesting English may not be their first language.
 Support for Families: The U.S. performed in the lowest third on public spending for services that benefit children and families, including preschool.
 Support for Schools: Americans seem willing to invest in education: The U.S. leads the nine-nation group in spending per student, but the national estimates may not be truly comparable. U.S. teachers spend about 40 percent more time in the classroom than their peers in the comparison countries.
 Student Outcomes: Performance in American elementary schools is promising, while middle school performance can be improved. U.S. students excel in 4th grade reading and high school graduation rates, but perform less well in reading at age 15. All nations demonstrate an achievement gap based on students’ family income and socio-economic status.
 System Outcomes: The U.S. leads these nations in educational levels of its adult workforce. Measures included years of schooling completed and the proportion of adults with high-school diplomas and bachelor’s degrees. American students also make up 25 percent of the world’s top students in science at age 15, followed by Japan at 13 percent.
“Too often, we narrow our focus to a few things that can be easily tested. To avoid that scoreboard mentality, we need to look at many measures important to shaping our future citizens. Treating education as a horse race doesn’t work,” said HML President Gary Marx.
A call for more nuanced assessments
American policymakers from both political parties have a history of relying on large, international assessments to judge United States’ performance in education. In 2013, the press reported that American students were falling behind when compared to 61 other countries and a few cities including Shanghai. In that comparative assessment—called the Program for International Student Assessment—PISA controversially reported superior scores for Shanghai.
“We don’t oppose using international assessments as one measure of performance. But as educators and policymakers, we need to compare ourselves with similar nations and on a broader set of indicators that put school performance in context—not just a single number in an international ranking,” said Harvey.
“Our study suggests the U.S. has the most educated workforce, yet students confront shockingly high rates of poverty and violence. Research shows that those larger issues, outside the classroom, are serious threats to student learning,” noted HML Executive Director Jack McKay.

Wednesday, December 10, 2014

More Opportunities for Kids AND Save Money?

More Opportunities for Kids AND Save Money?
By Jay Nichols

Last year, I testified in front of the General Assembly in favor of a proposed bill that would have consolidated Supervisory Unions (SU’s) into Supervisory Districts (SD’s). What is the difference between the two, you might ask? To me, the difference really comes down to two things. First, SD’s can offer more educational opportunities for students and families. Secondly, there is more potential for cost savings and better-cost sustainability in a Supervisory District for taxpayers. I believe that the issue really became convoluted last year and many people were worried that consolidation of SU’s would mean closing of schools. The bill that was proposed did not call for the closing of any schools and would have left this up to local communities. However, the bill would have changed what “local” means.

By way of example, let’s say the schools of Franklin Northeast became one Supervisory District instead of five separate school districts with eight school buildings. If this were to occur, Bakersfield, Berkshire, Enosburg, Montgomery, and Richford would be a single school district. Local would mean all five towns together.  We would share all services. The reduction in duplication of services alone makes this worth considering. We would have one budget voted on by all voters co-mingled throughout the new district. Employees could be shared amongst schools, as needed, to benefit students. We could provide more opportunities for students in terms of sharing resources.  For example, the two high schools could share one teacher for specialized areas with lower enrollment such as advanced mathematics and world language courses. Right now with each town district serving as an employer we don’t have that option. The teacher’s  union in each town has the right to the job – this leads to hiring more teachers than numbers indicate we should have simply because each town obviously wants to offer high quality offerings to all of their students. If we were one district, we could share staff similar to how a school can have a teacher teach more than one grade or subject.

Taxpayers are hurting in Vermont. I believe we can reduce, or at least slow down, the rising costs of education spending in Vermont. Remember, we have a statewide education finance system. Although, FNESU is one of the lowest spending Supervisory Unions in Vermont as evidenced by both our budgets and per pupil spending over the last half-decade, our taxpayers are at the mercy of ever increasing costs across the rest of the state. Many of these costs could be mitigated by districts consolidating to save costs – and in some extreme cases – the closing of some very small schools, ones with less than 50 students, for example. Could we cut costs within Franklin Northeast if we became a single school district? I believe we would. However, I don’t believe it would be a great deal of money. I see savings more at the high school level by sharing personnel as well as some savings in transportation, which would come by allowing students in elementary schools to go to schools closer to where they actually live regardless of where they reside. Right now, we have a number of students who are on the bus for a long time, and physically live closer to another one of our elementary schools than the school they actually attend.

Our students do not have equity of opportunities across the state or even within our local area. For example, students that attend Enosburg Falls High School have more general elective, and advanced placement (which more and more colleges/universities are looking for when admitting students), course choices than students that attend Richford Junior-Senior High School. If we were one district, students at Enosburg and Richford could take classes at either school, share classes via technology, and have access to more educational opportunities.

In closing, I firmly believe we could unify districts, better serve our students, save money, and create a more sustainable educational funding system for our taxpayers, while maintaining more local control than any other state in the United States. Or we can continue to keep doing what we have been doing since 1892 with fewer opportunities for some students based on where they live. In a weak economy … with declining enrollments … and students who clearly need more opportunities to be college and career ready … can we continue to keep doing the same thing we’ve always done?

Monday, November 24, 2014

Sec. Holcombe's Memo on testing to SBAC State Leaders.

TO: SBAC Governing States
FROM: Rebecca Holcombe, Vermont Secretary of Education
SUBJECT: Limitations of performance categories for supporting improvement in learning and assessing school effectiveness
DATE: November 2, 2014

Our purpose in meeting today is to approve performance thresholds on the SBAC assessments in compliance with the federal requirement that States report on student performance in terms of performance categories and review the “percent
proficient.” Vermont does not support the use of performance thresholds as a valid means of communicating about performance. In addition, as of yet we have little empirical evidence related to the validity of the proposed cut scores for actually discriminating between those who are “college and career ready” and those who are not.

Our concern is all the more relevant given the high stakes and sanctions attached to performance relative to these threshold scores under federal policy. This is all the more worrisome given extensive research on the effect of persistent inequality on school outcomes. Since Vermont schools, despite being high scoring relative to the nation, are already considered “low performing” under NCLB, this will have little material impact in Vermont. However, I am sure it is a concern for other states at the table, where schools will predictably fail due to problems that are bigger than the tremendous efforts their teachers are making to improve the learning of our children. As Gary Orfield recently wrote:

“Setting absurd standards and then announcing massive failures has undermined public support for public schools. . . . We are dismantling public school systems whose problems are basically the problems of racial and economic polarization, segregation and economic disinvestment.” (Educational Researcher, August/September 2014, p.286)

Instead of reporting in terms of performance categories, we could report performance in terms of scale scores. A solid body of empirical research suggests that scale scores provide more complete information on performance and are more useful for the purpose of informing improvement efforts. We are indebted in particular to the research of Andrew Ho1 of the Harvard Graduate School of Education in coming to these conclusions:

1.     Cut scores are arbitrary: Cut scores are set through human judgment in a standard setting process, and are therefore subjective determinations of “appropriate” levels of performance. SBAC is following best practice to set these scores, but there is no truly objective way to set threshold scores. As SBAC tests are new, these performance levels are not yet predictive of future outcomes. In fact, by design, SBAC is a test of mastery of established curricular standards. We can only hypothesize that it also captures elements of cognitive and other skills that will be useful in communities, college and in careers.
2.     Proficiency bands reward schools who push students over band levels and provide no recognition for growth within bands: Any analyses of trend or magnitude of score gaps depend on where the proficiency thresholds sit relative to the distribution of test scores. For example, a school whose students initially score far below proficient might have very large gains in mean scale scores, but if the students are still scoring far below the threshold for proficiency, these remarkable gains will not be captured by reporting in performance categories. In contrast, a school that demonstrates very modest gains (perhaps one point average scale score) may see a very large increase in the number of students scoring as proficient, if its students, on average, are scoring very close to the proficiency threshold to start. In this case, the second school would appear to be more effective, while in truth the first is the more effective school. Thus, reporting in performance categories distorts and misrepresents the true story of improvement.
3.     Comparisons of the performance of subgroups (e.g. students in poverty vs. more affluent students) are distorted in the same way comparisons in the performance of schools are distorted.

Reporting in performance categories has a public appeal: We like the apparent certainty and clarity of being able to categorize our students as “proficient.” However, this misrepresents the underlying complexity of achievement and contributes to simplistic policies that make it difficult to achieve our public purposes.

Vermont is present at this exercise because we are compelled to comply with threshold reporting by NCLB. However, the focus of state-level reporting in Vermont will be scale scores, and in particular, mean scores and changes in individual and mean scores. We believe this will support more responsible use of test data to inform improvement efforts at the school, district and supervisory union level. We encourage our SBAC partners to also report using scale scores at the state level, so that we can collectively emphasize the commitment we share with respect to constructive use of assessments to support systematic improvements in learning.

We also invite our SBAC partners to join us in transparent discussions about what we currently can and cannot infer about students’ “college and career readiness” based on SBAC scores.

1 Ho, Andrew Dean. (2008). The problem with proficiency: Limitations of statistics and policy under No Child Left Behind. Educational Researcher. 37, 6, p. 351.


Tuesday, October 28, 2014

VSBA Healthcare Reform Analysis/Vermont School Districts

The VSBA has just released a report detailing the financial impact of health care reform on Vermont’s school districts. This issue is the perfect nexus between two of our state’s most pressing policy issues—the implementation of health care reform and rising property tax rates.

The VSBA commissioned the study in order to provide its members with the most accurate information about the current cost of health care and the potential savings that could be achieved if school district employees’ health care benefits were more closely aligned with the benefits available to all other Vermonters.

The VSBA is interested in this issue for three reasons. 
Change is coming – through full implementation of federal and state health care reform.  School board members need to be prepared for these changes.
Health care for employees is a very significant education cost in school budgets. The current cost of health care for public school employees is contributing to rapidly rising property taxes and limiting the ability of schools to invest in the education of our students.
Vermont’s school districts provide very generous health insurance benefits to their employees – among the most generous available to any Vermont employees in either the public or private sectors.   Any change in health care which brings teacher benefits more in line with the majority of Vermonters is likely to result in substantial savings to school districts and the property taxpayers that fund most school expenditures.
The findings of our analysis show that the magnitude of potential savings is substantial:
If the state transitions to Green Mountain Care (single payer), the savings could be as high as $120 million a year. This translates into about 12 cents on the property tax rate.
Should Green Mountain Care not come to pass, school districts could still save close to $39 million per year if all school district employees moved to a Gold Plan on Vermont Health Connect.
School employees have been largely insulated from the health care costs and challenges that all other Vermonters have been experiencing for over a decade.  It’s time to move toward a health care system where the coverage school employees have is comparable to the high quality coverage available to all other Vermonters.

We will be continuing this conversation Friday morning at our annual conference in Lake Morey and throughout the fall.  If you have any questions about the report, please contact Nicole Mace, VSBA General Counsel at

Tuesday, October 14, 2014

Vermont School Boards Association Press Release on South Burlington Teachers Strike

South Burlington School Board Leading Critical Conversation

A critical function of a school board is to negotiate contracts that are responsive to the needs of the taxpayers and students served by a school district and that are also fair to the teachers and staff employed by that district.

 The South Burlington School Board, after seeing two years of significant property tax rate increases impact the community, offered a compensation package to their teachers that included an innovative health care proposal that would have saved both the taxpayers and teachers money. The health care plan offered by the Board was not only less expensive, it provided fundamentally the same level of health care coverage employees currently have.

 When the South Burlington Teachers Association refused to accept the Board’s offer of a more cost-effective health insurance plan, it made no alternative proposal that would offset some, if not all, of the incremental expense to taxpayers of preserving the current healthcare benefit or the rationale to forego savings with an alternative plan. In doing so, the Association is demonstrating that it is out of touch with growing concerns about property taxes.

 Property tax rates, and property tax burdens on many Vermonters, have grown substantially in recent years. The cost of health care to public school districts is contributing to these rapidly rising taxes. Since school districts spend nearly 20% of total compensation on health insurance benefits, it was entirely appropriate for the South Burlington Board to offer an alternative health insurance package in an effort to curb increases in property tax rates.

 School employees have been largely insulated from the health care costs and challenges that all other Vermonters have been experiencing for over a decade. Vermont’s school districts provide very generous health insurance benefits to their employees – among the most generous health insurance benefits available to any Vermont employees in either the public or private sectors.

 Vermont’s public school employee plans have been materially unchanged in two decades. As different plan designs became available to schools, local unions have actively resisted those new plans even being offered to employees.

 Local unions have also been resistant to school employees having a greater share in paying the premium costs of these generous health plans. Agreements negotiated between boards and the unions have often resulted in increases in salary in order to offset any small increases in employees’ contribution to health care premiums. This dynamic has left school districts, and therefore property tax payers, bearing the lion’s share of the cost of health care that has consistently outpaced the cost of living.

 The South Burlington School Board ought to be commended for its efforts to respond to the concerns of its community by crafting a health care proposal that would have saved taxpayers and teachers money. The fact is, health care reform means change is coming to all Vermonters. The South Burlington Board is leading an important conversation with their community about the need to adjust our current health benefits structure for public school employees.