 |
A weekend interview with Miami-Dade schools superintendent Alberto Carvalho
It didn't take long after Florida school districts received their late FCAT results for questions to arise about whether the scores were accurate. Miami-Dade superintendent Alberto Carvalho helped organize a group of some of the state's largest districts to publicly challenge the results and call for an independent review of the test. Carvalho spoke with reporter Jeff Solochek about how the questions first came about, why the districts joined forces and what he expects to happen in the end.
I am really interested in how you came to find the situation with the FCAT scores. I understand that you were one of the leaders in determining that there were problems.
I was one of about five or six superintendents that had conversations about the anomalies we detected. The way I personally got to develop a concern regarding this matter was, after a release of NAEP data ... we were a participating district, and according to NAEP, which is the nation's report card, Miami-Dade outperformed every other urban district in America in reading and math. Subsequent to that, we received the FCAT third grade and then the full release of the FCAT data. That data also revealed that our students performed exceptionally well compared to the state. So our rate of growth in reading and math was five times and two times the rate of growth for the state, respectively.
So we celebrated that. We were very surprised as we began to analyze the data in more detail. What we do is we look at the accountability formula and we understand clearly how points are assigned. And we realized that notwithstanding the very healthy growth for Miami-Dade children, that there were specific anomalies that our data people recognized ... in grades four and five in elementary school, primarily in reading, to a lesser extent in math, and senior high school ninth and tenth grade. Where the anomalies were identified were primarily in the gains from last year - 2009 - to the current year - 2010. The gains account for 50 percent of the points in the school grade formula.
So what we noticed was, overall students did very well in Miami-Dade. They outpaced the rate of growth for the state as a whole, as I noted earlier. But the learning gains, particularly for the lowest quartile of students, were so out of joint with the previous trends that alarmed us.
Did you worry at all that by bringing this up you might throw a wrench in the entire FCAT system?
I hope it doesn't. Let me wrap up your question. The anomalies were detected by data people, statisticians. And data people began to talk with data people across the state. And they began to ask questions - are you seeing the same trends that we are seeing? Are you seeing the same anomalies? And data folks bumped up these concerns to superintendents. I know some superintendents began contacting the Department of Education independently with these questions. And a group of superintendents - I think at that time a group of five or six superintendents ... basically felt very, very strongly about our collective position on a number of issues. That's when we decided to raise the questions with the Department of Education.
To answer your second question, actually, we decided to do this to protect the integrity of the state's accountability system. I believe if we detected anomalies, they ought to be corrected in fact not to implode the accountability system but to protect it. And the only way you protect the integrity is by ensuring that there is an honest and transparent accounting of student learning.
Do you find that the state's reaction is one that gets to that transparency? I know the questions of HUMRRO and its connection to Pearson immediately came up.
Yes. We had a concern about that. And that's why we urged the commissioner and the state Department of Education to consider a true third-party independent entity that would conduct the audit. And I believe the commissioner who, by the way, has always listened to the concerns of the state superintendents, and I believe he will do it again as he is doing it again. So in addition to that first entity that has been working with the department all along, he has commissioned another entity to conduct this third-party independent audit. And I am quite frankly appreciative of the fact that he has expanded the audit, the examination, dramatically to include an audit of the design of the instrument going back to 2007, ... the actual scores over time, as well as reliability and validity issues that have been questioned this year.
What happens if the results come back and they show that there are huge problems?
Well then I believe with so much riding, with so many consequences which result from this data with real human impact, specific to the appointment of principals, in this era of accountability particularly as we go into school improvement grant requirements, as well as Race to the Top requirements, I think if the anomalies are validated, then it is incumbent upon the state to provide solutions. Immediate solutions as well as long-term solutions.
What I mean by immediate solutions are solutions that do not penalize students and schools and teachers - educators across the board - because of technical deficiencies in the process. The state did this a few years ago when it encountered similar anomalies in third grade performance. So that year they discounted the third grade results. There are ways that, using multi-year data and providing a bridge this year without relying on this year's data if it is compromised.
Long term, I think the state needs to think long and hard. If this is in fact a permanent fix for the long haul, the state ought to bring together some of the stakeholders from across districts, individuals who understand and know data, listen to the recommendations on how some of the elements of the accountability system can be improved. ...
I am one, my position when I was asked to chair the Race to the Top task force, and I have been asked once again to work with the commissioner, the chancellor, FEA president Andy Ford and the governor on the Phase 2 application of Race to the Top, I have been very clear. I support teacher quality. I support accountability. I support increased standards. But I am also saying that in the global context of a system that adequately, accurately and transparently captures it all. So that if in fact we are going to look at teacher performance based in part on student achievement data, that our confidence in that data is unquestionable.
Does this affect the Race to the Top application that you were working on?
I don't think so. Because I believe that the commissioner is extremely interested in taking on this issue and erasing any questions to a valid process that may exist regarding the validity and reliability of this data. I understand that mistakes happen. I understand glitches occur. But I think ultimately we're measured on our ability to respond to them, rectify them and move forward. And I believe that is what the commissioner is in the process of doing.
What about the schools and the students? Because like you said, a lot is riding on this, yet there are questions about the results.
We don't have time to wait. That is why we urged the commissioner to indefinitely postpone the release of the school grades. After all, we were not expecting grades for senior high schools to be released until some time in October or November because of the new high school grading formula ... So we urged the commissioner, 'Take your time and do it right. Don't rush and get it wrong. ... Release the school grades after this careful examination. And if we don't get them until October, I'm okay with it.' Because basically we have used other data points including interim assessments that we administer here in Miami-Dade, in addition to ACT and SAT data, in addition to many other ways of generating student achievement data that have allowed us to place students in the right courses, in the right grades, in the right programs. We did the same thing with teachers and principals. So as I made my moves ... in my schoolwide reorganization, I didn't depend this year as much on the state data as I would have liked to. I looked at it. I examined it carefully. But I looked at it with a doubtful eye toward fourth and fifth grade performance because that is exactly what I am questioning. And I substituted for that data other internal inputs using the same benchmarks, addressing the same competencies via locally administered assessments that we administer at least four times a year.
Do you think smaller school districts have the same capacity to do what your district, and Hillsborough and some of the bigger ones can in that regard?
Actually, I don't think so. I actually think that is one of the vulnerabilities statewide. That is why after the first wave of concern was raised by the first five or six superintendents who came together, there was a second wave of about 30 districts that joined our concerns. ... They in fact realized they were vulnerable to the same anomalies. But as you and I know, there are some small districts in the state of Florida that depend on consortia to address data needs, to address purchasing power, etc. So I think they came to it a little later, even though some independently had expressed concerns to the commissioner themselves. But I think they are at a disadvantage because some of these smaller districts don't necessarily have research and evaluation departments that can very quickly process massive amounts of data and conduct analysis that led to our identification of the anomalies.
So what can they do to get their kids into the right places, and their teachers and principals to the right assignments? Or is that just something they are going to have to struggle through?
I think in this era of accountability, as we are moving schools through differentiated accountability and school improvement grant requirements, I think all districts in the state of Florida have to use data strategically to make appropriate decisions about the placement of students. I think effective districts, and I think this takes place pretty much across the board, don't rely solely on state data. They also have internal assessments that gauge student achievement more periodically than the one time a year assessment of the FCAT. So the more they have of those instruments available to them, and they have that data, I think they can move toward the appropriate placement of students not necessarily relying solely on the FCAT.
Is there a message that parents and students can take from what is happening with the FCAT and the summer of discontent over it?
The message I have been sending out is ... when our doors open in August, students will not be able to discern any difference between the level of quality of teaching they benefited from last year and that of this year. This is an issue that professionals in the field of education will keep wrestling with, but that will not be noticeable to the students. And I think that's the most important message to the students and to the parents. There will be an evolution no matter what, that hopefully will lead to a stronger, more accountable system in the state of Florida. Unfortunately it comes as a result of some confusion, some doubt. But if this leads to a more accountable, transparent, valid and reliable accountability system, then I think it was worth the pain.
Shame on us if after detecting anomalies, after testing anomalies and getting conclusive results either way, shame on us if we don't do something with it to in fact improve the accountability system in the state of Florida both respecting the effort of teachers and the contributions the students have made.
Post a comment
Abuse Policy
Well said, Superintendent Carvalho, well said.
hopefully this will be the beginning of the end of FCAT and Jeb Bush smoke and mirrors education policy!
get rid of all incumbent board members in August
get rid of all incumbent state legislators in November.
elect Alex Sink and Dan Gelber to the state Cabinet and we'll wipe Jeb Bush, Johnny Trasher and the fascist R's from Florida education forever!
Teachers have the power!
Carvalho is so eloquent, he should have been Pinellas County's superintendent...