I am one member of a five person board. The opinions I express on this forum are mine only, and do not necessarily reflect the views or opinions of the Escambia County Staff, Administrators, Employees, or anyone else associated with Escambia County Florida. I am interested in establishing this blog as a means of additional transparency to the public, outreach to the community, and information dissemination to all who choose to look. Feedback is welcome, but because public participation is equally encouraged, appropriate language and decorum is mandatory.

Monday, December 8, 2014

Interesting Insights into How We Evaluate Teachers Locally

...“Why would anyone work here with so much at stake?” one teacher related to me recently, a teacher who’s ranking went from “Effective” to “Needs Improvement” because her VAM was developed utilizing a small sample size due to a small stability group because she works in a high-poverty school with tremendous student mobility..

The formula for evaluating teachers is complex.  Not only is it intricate-it can be unfair to teachers in some locations and to those teachers that teach ESE students.

Previously in our district, we had an evaluation system that was a joke, it was horrendous.  And I discussed it frequently because it was so bad; it was almost as bad as a pass/fail civil service evaluation.  Everybody is great, everybody wins-you know the type...  But it was terrible, it did nobody any good, and needed to be scrapped.  Eventually the district put together a much better system that was much more objective.

Recently the state mandated that student test data become a component in the teachers’ evaluation—an idea I strongly support if it is done fairly and correctly.  Under this scenario, not only does the teacher get evaluated based upon the test scores of the students the individual teacher teaches-but this data also has huge consequences and can significantly impact a teachers’ overall rating.  

In some cases, the addition of the test score data (or VAM data),  can take a “highly effective” or “effective” teacher all the way down to “needs improvement” or “unsatisfactory!” (Under state law now, two consecutive teacher evaluations that are “unsatisfactory” can lead to removal of a teacher from the profession—so the stakes could not be higher)

So how can this happen?  I was wondering that too so I had a long conversation with the district’s director of evaluation services so that I could understand how the process works.  Here it is in a 


A teacher in Escambia County has two parts to his/her evaluation; each part is weighted at 50%. 

--Part 1 is generated at the individual school-site level and is very comprehensive, built on a multi-part evaluation modeled after the Charlotte Danielson framework, assigning point value(s) to various teaching attributes- pedagogy, instructional delivery, classroom management, after school participation, continuing professional development, and other school-based observations.  The Danielson model is very comprehensive, very thorough, and very good.

--Part 2 is generated via test scores.  According to the evaluation services department, the state looks at individual students and has developed an algorithm  that accounts for historical achievement, absenteeism, previous test scores, and other socioeconomic factors unique to each student.  The state’s algorithm projects what academic progress a student should attain in a year, and matches actual student outcomes to this projection to assign a “score” to the teacher.  These student achievement scores used for the 50% addition to the teachers’ locally completed evaluations are built using only students that the teacher had in their classrooms the previous year.  This group of students is referred to as the teachers’ “stability group.”   The state determines this because they mandate that thelocal school  district match students to teachers at three points during a year—October, February, and again in May.  The district then publishes the list, and notifies teachers by “flagging” students on the list who are in each individual teacher’s stability group.  Pretty complex, right?

But here’s where it gets even more complex.  

In previous years, if a teacher did not have a large stability group, the teacher was assigned a school-wide average.  The utilization of a school-wide average ameliorated the negative effect of small sample sizes for some ESE teachers and for teachers in some schools where there is high student “mobility” (inner-city schools).  

This year, however, only the students that the individual teacher taught were used, even if in some cases that resulted in a sample size as small as 12, 10 or even as few as 5 students.   A small sample size can skew results badly, and that is why most surveys with extremely small sample sizes are considered invalid.

And here’s where it gets worse.

Suburban schools with good student attendance and decent sized stability groups (18-22) will not be impacted the same way by the way the stability group scores are added in to the overall evaluation;  one or two students having a bad test day, or not reaching the target, won’t sink a teacher out in suburbia.

Not so in the inner-city schools. 

We now use the stability group of students for teachers in inner-city schools, which all but guarantees small sample sizes for calculating the VAM score.  Add to this the tendency for such students in such environments to be frequently absent, tardy, and the recipients of very little support from home, and you suddenly have the potential for a handful of students who are struggling academically to exert huge impacts on the evaluations of the teachers who are trying to help them.  If just one (1) student from a small sample size group tanks on test day—that can badly skew the results for the whole group.  Imagine if all the students do badly and the sample size is small?

And that’s how an ESE teacher with a dangerously small “stability group”—even from a high performing elementary school --can go from “highly effective” to “unsatisfactory”. 
It’s how a teacher from an inner-city school ( with constant student churn) who is rated “highly effective” under the Danielson framework alone can be bumped downward all the way to “needs improvement” when the VAM scores are added in.

So the fact that very few teachers, percentage wise, have contacted the district about their stability group as they’ve been notified, is somewhat surprising.  I think they will pay closer attention to these notifications next year.

I just hope that they don’t disproportionately concentrate teaching attention on the students in their individually flagged stability groups to the exclusion of the other students in their classroom;  I’d like to think no teacher would do this, but could the way we are evaluating  teachers drive them to do this?  That would be a concern.

Or what about this one:  Encouraging some students not to come to school on test day, based upon some notion of how such students might do if they came and took the test.  I hate to think this would ever happen.  But I know the potential is there when the stakes are high....

Unfortunately, this method of evaluating teachers is yet another stressor on the teachers that  already work in challenging schools, and it needs to be fixed.

First and foremost---these teachers need to be compensated, paid an additional stipend to work in this environment, given all the dysfunction, discipline issues, social issues, and now these uneven testing machinations ----if  we don’t pay them more  we are going to see even higher levels of teacher churn out of these schools than we already see.  It will be bad. 

“Why would anyone work here with so much at stake?” one teacher related to me recently, a teacher who’s ranking went from “Effective” to “needs improvement” based upon a small sample size as described above.  This teacher is devastated and she’s actively looking to escape from this school…..
 If this anomaly in the evaluations is not addressed, I don’t know how I could disagree with such a teacher in the inner city; I don’t know how I could disagree with her assessment of the situation—why would she stay??.

Their careers are at stake, so it is understandable that they would want to work where the deck is not so completely stacked against them!


Gulagathon said...

This is really bad, especially for the inner city schools. You gotta wonder a little, is this on purpose to cause more confusion, and to make the progress of inner schools even more difficult than the chaos that already engulfs these establishments. Who would want to be a teacher at inner city schools knowing that there is such a labyrinth of a mind warp to if not drive you crazy, it drives your self confidence in the gutter.

This is just another obstacle for teachers. This will cause more corruption in these schools. Teachers will hope, and even subtly encourage students not to come on test day. But that won't save them. There will be a lot more teaching for the test.

Too much testing, and not enough life skill lessons. Well, I guess they do need some of these prisons filled in the next few years, and they sure will. Education as we know it is collapsing, just like various economies. The only way education will be bailed out, is if a heavy dose of common sense is injected into this stubborn whithering system.

Anonymous said...

The VAM is just one more reason that I am getting out of teaching as soon as I am able.

Jeff Bergosh said...

Anonymous-I am sorry to hear that you feel this way. You are not alone. Three teachers quit one school in a one week period last week, and I heard just yesterday that one of the best 4th grade teachers at an inner-city elementary school I follow closely is quitting and leaving our district to work in Santa Rosa County. Sadly, I see no one calling for change to eliminate this huge attrition. I will continue to focus on this as it appears to be the front lines of the war to save schools, and in my opinion it starts with respecting teachers, not stifling or stymieing them, ensuring they work in safe environments and supporting them and not allowing political correctness run rampant to allow the same disruptive students to remain in classes even as these same students destroy the learning environment for everyone else . We have much work to do or we will continue to see massive churn out of these inner-city schools.
It appears as if other Florida districts are going to bring in Teach For America recruits to fill the shortages in hard to staff, urban, and title one schools where these shortages are intensifying. From the Orlando Sentinel:

Other school districts are seeing higher than normal attrition due to all the pressure that is being applied to teachers and the way their teaching is being micromanaged.

Anonymous said...

"It appears as if other Florida districts are going to bring in Teach For America recruits to fill the shortages in hard to staff, urban, and title one schools where these shortages are intensifying."

How valued this makes me feel as a teacher! Instead of improving the system (or throwing it out) we are going to work you under impossible conditions until you quit and replace you with someone else who will do the same. With any luck they may last more than one year.

I do realize this is a quote from a news source and not Mr. Bergosh's personal feelings, but how blind can we as a state be? I have been teaching 18 years in Escambia County. Things have been going downhill since I started teaching and don't look like they will get any better. As soon as my two children are through the system I will be leaving as well.

Gulagathon said...

This just sucks. How can anybody make a career out of teaching anymore. People used to want to teach for years and years, and make it a long lasting job and career. This system is making it almost impossible in one way or another. It's either the micromanagement of teachers or it's the bad kids. Everywhere you turn there's something to turn you off from truly wanting to be a teacher. I mean, who really likes the way it is. No career is perfect, but geez, this teaching stuff is becoming miserable. This is probably why there are teachers who are working so hard to get into an administrative position. They want to be out of the classroom.

Where is all of this going anyway, and what do they truly want out of this? All of this testing and evaluations is taking away from actual true blue learning. If they just got the discipline straight, and less testing and evaluations, then everything would be a whole lot better.

Anonymous said...

Mr. Bergosh--In regards to your posting, ESE teachers can be evaluated on a minimum of three students. I am one of those teachers and it is unfortunate that my students' growth (in a K-5th grade VE class of 12 students) is only based on three students. One of them is in gen ed full time now and making A/B honor roll--how is that not success or growth? But according to VAM this child didn't make growth. It saddens me to see my colleagues all leaving the ESE field over this evaluation system. I love what I do. No evaluation system will take the joy of teaching away from me. However, I must admit this system is frustrating.

Anonymous said...

I have been teaching in Escambia County for a little over 26 years.
I believe you are mistaken about highly effective rated teachers being knocked down to needs improvement. I learned, at a summer workshop, that MOST teachers at low performing schools are rated Highly Effective in ALL of the areas on the teacher evaluation system. The reason for this is so that when the other 50% school grade comes in then their ratings will be dropped only to Effective. Interesting because I work at an "A" school, (where for everyone's information, is not the "Country Club" people think it is). I don't like this evaluation system because I don't see it as being objective at all. Most teachers at an "A" school DO NOT get all Highly Effective ratings because it is not necessary. I was shocked to see in last year's newspaper when No Teacher in Escambia County was "Highly Effective". If anything, those teachers at an "A" school should all get Highly Effective!!! I think I can tell you that we are an "A" school. It is not because of our demographic, because it is changing, it is because of principals that leave us alone, let us teach how we know is best (not how we are told to by people at the Hall Center and in Tallahassee who are not in the classroom), and have high standards for us and our students. Referrals are handled quickly and justly, parents are notified and bad behavior is not tolerated. I know each year that I will get highly effective in some areas and effective in others. I am fine with that. I am not fine with some teachers being given all Highly Effective just so they won't be knocked down by the results of ONE test!

Anonymous said...

I have been a teacher for almost 10 years. Last school year I taught 3rd grade ESE students. My observation from my principal was superb. I has 8 highly effective marks. In fact, not a single ESE student under my name in 3rd grade failed FCAT or portfolio. You can imagine my shock when I saw my final grade as needs improvement. I went to the Union and changes have been made. I am now labeled as effective.... Stability groups are not visible and I was recorded as teaching 6 students in another grade in another subject I never taught. Was told it can not be fixed because the deadline was in May 2014. If I had seen a report I might have known this, I went to FCAT star and yet again no stability groups show for this year or last. I have decided that teaching is not for me. I put in on an average 12 hours extra a week on campus and countless hours at home doing school related things. What is the point? Sure I used to love my job, I used to love making a difference but I can not work somewhere that shows such disrespect for a person. I am not renewing my teaching certificate in June, So very sad!

Jeff Bergosh said...

Anonymous 12-11 6:13PM-Thank you for the comment-I do not believe that the inner city school teachers automatically receive a rating on the Danielson framework of "Highly Effective" to act as a counterweight to the addition of what is assumed to be low VAM scores. Perhaps I need to look at that but I do know of a teacher that went from effective to needs improvement. She is leaving our district to teach elsewhere. The commonality that I hear and see in speaking with teachers and reading these comments is that the evaluation system is still deeply flawed. At the workshop yesterday, it was discussed among the board and Superintendent that we need to lobby our legislators to change the weighting, giving 75% to locally developed evaluation and 25% to VAM scores. Everyone was in agreement that this sort of an adjustment could help to fix the issue.

Anon 7:13-Thanks for your comment and the positive outlook and attitude. Please know there are people working to make these evaluations as fair as they can be

Anon 11:02-I understand the frustration but please know we are trying to fix it. If I might, I'd suggest that you watch our workshop on the website from yesterday under the heading "district driven assessments" we have a strategy to fix these assessments and we won't be letting up until we get it done.

Anonymous said...

Oh the humanity.