June 17, 2010

The Good Old Days

I highly recommend reading the new report released by the New School’s Center for New York City Affairs on "Empowerment and Accountability in New York City's Schools".  It is detailed, balanced, and extremely educational.  Reviewers have focused on the report's conclusion that the DOE's grading system is "deeply flawed", perhaps the report's most important conclusion.  Within the report's 68 pages, though, are some powerful reminders of our system's recent history:


"When I [author Clara Hemphill] visited 30 schools in District 7 in the South Bronx as a reporter for the Insideschools.org website early in Mayor Bloomberg's first term, the schools, with a few noteworthy exceptions, were in a sorry state.  I met principals who routinely called for an ambulance to take an out-of-control child to the nearest psychiatric emergency room because they didn't know what else to do.  The middle schools were chaotic, with children wandering aimlessly in the hallways as teachers lectured to half-empty classrooms.  Some of the elementary schools were sweet, warm places with kindhearted teachers doing their best -- but the children didn't know how to read.  While I saw pockets of good instruction, some parents complained to me that their children were taught mostly in Spanish for as many as five or six years, learning almost no English.  Books and supplies were scarce.


Returning to a dozen of those District 7 schools recently, I found much has changed.  Books and supplies are abundant.  Most of the schools I visited were orderly, with children in classrooms rather than roaming the corridors.  Instruction is mostly in English... Principals are now appointed from the applicant pool selected by Tweed, rather than by the district office.  Some of these new principals have a wealth of talent and experience... The principals... say it's easier to recruit and retain staff largely because teacher salaries are substantially higher than they were before the Bloomberg-era increases...  District 7's test scores started at the absolute bottom in 2002 and made some of the most dramatic gains of any large district in the state..."


Looking even further back, the report tells of the dark side of past governance:


"... District 7 had a long history of hiring driven by patronage and nepotism... According to a 1996 report by the city's special commissioner of investigation, the district superintendent, Pedro Crespo, hired unqualified friends and relatives of school board members and approved expensive junkets and perks.  In one instance, Crespo appointed a principal with a poor command of English who had failed eight licensing exams.  Teachers and principals were pressured to buy and sell tickets for large parties organized to raise campaign funds for local politicians.  School board meetings regularly erupted into shouting matches during which, for example, school board members were accused of stealing computers from the district office.


Reports of corruption and nepotism declined after a 1996 state law limited the powers of the city's community school boards and expanded those of the chancellor.  Still, achievement in District 7 remained pitifully low.  Although overt political influence declined, principals still paid homage to elected officials: In 2002, five District 7 principals made contributions to the re-election campaign of Carmen Arroyo, a longtime member of the state Assembly; in 2005, six principals did, according to financial disclosure reports filed with the state Board of Elections."


To be clear, the report suggests that there is much work still to be done:

"Yet for all these gains, significant problems remain.  While some schools have a rich curriculum, others offer bare-bones instruction narrowly designed to help children pass standardized tests.  Many of the newly hired principals have had minimal teaching experience and almost no administrative experience, and struggle mightily with basics like student discipline.  While middle school attendance has improved, attendance in District 7 elementary and high schools has not improved significantly since 2002... Little progress has been made in special education... and, while high school graduation rates have increased markedly, a number of principals openly acknowledge that their students have met only the bare minimum requirements for graduation and are poorly prepared for college."

May 20, 2010

Brill-ing Down: Adding to Steve Brill’s NYT Magazine Report

UPDATE: Thanks to Aaron Pallas for pointing out an error. The enrollment for Harlem Success Academy in 2008-2009 was 398 students. The enrollment was 276 students in 2007-2008.

Steve Brill’s latest article chronicling the politics of the Race to the Top competition is already making waves. One contentious aspect of the piece is Brill’s comparison of two schools that share the same building: Harlem Success Academy and P.S. 149. We thought it would be helpful to augment Brill’s commentary with additional data. The first table has 2008-2009 demographic data, and the next two have 2008-2009 3rd grade state test scores. (Harlem Success only had this one testable grade.) Information was culled from NY State Accountability Report Cards as well as special education invoices provided to the UFT by the New York State Education Department.

According to this data, Harlem Success Academy does appear to serve less needy students, both in terms of economic status, limited English proficiency, and special education needs.  On the other hand, Harlem Success dramatically outperforms P.S. 149 on 3rd grade test results. 

DEMOGRAPHIC DATA:

Demographic
3rd GRADE TEST DATA: ALL STUDENTS

Testall
3rd GRADE TEST DATA: SUBGROUPS*

Testsub
*There were not enough Limited English Proficient students tested in 3rd grade at P.S. 149 to report a score. There were no Limited English Proficient students tested in 3rd grade at Harlem Success.

May 11, 2010

Closing the Gap: Charter School Special Education Statistics

Last week, the New York State Senate passed a bill that would increase the number of charter schools in New York from 200 to 460. Included in the bill was a provision that charter schools increase efforts to enroll students with learning disabilities — an attempt to appease critics who claim that charters significantly under-enroll students with disabilities.

Yet an examination of data provided to me by the city shows that while charters enroll fewer students with disabilities, the gap is not as large as initially reported by the state teachers union, known as NYSUT. According to the Department of Education data, 13 percent of charter school students have Individualized Education Plans, indicating that they have special needs, compared to 15 percent at traditional public schools. NYSUT reported the numbers as being 9.4 percent at charter schools and 16.4 percent at district schools.

The discrepancy stems from problematic data NYSUT received from the state education department (NYSED). According to NYSED, the number of students with disabilities that a charter school reports enrolling often does not match up with numbers reported by school districts.  As a result, NYSED does not consider their own data to be reliable.

As an alternative, I used a database known as CAPS, which is compiled by the Committee on Special Education.  CAPS includes information about every student in the city who has an IEP, so it provides a more accurate breakdown of the number of special education students at each school. 

I found that the percentage of charter schools enrolling students with disabilities at comparable or greater rates of their traditional public school counterparts increased by 5 percent, from a quarter of schools last year to almost a third of schools this year. Meanwhile, charter high schools actually enroll a larger percentage of students with IEPs than traditional public high schools both within their geographical districts and citywide. Of course, there are only a small number of charter high schools.

I also noticed that the 54 charter schools that are authorized by the DOE enroll, on average, 4 percent more students with IEPs than the 37 charters that are authorized by SUNY's Charter Schools Institute. (City-authorized charters have special education populations of around 14 percent, versus 10 percent for the SUNY schools.) The New York State Education Department authorized just seven charter schools in 2009, which enrolled special education students at about the same level as the city-authorized schools.

A full breakdown of the data is available at the end of this post. To see my calculations as well as special education enrollment by school, you can check out this spreadsheet.

Notes on Methodology:
1. The number of students in need of IEPs can fluctuate as the year progresses. I used the end of the school year data, since the DOE classifies this as the "high-water mark" for special education numbers.

2. To compare charter schools to traditional public schools, I looked at the average IEP rates of traditional public schools in the same district as the charter school, divided by year. So, for example, if a charter school enrolled students in Kindergarten through fifth grade in District 5, I looked at the average IEP rate of kindergartners through fifth-graders in traditional public schools in that district. I did this because IEP rates vary significantly with grade year, with lower rates in the early years (K-2), the highest rates in middle school, and average rates in high school.

3. Since the number of students requiring special education services has increased dramatically over the past five years (see chart below), there has been a significant increase in the percentages of students who need services at both the charter and district schools. Thus, if you look at the data for two consecutive years, you will notice significant increase at both the charter and district level—this is not a data error.

Image001

4. Unfortunately, my data did not differentiate among the kinds of services required by students with IEPs, so I cannot make any inferences about the level of need among special education students at the two kinds of schools. Because of this, I did not include data from District 75 schools, which tend to serve the most severely disabled students.

As always, I welcome your feedback and suggestions in the comments section.

May 04, 2010

April Research Round-Up

This is the second in our series examining the most interesting, most discussed academic research on education published each month. We’re publishing it a little early since May 1st was a Saturday this month. As always, let us know if we forgot anything – or if you have any researchers that you’d like us to add to our list!

Study: Evaluation of the Comprehensive School Reform Program Implementation and Outcomes: Fifth-Year Report Source: Daniel K. Aladjem, Beatrice F. Birman, Martin Orland, Jennifer J. Harr-Robins, Alberto Heredia, Thomas B. Parrish, Stephen J. Ruffini, WestEd Research Report

Results: The Comprehensive School Reform program was designed to improve high poverty, low-achieving schools by instituting scientifically-proven comprehensive reforms. A new report by WestEd researchers finds that of the 7,000 schools that received funding, only a third "selected reform models identified as having a scientific research basis". Overall, the program was "not associated with widespread achievement gains". Indeed, out of the 1,037 CSR elementary schools that were low performing, there were only 47 that showed “dramatic and sustained” achievement gains.

What’s Interesting: Of the schools that did make achievement gains, there was no one strategy that all of the schools had in common that seemed to lead to achievement gains.

Caveat: There was very little cohort data available before 2004-2005, so it was impossible to track cohorts of students. Differences in state standards for testing made it difficult to compare students nationally.

Study: Financial Incentives and Student Achievement: Evidence from Randomized Trials Source: Roland G. Fryer, National Bureau of Economics Research Working Paper

Results: Roland Fryer’s experiment to incentive students via monetary rewards did not lead to measurable increase in student achievement in New York City or Chicago, where students were paid for good test scores or grades in core courses. More effective were two smaller experiments that rewarded students for so-called "inputs" (as opposed to "outcomes" like test scores). These "input" experiments, which compensated students for the number of books read and for attendance and homework, had statistically significant effects on student achievement.

What’s Interesting: The Dallas program—which paid second graders $3 for each book read—had the strongest effect on increasing reading comprehension and vocabulary and was the least expensive program. The effect of this intervention did not lead to negative results once the incentive was taken away—rather, there were continued, albeit smaller, gains in achievement after the program ended.

Caveat: White and Asian students did not experience gains, although the sample size was small, and students not eligible for free lunch improved more than students who were eligible. The gains measured once the program ended only looked at one year of data.

Study: Teacher Quality in Educational Production: Tracking, Decay, and Student Achievement Source: Jesse Rothsein, The Quarterly Journal of Economics

Results: This paper looks at some of the assumptions behind so-called “Value-Added” models of teacher quality ("VAMs") and finds that "the assumptions underlying common VAMs are substantially incorrect, at least in North Carolina". Classroom assignments are not sufficiently random to allow for a causal conclusion in common models.

What’s Interesting: Rothstein makes two main points: (1)Accountability policies that rely on measures of short-term value added would do a poor job of rewarding the teachers who are best for students’ longer-run outcomes; and (2) Models that rely on incorrect assumptions are likely to yield misleading estimates, and policies that use these estimates may reward and punish teachers for the students they are assigned as much as for their actual effectiveness in the classroom.

Caveat: Only looked at data from students in North Carolina.

Study: A Closer Look at Charter Schools and Segregation Source: Gary Ritter, Nathan Jensen, Brian Kisida, Joshua McGee, EducationNext

Results: A report issued by Education Next takes issue with the January 2010 report by the Civil Rights Project which found high levels of racial segregation in charter schools as compared to traditional public schools. The Education Next report argues that by comparing the demographic composition of all charter schools to that of all traditional public schools, the CRP’s report ignored significant differences in neighborhood demographics within school districts. In order to correct for this, Education Next compares the levels of segregation for students in charter schools to that of their TPS counterparts in the central city districts of the 8 largest metropolitan areas in the CRP report. The authors found that the segregation gap narrows from the 20% reported in the CRP report to 10%.

What’s Interesting: Authors demonstrate that the majority of students in central cities, in both charters and TPS, attend highly segregated minority schools.

Caveats: Authors caution that their methodology could be improved one step further, by comparing the demographics of charter schools with the traditional public schools that charter students would have attended had they gone to their local zoned school.

April 29, 2010

In and Out: Charter School Transfers

This is the second post in a series that looks at data from charter schools’ Basic Education Data System (BEDS) reports. This data was provided to us by the New York State Education Department via a Freedom of Information Law request. A full spreadsheet with the data we used is available here.

On Tuesday, the state teachers union released a report that said that charters in New York State had a student turnover rate of 8 to 10 percent each year. While statistics on overall turnover rates are hard to come by, data that city charter schools file with the state shows that one measure of transfer rate for city charter schools—that is, the number of students that transfer out of a charter school during the school year—is 6 percent. To be clear, this necessarily leaves out of the number of students who finished the school year but did not decide to return the following year.

Overall, this rate of transfers has decreased slightly from 7% in 2007-2008 to 6% in 2008-2009. Generally, the longer a school has been in existence, the lower its transfer rate. For instance, the NYC Charter High School for Architecture, Engineering, and Construction Industries had the highest transfer rate—26%--in 2008-2009, but it has only been open for one year. Achievement First Endeavor and Ross Global Institute had the highest rates in 2007-2008, of 23% and 24% respectively. By 2008-2009, these numbers decreased to 15% at each school— numbers that are still higher than average. Some schools, such as Achievement First Crown Heights, Achievement First East New York, Community Partnership Charter School, KIPP Academy, and the South Bronx Charter School for International Cultures and the Arts, reported no transfers during both the 2007-2008 and 2008-2009 school years.

To look at the transfer rates at individual charter schools, you can scroll down the list below.

Although this data gives some insight into the number of students that choose to leave their charter school to go elsewhere each year, it’s not clear that it takes into account the full number of students who leave a charter. This is because the question on the BEDS survey asks charters to record the number of students who transferred out of the school between October and the end of the school year. Students who choose not to return are not counted in this number. Although this gap is somewhat made up by the stability number that charters must report, the stability number only records the statistics for one grade, so it’s hard to generalize school-wide.

In a future post, I will be looking at enrollment statistics to see if they can further illuminate the turnover rate at charters. As always, I welcome your feedback!

April 27, 2010

Charter School Stability

This is the first post in a series that looks at data from charter schools’ Basic Education Data System (BEDS) reports. This data was provided to us by the New York State Education Department (NYSED) via a Freedom of Information Law (FOIL) request. A full spreadsheet with the data we used is available here.

One of the largest issues in the charter school debates has been accusations that charters “counsel out” students who have learning disabilities or who do not adhere to the schools’ strict codes of conduct. While we haven't found comprehensive statistics that track individual students enrolled in charter schools from year to year, the BEDS reports include a “student stability” number that is relevant to this issue. Student stability counts the number of students who are currently enrolled in the highest grade that the charter serves who were also enrolled in the school last year. For instance, if a charter school serves students in kindergarten through 8th grade, the student stability number would look at the number of current 8th graders who were also 7th graders last year.

We found that, on average, charter schools retain 84% of their students, compared to 93% for traditional public schools city-wide. This percentage has remained constant for the past three years but the percentage at individual schools varies widely. Some schools, such as the Beginning with Children Charter School and the Harbor Sciences and Arts Charter School, experience almost no attrition. Others, such as Harlem Day Charter School and the John V. Lindsay Wildcat Academy, consistently lose more than one third of their class. And for many charter schools whose highest grade was 9th, the attrition was noticeably high, probably because many of their eighth graders chose to go to other, perhaps more well-known, high schools.

To better visualize the data, we have created a map that shows all of the charter schools that had applicable data. The size of the dot corresponds to the percentage of students that left the school, and if you hold your mouse over the dot you will be able to see relevant information such as grade studied, number who stayed in the school, and number who left. You can zoom in on certain districts, choose to look at the stability ratio for specific grades, or click to see the stability number of specific schools by using the menu to the right. Unfortunately, this data is only for the 2008-2009 school year – to see the numbers for 2007-2008, you will have to look at the spreadsheet.

It is important to note that these stability numbers only look at one grade in a particular charter school. Furthermore, the BEDS data, while vetted by the NYSED, is not without its flaws, which include the timing of the report (charter schools must report their numbers in mid-October) as well as the lack of substantial follow up by the relevant parties who collect the data.

Nevertheless, we believe that this information provides important insight into charter school stability. As always, we welcome your feedback for ways we can improve and build on this report.

April 23, 2010

Mapping Education

We're again experimenting with the best ways to display data on charters and public schools in New York City. We've assembled a map that has some preliminary data: the location and name of every school in New York City, the grades the school serves, how many students are in each grade, the school's Progress Report Score, and its ranking compared to all other schools in New York City. You can check and uncheck the boxes to the left to see which schools are in certain districts or which schools received certain Progress Report scores. What else should we add? What should we take away? What's hard to understand? Thanks in advance for the feedback!

April 19, 2010

Experimenting: New Data on Learning Environment Surveys

Over here at Curious2, we're experimenting with new ways to show you our data. Check out the alternate view to our latest report on Teacher Responses to the Learning Environment Surveys. The first graph includes individual data points that you can scroll over to get information such as the school name, the principal name, the school's Progress Report Score, and the percentage of teachers that actually replied to the survey. The second graph shows you the four questions we focused on and overall teacher responses to those questions. Let us know what you think!

April 16, 2010

Trust Falls: Teacher Responses to the 2008-2009 Learning Environment Survey

Yesterday, the Post published an article exposing a principal at P.S. 38 who tried to pressure her staff into giving her a good review on the annual Learning Environment survey. This prompted  Joel Klein to respond today that he doubted teachers bowed to principal pressure since the surveys are anonymous. To investigate teacher ratings of their principals, we looked at responses to four questions from last year’s Learning Environment survey:

  1. How much do you agree/disagree? The principal places the learning needs of children above other interests.
  2. How much do you agree/disagree? The principal is an effective manager who makes the school run smoothly.
  3. How much do you agree/disagree? I trust the principal at his/her word.
  4. To what extent do you feel supported by your principal?

We found that the majority of teachers rate their princi
pals highly. For instance, over 85% of the teachers at the schools in the survey agreed that their principal supported them.

PrincipalDataforTableau_15751_image003

However, we found that for the teachers who gave their principal a negative rating, there was a small, but significant, correlation between this low rating and a lower Progress Report score.

In order to do this analysis, we first subtracted out the Environment Score from a school’s overall Progress Report, since this component of the score is based partially on the responses to the questions we have isolated. We then used this adjusted number and compared it to the percentage of teachers who disagreed or strongly disagreed with each of the four questions from the Learning Environment Survey that had to do with principal competence. For each of these questions, we found that there was a statistically significant correlation between a high percentage of teachers who had issues with principal performance and a lower Progress Report Score.

PrincipalDataforTableau_15751_image001
Beyond these numerical results, we observed some interesting trends. For instance, the only five charter schools that had over 50% of their teachers rate their principal negatively were East New York Preparatory Charter School, Bronx Charter School for Children, PAVE Academy Charter School, KIPP AMP Academy Charter School, and Harlem Day Charter School. All of these schools (with the exception of PAVE and East New York, which did not have Progress Report Scores) were in the bottom third of schools citywide according to their Progress Report scores. PAVE and East New York Charter School have both been the subject of media scrutiny due to space fights and corruption charges, respectively. KIPP AMP teachers voted to unionize last spring.

Interestingly, out of the schools that the DOE had slated for closure, only three—KAPPA II, the Academy of Collaborative Education, and the Choir Academy of Harlem—had a plurality of teachers say that their principal didn’t support them.

As always, we welcome feedback for ways to improve or re-examine the data.

April 14, 2010

Charter School Lottery Statistics

Mid-April marks the beginning of the charter school lottery season, and with it, news reports of staggering numbers of applications to schools with limited slots. Already, the Post reported that 3,800 students applied for 588 spots in the Achievement First charter schools. In order to review the results for past lotteries, I submitted a Freedom of Information Law (FOIL) request to the State Education department, who provided us with the Basic Education Data System (BEDS) data that all charters file with the state. I found that applications to charter schools have increased by 50% since 2007, with over 50,000 applications submitted last year. By comparison, enrollment in charters has only increased by 40% to just shy of 40,000 students last year. The chances of getting admitted to a charter school in New York City have declined from an average acceptance rate of 36% in 2008-2009 to a rate of 28% in 2009-2010. A full spreadsheet of the admissions data, with statistics for individual schools, is available here.

Applications To be clear, these numbers only tell part of the story. They do not, for instance, take into account double counting—i.e. the fact that many parents apply to multiple lotteries.  Nor do they offer detailed information about where these applicants live, so it is hard to say if charters get many applicants from far away districts or from close to the school. In fact, charter operators are instructed NOT to list the district numbers of their students on the BEDS form, despite the fact that the information could be easily collected. Despite the drawbacks, this data does offer insights into where applications are the most numerous—and how the number of applications correlates to things like age of the school and its Progress Report score. The 11 schools that had admissions percentages of less than 10% last year were*:

  1. Bronx Charter School for Better Learning
  2. Bronx Charter School for Children
  3. Bronx Charter School for Excellence
  4. Community Partnership Charter School
  5. Community Roots Charter School
  6. Future Leaders Institute Charter School
  7. Grand Concourse Academy Charter School
  8. Icahn Charter School 1
  9. Merrick Academy/Queens Public Charter School
  10. Renaissance Charter School
  11. The UFT Charter School
I found that schools that have been in operation longer generally have lower admissions percentages—and this is a trend that has increased in recent years. Schools such as the Sisulu-Walker Charter School, Renaissance Charter School, and Grand Concourse Academy, all of whom have been open since at least 2004, have had admissions percentages below 15% for the past three years. There appears to be no correlation between Progress Report scores and admission numbers in the following year. Indeed, out of the 19 new charter schools opened in 2009-2010 (for which no scores yet exist) 15 had admissions percentages of less than 50%.

Year opened

I also looked at admissions to charter schools based on neighborhood. I focused on Central Brooklyn, Harlem, and the South Bronx, as they are the areas that have the most charter schools. It has actually gotten easier to be admitted to a charter school in the South Bronx, where approximately 1 in 3 applicants is offered a spot. In Harlem, however, the chances of admittance have plummeted: in 2007-2008, the rate was 40%. Last year, it was 18%. Harlem

In future posts, I will be looking at the BEDS data related to suspensions, class size, and pupil retention. As always, I welcome comments and suggestions for ways to improve my calculations.

*I am leaving out the four Harlem Success Academy schools that were open in 2009-2010 because it is unclear whether the numbers refer to each individual school or all four schools. I will update when I have better information.

Categories

--> -->

My Other Accounts

-->

Or enter your email address:

Delivered by FeedBurner

Blog powered by TypePad
Member since 11/2008
-->