Curious2

In and Out: Charter School Transfers

This is the second post in a series that looks at data from charter schools’ Basic Education Data System (BEDS) reports. This data was provided to us by the New York State Education Department via a Freedom of Information Law request. A full spreadsheet with the data we used is available here.

On Tuesday, the state teachers union released a report that said that charters in New York State had a student turnover rate of 8 to 10 percent each year. While statistics on overall turnover rates are hard to come by, data that city charter schools file with the state shows that one measure of transfer rate for city charter schools—that is, the number of students that transfer out of a charter school during the school year—is 6 percent. To be clear, this necessarily leaves out of the number of students who finished the school year but did not decide to return the following year.

Overall, this rate of transfers has decreased slightly from 7% in 2007-2008 to 6% in 2008-2009. Generally, the longer a school has been in existence, the lower its transfer rate. For instance, the NYC Charter High School for Architecture, Engineering, and Construction Industries had the highest transfer rate—26%--in 2008-2009, but it has only been open for one year. Achievement First Endeavor and Ross Global Institute had the highest rates in 2007-2008, of 23% and 24% respectively. By 2008-2009, these numbers decreased to 15% at each school— numbers that are still higher than average. Some schools, such as Achievement First Crown Heights, Achievement First East New York, Community Partnership Charter School, KIPP Academy, and the South Bronx Charter School for International Cultures and the Arts, reported no transfers during both the 2007-2008 and 2008-2009 school years.

To look at the transfer rates at individual charter schools, you can scroll down the list below.

Although this data gives some insight into the number of students that choose to leave their charter school to go elsewhere each year, it’s not clear that it takes into account the full number of students who leave a charter. This is because the question on the BEDS survey asks charters to record the number of students who transferred out of the school between October and the end of the school year. Students who choose not to return are not counted in this number. Although this gap is somewhat made up by the stability number that charters must report, the stability number only records the statistics for one grade, so it’s hard to generalize school-wide.

In a future post, I will be looking at enrollment statistics to see if they can further illuminate the turnover rate at charters. As always, I welcome your feedback!

Powered by Tableau

Charter School Stability

This is the first post in a series that looks at data from charter schools’ Basic Education Data System (BEDS) reports. This data was provided to us by the New York State Education Department (NYSED) via a Freedom of Information Law (FOIL) request. A full spreadsheet with the data we used is available here.

One of the largest issues in the charter school debates has been accusations that charters “counsel out” students who have learning disabilities or who do not adhere to the schools’ strict codes of conduct. While we haven't found comprehensive statistics that track individual students enrolled in charter schools from year to year, the BEDS reports include a “student stability” number that is relevant to this issue. Student stability counts the number of students who are currently enrolled in the highest grade that the charter serves who were also enrolled in the school last year. For instance, if a charter school serves students in kindergarten through 8th grade, the student stability number would look at the number of current 8th graders who were also 7th graders last year.

We found that, on average, charter schools retain 84% of their students, compared to 93% for traditional public schools city-wide. This percentage has remained constant for the past three years but the percentage at individual schools varies widely. Some schools, such as the Beginning with Children Charter School and the Harbor Sciences and Arts Charter School, experience almost no attrition. Others, such as Harlem Day Charter School and the John V. Lindsay Wildcat Academy, consistently lose more than one third of their class. And for many charter schools whose highest grade was 9th, the attrition was noticeably high, probably because many of their eighth graders chose to go to other, perhaps more well-known, high schools.

To better visualize the data, we have created a map that shows all of the charter schools that had applicable data. The size of the dot corresponds to the percentage of students that left the school, and if you hold your mouse over the dot you will be able to see relevant information such as grade studied, number who stayed in the school, and number who left. You can zoom in on certain districts, choose to look at the stability ratio for specific grades, or click to see the stability number of specific schools by using the menu to the right. Unfortunately, this data is only for the 2008-2009 school year – to see the numbers for 2007-2008, you will have to look at the spreadsheet.

Powered by Tableau

It is important to note that these stability numbers only look at one grade in a particular charter school. Furthermore, the BEDS data, while vetted by the NYSED, is not without its flaws, which include the timing of the report (charter schools must report their numbers in mid-October) as well as the lack of substantial follow up by the relevant parties who collect the data.

Nevertheless, we believe that this information provides important insight into charter school stability. As always, we welcome your feedback for ways we can improve and build on this report.

Mapping Education

We're again experimenting with the best ways to display data on charters and public schools in New York City. We've assembled a map that has some preliminary data: the location and name of every school in New York City, the grades the school serves, how many students are in each grade, the school's Progress Report Score, and its ranking compared to all other schools in New York City. You can check and uncheck the boxes to the left to see which schools are in certain districts or which schools received certain Progress Report scores. What else should we add? What should we take away? What's hard to understand? Thanks in advance for the feedback!

Powered by Tableau

Experimenting: New Data on Learning Environment Surveys

Over here at Curious2, we're experimenting with new ways to show you our data. Check out the alternate view to our latest report on Teacher Responses to the Learning Environment Surveys. The first graph includes individual data points that you can scroll over to get information such as the school name, the principal name, the school's Progress Report Score, and the percentage of teachers that actually replied to the survey. The second graph shows you the four questions we focused on and overall teacher responses to those questions. Let us know what you think!

Powered by Tableau

Trust Falls: Teacher Responses to the 2008-2009 Learning Environment Survey

Yesterday, the Post published an article exposing a principal at P.S. 38 who tried to pressure her staff into giving her a good review on the annual Learning Environment survey. This prompted  Joel Klein to respond today that he doubted teachers bowed to principal pressure since the surveys are anonymous. To investigate teacher ratings of their principals, we looked at responses to four questions from last year’s Learning Environment survey:

1. How much do you agree/disagree? The principal places the learning needs of children above other interests.
2. How much do you agree/disagree? The principal is an effective manager who makes the school run smoothly.
3. How much do you agree/disagree? I trust the principal at his/her word.
4. To what extent do you feel supported by your principal?

We found that the majority of teachers rate their princi
pals highly. For instance, over 85% of the teachers at the schools in the survey agreed that their principal supported them.

However, we found that for the teachers who gave their principal a negative rating, there was a small, but significant, correlation between this low rating and a lower Progress Report score.

In order to do this analysis, we first subtracted out the Environment Score from a school’s overall Progress Report, since this component of the score is based partially on the responses to the questions we have isolated. We then used this adjusted number and compared it to the percentage of teachers who disagreed or strongly disagreed with each of the four questions from the Learning Environment Survey that had to do with principal competence. For each of these questions, we found that there was a statistically significant correlation between a high percentage of teachers who had issues with principal performance and a lower Progress Report Score.

Beyond these numerical results, we observed some interesting trends. For instance, the only five charter schools that had over 50% of their teachers rate their principal negatively were East New York Preparatory Charter School, Bronx Charter School for Children, PAVE Academy Charter School, KIPP AMP Academy Charter School, and Harlem Day Charter School. All of these schools (with the exception of PAVE and East New York, which did not have Progress Report Scores) were in the bottom third of schools citywide according to their Progress Report scores. PAVE and East New York Charter School have both been the subject of media scrutiny due to space fights and corruption charges, respectively. KIPP AMP teachers voted to unionize last spring.

Interestingly, out of the schools that the DOE had slated for closure, only three—KAPPA II, the Academy of Collaborative Education, and the Choir Academy of Harlem—had a plurality of teachers say that their principal didn’t support them.

As always, we welcome feedback for ways to improve or re-examine the data.

Charter School Lottery Statistics

Mid-April marks the beginning of the charter school lottery season, and with it, news reports of staggering numbers of applications to schools with limited slots. Already, the Post reported that 3,800 students applied for 588 spots in the Achievement First charter schools. In order to review the results for past lotteries, I submitted a Freedom of Information Law (FOIL) request to the State Education department, who provided us with the Basic Education Data System (BEDS) data that all charters file with the state. I found that applications to charter schools have increased by 50% since 2007, with over 50,000 applications submitted last year. By comparison, enrollment in charters has only increased by 40% to just shy of 40,000 students last year. The chances of getting admitted to a charter school in New York City have declined from an average acceptance rate of 36% in 2008-2009 to a rate of 28% in 2009-2010. A full spreadsheet of the admissions data, with statistics for individual schools, is available here.

To be clear, these numbers only tell part of the story. They do not, for instance, take into account double counting—i.e. the fact that many parents apply to multiple lotteries.  Nor do they offer detailed information about where these applicants live, so it is hard to say if charters get many applicants from far away districts or from close to the school. In fact, charter operators are instructed NOT to list the district numbers of their students on the BEDS form, despite the fact that the information could be easily collected. Despite the drawbacks, this data does offer insights into where applications are the most numerous—and how the number of applications correlates to things like age of the school and its Progress Report score. The 11 schools that had admissions percentages of less than 10% last year were*:

1. Bronx Charter School for Better Learning
2. Bronx Charter School for Children
3. Bronx Charter School for Excellence
4. Community Partnership Charter School
5. Community Roots Charter School
6. Future Leaders Institute Charter School
7. Grand Concourse Academy Charter School
8. Icahn Charter School 1
9. Merrick Academy/Queens Public Charter School
10. Renaissance Charter School
11. The UFT Charter School

I found that schools that have been in operation longer generally have lower admissions percentages—and this is a trend that has increased in recent years. Schools such as the Sisulu-Walker Charter School, Renaissance Charter School, and Grand Concourse Academy, all of whom have been open since at least 2004, have had admissions percentages below 15% for the past three years. There appears to be no correlation between Progress Report scores and admission numbers in the following year. Indeed, out of the 19 new charter schools opened in 2009-2010 (for which no scores yet exist) 15 had admissions percentages of less than 50%.

I also looked at admissions to charter schools based on neighborhood. I focused on Central Brooklyn, Harlem, and the South Bronx, as they are the areas that have the most charter schools. It has actually gotten easier to be admitted to a charter school in the South Bronx, where approximately 1 in 3 applicants is offered a spot. In Harlem, however, the chances of admittance have plummeted: in 2007-2008, the rate was 40%. Last year, it was 18%.

In future posts, I will be looking at the BEDS data related to suspensions, class size, and pupil retention. As always, I welcome comments and suggestions for ways to improve my calculations.

*I am leaving out the four Harlem Success Academy schools that were open in 2009-2010 because it is unclear whether the numbers refer to each individual school or all four schools. I will update when I have better information.

No Victory Yet for Victory Schools

Educational management organizations (EMOs) are for-profit management companies that provide operational assistance to charter schools, usually focusing on back office functions such as accounting, legal help, and human resources. EMOs frequently come under fire for charging higher management fees than non-profit CMOs and, of course, for being for-profit entities that are possibly more prone to corruption. In New York, 50% of closed charter schools were operated by an EMO at one point in time.

Victory Schools operates 7 of the 9 EMO charter schools in New York City. (The other two – Harriet Tubman Charter School and Brooklyn Excelsior Charter School—are run by EdisonSchools and National Heritage Academies, respectively.) On average, each Victory charter school paid around $2,163 per pupil for the EMO’s services. This comes out to be around 17% of their total per pupil revenues from the State.

Meanwhile, the five Victory Schools had an average Progress report score in 2008-2009 that placed them in the bottom 35% of all charter schools and in the bottom 20% of schools citywide. The two schools that did not have scores – the NYC Charter High School for Architecture, Engineering, and Construction Industries and the Bronx Global Learning Institute for Girls – both opened in 2008 and thus did not receive progress report scores. Both, however, received evaluations of “underdeveloped” from the NYC DOE.

The full spreadsheet, with our calculations, is available here. For more on management fees and charter schools, check out our previous post here.

March Madness: Research Round Up

We’re instituting our first regular feature: a roundup of the most interesting, most discussed education studies published each month. In the future we’ll try to publish the list on the last Friday of each month. Let us know if we forgot anything – or if you have any researchers that you’d like us to add to our list!

Study: Constrained Job Matching: Does Teacher Job Search Harm Disadvantaged Urban Schools?

Results: Teachers in an anonymous Texas school district who remain in their “disadvantaged urban school” tend to outperform those who leave, particularly those who exit the Texas public schools entirely.

What’s Interesting: Implies that less productive teachers tend to leave lower-achieving districts and/or exit the profession entirely and that attempting to limit teacher mobility – i.e. to prevent teachers from transferring out of low-income schools – would not necessarily lead to increases in achievement. Further implies that the high level of teacher turnover observed in lower income urban schools is not necessarily negative.

Caveats: Authors recognize that assessing teacher productivity is difficult and is hampered by other factors such as student background, classroom sorting, and testing variability and have attempted to correct for many, but not all, of these factors.

Study: Identifying Effective Classroom Practices Using Student Achievement Data

Results: Using ten years of data from the Cincinnati Teacher Evaluation System, researchers found that classroom based measures of teaching effectiveness are related in substantial ways to student achievement growth. In particular, they found that improving classroom management skills and adding “question and discussion” practices to lessons had a marked effect on improving student achievement.

What’s Interesting: Shows that there are particular classroom practices that are more effective at increasing student achievement and offers recommendations for teacher evaluation systems that can integrate both student achievement data and helpful classroom observation feedback.

Caveats: There could be factors outside of the eight classroom practices that researchers identified that could contribute significantly to student achievement.  Researchers also acknowledge that “a district may value outcomes for its teachers and students beyond growth in standardized test scores.”

Study: How Many Schools Have Not Made Adequate Yearly Progress Under the No Child Left Behind Act?

Results: One third of U.S. public schools did not make adequate yearly progress, as mandated under the Bush-era No Child Left Behind act.

What’s interesting: Researchers found that finding public data on AYP was extremely difficult, and that comparing pass rates across states was hampered by different methodologies and targets, making it difficult to asses the nation as a whole.

Caveats: The study only looked at publicly available data, which, for some states such as New York, was not available for 2008-2009. Furthermore, researchers didn’t go into any great amount of detail about how standards varied across states and which states’ pass numbers might be the most reliable.

Study: Who Benefits Most from College? Evidence for Negative Selection in Heterogeneous Economic Returns to Higher Education

Results: Students who are least likely to get a college education benefit most from college in terms of economics benefits later in life.  

What’s Interesting: Researchers followed a group of over 2,000 people who were between the ages of 14 and 17 in 1979 for 30 years to see the impact of a college education on future earning potential. They found that respondents who were not minorities, were economically advantaged, and who had college educated parents did not benefit as much from a college education as their less advantaged peers.

Caveats: It may be that there is a selection bias in terms of individuals from disadvantaged social backgrounds. Researchers acknowledge that if “educational expansion results in a larger number of college goers who are otherwise unlikely to attend college” this selection bias might be mitigated and/or lead to flat results.

Study: Beating the Odds: Analysis of Student Performance on State Assessments and NAEP

Results: Low income minority students in urban environments have made significant improvements in math and reading and have narrowed the achievement gap. However, on the whole, disadvantaged urban youth still score lower than their more well-off white peers.

What’s Interesting: Black fourth graders and lower income fourth graders in New York City performed well above average than other fourth graders in mathematics. Students in Charlotte and Austin outperform students in other urban school districts.

Caveats: The data is limited and cannot be used to compare state to state, although the city to city data seems more reliable. Also, they did not differentiate based on the rigor of the state assessments.

Research Notes:

WWC Quick Review of the Article “Are High-Quality Schools Enough to Close the Achievement Gap? Evidence from a Social Experiment in Harlem”

Will Dobbie and Roland Fryer’s controversial study of the Harlem Children’s Zone Promise Academies that found that the schools closed the so-called Scarsdale achievement gap was deemed methodically sound by the What Works Clearinghouse. Dobbie and Fryer found that the Promise Academy public charter schools in the Harlem Children's Zone will increase a student's probability of scoring at grade level by 11.6 percent in sixth grade, 17.9 percent in seventh grade, and 27.5 percent by eighth grade.

Close the Hidden Funding Gaps in Our Schools

Researchers looked at data on the DOE’s website and found that while overall Title 1 schools in New York City receive more per pupil than non-Title 1 schools, there were over 250 Title 1 schools actually received significantly less in 2007-2008. In particularly, two schools in the Bronx received noticeably less money for teachers than other public schools in New York City.