« The Green Dot Contract | Main | Creaming, Part 2 »

February 11, 2009

Creaming, Part 1

When I first heard the term "creaming", I assumed it was a malapropism for the term "skimming".  It turns out, though, "creaming" is the more common term used by educators when complaining that certain schools have an unfair advantage because they select a student population that is easier to educate than traditional public schools.  Recently, the "creaming" argument has been used to attack charter schools.  Currently, there is an entertaining debate going on at GothamSchools.org.   

I can think of three reasons that someone might be concerned with "creaming":

1. Analysis: The analysis of some schools as "better" than others might be unfairly biased by differences in student populations. 

2. Equity: It could be perceived by some as unfair that some schools have to deal with more difficult student populations than others.

3. Funding: It could be viewed by some as appropriate to provide additional funding to schools that have to serve more challenging student populations.

For each component of the creaming argument, we can address two issues: accuracy and consistency.  By "accuracy", I refer to the validity of the complaint.  By "consistency", I refer to the extent to which this complaint is applied consistently in a variety of settings.

With this framework, what can we say about the "creaming" accusations with respect to charter schools in New York City?

1. With respect to the accuracy of the analysis complaint, it is certainly possible that charter schools are educating different populations of students.  Of course, charter schools use lotteries to accept students. They are not allowed to use exams or any other measures typically used to differentiate students.  Therefore, any population difference would be based on either the efforts of charter operators to convince certain parents to apply or the independent tendency of a certain type of parent to apply.  For example, many people argue, I think fairly, that parents that select charter schools are probably more engaged in their child's education than those that don't.  To address these potential differences, the "gold standard" method of analysis is called "randomized assignment".  This technique controls for differences like parental involvement by tracking the difference of performance between students that get into charter schools versus those that apply to charter schools but "lose" the lottery.  A recent study by the Boston Foundation, for example, used this technique and determined that charter schools performed significantly better than traditional public schools.  Another study, still under way, is being conducted in NYC.  The first round of results showed that charter schools outperformed traditional public schools.  So early results seem to indicate that charter schools provide better results.  Let's continue to conduct these types of studies.

2. With respect to the consistency of the analysis complaint, the anti-charter movement has frequently pointed to unscientific studies that don't control at all for population differences in an effort to demonstrate that charter schools are no better than traditional public schools.  (Usually, even with these unscientific studies, they are unable to motivate that charter schools are worse than traditional public schools.)  They have sometimes been successful in getting the New York Times to publish these results with misleading headlines, e.g. here and here.  On the other hand, anti-charter people require the "gold standard" when all of the simpler statistics favor charters.  Then, when a "gold standard" test is conducted, they make believe it never happened.

Tomorrow, I will finish up by addressing the "equity" and "funding" components of the "creaming" debate.

TrackBack

TrackBack URL for this entry:
https://www.typepad.com/services/trackback/6a010535e16b06970c0111685b962e970c

Listed below are links to weblogs that reference Creaming, Part 1:

Comments


The "gold standard" doesn't address the charge that the population of a school has an important effect on the individual students.

For example, it's certainly possible that the most disruptive 1% of students have a material affect on overall school performance, and, due to the selection bias you mention (and others that are present) these "disruptors" may be underrepresented in charters.

I don't know if this is true, just that I've heard a story like this asserted and don't think you've refuted it. Probably it's true to some extent and the "goal" is to demonstrate it only explains a small portion of the total measured advantage of charters.

Agreed! It is also possible that parents that "lose" in lotteries respond by being less involved in their child's education. I consider these plausible second-order effects worthy of study. Narrowly, the "gold standard" experiments suggest that if you are a parent that wants you child to go to a charter school, your child will probably perform better on standardized tests if he or she gets in.

The experiments also suggest that charter schools do a better job at educating kids, although it is possible that there are exogenous factors that allow for this. I am skeptical of these second-order effects and super-skeptical that, even if they are significant, it would materially change my policy recommendations. (More on this in a future post.)

Finally, I have a policy recommendation that would address this matter if people insisted on it. The idea would be to enter some children whose parents don't explicitly select traditional public schools into charter school lotteries. The problem with this is that many charter schools think it is important that parents actively select their school. I agree, but if voters were terribly concerned about this issue, I think my policy suggestion would address it.

The comments to this entry are closed.