Summary
Many school districts around the world have adopted centralized admissions systems to coordinate student assignments. These mechanisms ask students to rank the schools that they would like to attend and then use an algorithm to co-ordinate placement. These algorithms take into account student preferences, eligibility criteria at various schools and school capacities.
Relative to traditional decentralized school admissions systems, these mechanisms hold the promise of improving student assignments. Moreover, in the process of doing so, they collect rich data on student preferences collected by these mechanisms. These data can be used, in principle, to learn about the aspects of schools that students from various socio-economic backgrounds desire and to help direct efforts towards improving school quality.
Indeed, there is a wide array of school choice mechanisms used throughout the world. Most of these systems were instituted without the active involvement of economists. Despite long experience with different mechanisms, we have a limited understanding on which of the various mechanisms available do the best job at placing students to schools. To complicate things further, we also do not know how to utilize the data collected on student preferences. This issue arises because mechanisms do not always make it in the students’ best interest to report their true preferences. Instead, students can often gain an edge from gaming the system.
To better understand student preferences and the performance of various mechanisms, this study develops a general methodology to analyze the reports on student preferences submitted to the various school choice mechanism in place around the world. It then applies the method to analyze, as an example, the school choice mechanism used to place elementary school students in Cambridge, MA.
Main article
School choice mechanisms, preference manipulation and its effects
In the vast majority of school choice mechanisms currently in use, students have the incentive to manipulate their reported preferences to gain an advantage in admissions. For example, the Immediate Acceptance mechanism used in Cambridge, MA prioritizes students at schools they rank highly. So, a student who ranks a school as her first choice gains priority over those that rank the school second. This rule can make it risky for a student to rank a competitive school second because she will have lower priority than students that ranked the school first. In fact, it can be beneficial for a student to place a school she prefers less than her most preferred school as her top choice in order to secure admission at this school rather than risk the chance that she does not get both her first and second most preferred schools.
Most of the mechanisms with similar, strong incentives to manipulate were instituted before economists actively engaged in the school assignment reform. In fact, with the guidance of economists, the manipulable assignment mechanism used in Boston was replaced in the mid-2000s with one based on the Deferred Acceptance mechanism. This system gives students straightforward incentives to rank their schools truthfully because no student can improve their school placement by trying to game the reported preferences.Whether or not manipulation should be incentivized is controversial, with arguments on both sides. One argument against a manipulable mechanism is that sincere students who report their preferences truthfully, irrespective of these incentives are disadvantaged relative to sophisticated students. Specifically, in the Immediate Acceptance mechanism used in Cambridge, sincere students lose priority at their second choice onwards to sophisticated players who may strategically rank these schools higher (see Pathak and Sonmez, 2008). However, the overall effect of this force is unclear because sophisticated students will avoid ranking the top choices of sincere students at the first position as they correctly predict that those schools will be in high demand.
An argument in favor of manipulable mechanism is that students sort themselves by intensity of preferences. For example, suppose all students prefer school A to school B, but differ in how much they prefer school A. In this case, students who do not significantly dislike school B will systematically take the safe bet and secure the school by ranking it first. This can create better assignments on intensity of preferences in the school district overall (see Abdulkadiroglu, Che and Yasuda, 2011).
In addition to manipulation, school choice mechanisms also differ in a number of other dimensions. Some systems allow students to effectively trade school priorities while others do not. Systems that allow trade can disproportionately advantage students that have high priority at the most desirable schools. Other mechanisms strictly enforce specific school priorities at each school.
Analyzing student preferences
In order to understand the quantitative implications of adopting various school choice mechanisms, it becomes necessary to analyze student preference data. But, this task becomes complicated because of the possibility of strategic manipulation.Indeed, strategic pressures in Cambridge are intense and there is direct and anecdotal evidence of manipulation. Some schools are only assigned to students that rank it first while others are assigned also to students that ranked them second or third. Perhaps it is not surprising that many parents participate in online discussion forums, where they discuss how to play the system. Moreover, the data show that students with different neighborhood priorities that live right next to each other behave differently and in a manner consistent with manipulation.
We start by estimating the demand for schools assuming that parents are sophisticated in their understanding of how competitive schools are, and how the mechanism uses their rank order lists to determine assignments. This benchmark model probably accords an unrealistically high level of sophistication, but serves as an extreme case. We repeat all our exercises for alternative models in which parents have biases and summarize these results along with the baseline benchmarks.
In order to interpret the reported preferences, consider giving advice to a parent. Suppose school A is preferred to school B, but is more competitive. If school A is ranked first, there is a significant risk that the student will not be placed at that school. In the second position, the priority for school B is lowered, risking the chance that the student will be assigned at an even lower ranked option. But, if school A is significantly more desirable than school B, then a long-shot at school A may be worthwhile. Otherwise, a prudent decision is to rank school B instead of school A to secure a position at that school. Therefore, a choice of a rank-order list corresponds to a choice of the admissions probabilities into various schools.
Conventional methods that take rank order lists at face-value make important errors, some extremely policy relevant. For example, schools that are extremely undesirable are still ranked as fall back options or upgraded in their position because of the gaming considerations described above. A naïve analysis of the rank-order list would fail to identify these schools or understate their need for improvement. Indeed, our estimated the demand for various schools in Cambridge, MA using the logic outlined above shows that these effects are large.
As mentioned earlier, the benchmark results assume that all parents are sophisticated. Instead, the logic above can be modified very simply to account for biases. One form of bias is that parents may form beliefs about how competitive a school is based on previous year data. Alternatively, they may not be aware of the finely tuned priorities that are used in the mechanism. Both these forms of biases attenuate the results above, but the direction of the effects are similar. Finally, we also considered a model in which a fraction of parents are sophisticated, and another fraction of students are sincere. This model yields some interesting results that are further described below.
Deferred acceptance versus the alternatives
With demand for schools estimated, we return to the debate between manipulable and non-manipulable school choice systems. Specifically, we compare the manipulable Immediate Acceptance mechanism, which gives higher priority to students that rank a school highly, to the Deferred Acceptance mechanism, which does not.Previous theoretical results have shown that Immediate Acceptance can effectively screen students based on the intensity of their preferences. Recall the example with schools A and B in which school A is competitive. In the Immediate Acceptance mechanism, only students with a strong preference for the school will rank it first because of the implicit penalty at school B if it is not ranked first. In Deferred Acceptance, there is no penalty for ranking the school second, and so both types will rank it.
Our results show that the best case for these potential screening benefits are modest. We quantified these results based student preferences estimated in terms of an equivalent travel distance. That is, how much of an extra travel distance is the average student willing to endure in order to obtain a more preferable assignment. We estimate that the Immediate Acceptance mechanism is preferred over the Deferred Acceptance mechanism by the average student by the equivalent of 0.08 mile change in travel distance. These results weigh against arguments for adopting a manipulable system similar to the Immediate Acceptance mechanism for their potential screening benefits.
Interestingly, students from low-income families (those receiving subsidized lunch) do not have to be as strategic as middle- and higher-income students in the Immediate Acceptance mechanism. Part of the reason is that the Cambridge Public School system has quotas for students receiving subsidized lunch that gives them access to desirable schools without behaving strategically. Consequently, the effects of strategic behavior on their assignments is also lower.
These results were based on models in which all parents were sophisticated. But, one might wonder about the effects of some students being sincere, and whether sincere students lose to sophisticates. We estimated a model in which a fraction of students behave sincerely and the remaining are sophisticates. Our estimates suggest that indeed, approximately one-third of students may be behaving sincerely even though there are benefits from being strategic.
Remarkably, we find that sincere students still benefit from the Immediate Acceptance mechanism even though they are at a relative disadvantage when compared to sophisticated students. Our results suggest that sophisticated students avoid ranking schools that are desirable. They do so because a significant fraction of sincere students continue to rank it first. This fact makes these schools extremely competitive, a consequence that sophisticates realize, making the desirable schools less competitive for sincere students. Of course, sincere students do worse than sophisticated students, but they do better relative to the Deferred Acceptance mechanism. This finding gives pause to previous arguments against the Immediate Acceptance mechanism in a bid to protect sincere students.
Future Directions
A commonly overlooked aspect of designing school mechanisms is the priority system, which is usually taken as given by economists. But, one reason for using priorities such as sibling priority or neighborhood priority is to ensure that at least some students that have a strong desire to attend a particular school are able to. In fact, assigning students based on intensity of preference is why the Immediate Acceptance mechanism was able to improve school assignments, albeit marginally. Unfortunately, most current school choice designs do not have a way to directly gauge how strong a preference a student has for a given school. While priorities may be limited in their ability to do so, it is worth carefully considering the reasons for using priorities and to design them better.Another important issue is better measuring school demand and finding ways to use this information to improve schools. Understanding student preferences helps identify the investments in school capacity that will have the highest take-up. Alternatively, it can help us understand the effects of new schools or school expansions on the distribution of students of various types across schools.
Understanding both these issues requires models of school demand. The methods described above are a first-step in such analyses. They also need to be improved, in particular, by augmenting them with surveys of student preferences (see Kapor, Neilson and Zimmerman, 2018, for example) and improving the match of the models to the nature of student behavior in the real world.
