The Effects of Performance Feedback on School Choice

The Effects of Performance Feedback on School Choice

Context

Forward-looking investments in human capital are, by nature, made under uncertainty and rely on subjective expectations about present and future returns. This is the case for students’ school and career choices. Youth misperceptions of their own talent and skills may bias favorable choices. Access to information and knowledge is thus crucial to help students and/or parents make sound education choices, especially for students from less-privileged backgrounds who tend to face more acute information frictions. Providing them with tools to enable well-informed human capital investments may enhance social and economic mobility.

The Project

This study attempts to understand how individual expectations of one’s own academic ability shape curricular decisions in upper-secondary education. The study leverages the centralized assignment mechanism used in the metropolitan area of Mexico City to allocate students into public high schools. Two institutional features are key for the research design. First, the assignment system is regulated by strict and observable criteria: stated preferences on schools and scores in a standardized scholastic admission exam. Second applicants are required to submit their preferred rank–ordered lists of schools before taking the admission test.

The study used a subsample of applicants from the least-advantaged neighborhoods within the catchment area of the school assignment mechanism because these students are less likely to have access to previous signals regarding their academic potential. A randomized control trial was used to evaluate the impact of the intervention.

Behavioral Analysis

Behavioral Barriers

Prescriptive norms: These refer to what society approves or disapproves of—that is, what is considered to be right or wrong—regardless of how individuals actually behave. Such norms are useful for reaffirming or encouraging what are considered positive individual behaviors while discouraging negative ones. In the context of our study, the acceptable behavior is to pay taxes when they are due to the government. In this context, students from neighborhoods with high levels of marginalization are more likely to believe they are not good enough to pursue certain academic/professional paths.

Availability heuristics: Individuals tend to estimate the probability of a future event based on how readily representative examples of such an event come to mind. Students from less-privileged socioeconomic backgrounds may underestimate the probability of the admission into good schools because these examples are scarce for them.

Behavioral Tools

Feedback: It is an effective tool to enhance awareness of the consequences of various choices. It may fill knowledge gaps and foster the search for efficient alternatives.

Simplification: Reducing the effort required to perform an action. For example, cutting the number of steps involved in achieving a complex goal, or break it down into simpler steps.

Intervention Design

The intervention was a face-to-face performance feedback on the results of the mock exam. Treatment assignment was randomized within strata at the school level, with 44 schools in the treatment group, and 46 schools in the control group. The mock exam was administered to the control group without reporting the test results. Schools randomized out of the intervention (28) constituted a pure control group, which was interviewed only in the follow-up survey.

Figure 1 presents the timeline of the intervention. The delivery of feedback on performance in the mock exam took place at the beginning of the follow-up survey. Surveyors showed a personalized graph with two pre-printed bars: the average score in the universe of applicants during the 2013 edition of the COMIPEMS system and the average mock exam score in the class of each applicant. During the interview, a third bar was plotted corresponding to the student’s score on the mock exam.

Figure 1. The School Assignment Process and the Intervention: A Timeline

The School Assignment Process and the Intervention: A Timeline

Note: COMIPEMS rules in place in 2014.

Challenges

  • Absentee rate on the exam day and attrition: From the initial sample of 3,001 students assigned to either the treatment or the control groups at baseline, 2,790 were present on the day of the exam and 2,544 were present in the follow-up survey.

Results

  • Estimates suggest that taking the exam without the provision of performance feedback does not generate any differential updating behaviors. Comparison is thus performed between randomized treatment and control groups.
  • Students’ subjective beliefs about their academic ability respond to the provision of information about their own performance. Conditional on taking the mock exam, mean beliefs in the treatment group fall on average 7.5 points, while the standard deviation of beliefs goes down by about 2.6 points.
  • Relative to the control group, these effects represent a 10 and 15 percent reduction in the mean and standard deviation, respectively.
  • Heterogeneous effects of the treatment depend on the direction of the update. The estimated negative coefficient of the treatment on mean posteriors in the full sample can be explained by the fact that about 80 percent of the applicants in the sample had scores below their baseline mean beliefs.
  • Applicants receiving positive feedback in the treatment group increase their shares of requested academic options by 8.3 percent when compared to control group. This corresponds to approximately 18 percent of the sample mean in the control group. Large reductions in mean beliefs observed amongst the applicants receiving negative feedback in the treatment group do not appear to translate into any corresponding change in the demand for academically oriented programs.

Policy Implications

  • Students face important informational gaps related to their academic potential. Closing these gaps has a sizable effect on track choices in high school.
  • Providing youth with individualized information on their own academic potential can effectively alter career decisions during a critical period of their schooling trajectories.
  • Differential role of the first two moments of the belief distribution play in determining schooling investment decisions. Ignoring the changes in the dispersion of individual beliefs may systematically confound the effects of individualized information on academic ability on high school track choices.
  • Policies should aim at disseminating information about individual academic skills in order to provide students with better tools to make well-informed curricular choices. For example, middle schools could be incentivized to implement mock tests and deliver score results before students submit the rank-ordered lists of schools within the centralized assignment mechanism.