Peer effects are at the center of educational policy debates regarding school choice, ability grouping, and instructional design. Though emerging empirical evidence suggests that positive peer effects exist, less is known about how it affects students with varying cognitive abilities. Using a nationally representative sample from China, we generated a student-level measure of classroom composition of peers based on cognitive ability to understand the benefits or pitfalls of placing low-ability students with heterogeneous or homogenous classmates. We conducted this analysis separately for grades seven and nine students after controlling for student background, family characteristics, and school endogeneity. We reaffirmed the overall positive—but small—peer effects on the performance rankings. Low-ability children scored much lower than their counterparts when they studied in cognitively diverse classrooms. However, this effect negates the overall positive impact of studying with high-ability peers and the pattern is consistent across rural and urban schools.