Report: Are Students’ Assessments of Their Own Interpersonal Skills Reliable? [downloadable]

Researchers who examined data from five large California school districts have concluded that measures of students’ personal strengths and interpersonal skills are not reliable enough at this point to include in states’ and districts’ school accountability systems.

In a May 2019 report, the research nonprofit Policy Analysis for California Education, or PACE, concluded that surveys of students about their habits of mind, like self-control, are useful and can point to schools that are succeeding in developing individual and interpersonal skills. But the study said that scores vary too much from year to year within schools and grades to be included with more traditional measures — standardized test scores and graduation rates ­— that are included on statewide grading systems like the California School Dashboard.

The authors are Hans Fricke, the director of Quantitative Research of the CORE-PACE Research Partnership; Susanna Loeb, former Stanford University professor of education who directs the Annenberg Institute for School Reform at Brown University; and Heather Hough, PACE’s executive director.

Rick Miller, executive director of CORE, a nonprofit data collaborative of California school districts that developed the study’s annual student surveys, said he was unfazed by PACE’s finding. Neither CORE nor the State of California planned to use the results to judge school performance. More importantly, he said, the PACE analysis reiterated that students’ soft skills like self-confidence matter: They can have a positive impact on student success and some schools are consistently good at developing them.

SEL Research

Research into social and emotional learning has burgeoned as California and other states broadened their test-based accountability systems to include measures of school climate and culture out of recognition that they are foundational for student achievement. These include rates of student suspensions and chronic absenteeism.

About two-dozen districts are now using CORE’s annual survey of student perceptions of their own social and emotional learning skills, and many are delving into the differences in results by ethnicity, race and gender. PACE has established a research partnership with CORE to explore how scores should be calculated and reported and how educators can use social and emotional indicators to improve schools.

CORE surveys go to all students in grades 4 to 12. Students are asked to respond to statements that describe their views on four dimensions of social and emotional learning:

1. Self-management or self-control
2. Growth mindset
3. Self-efficacy
4. Social awareness

All of the measures, particularly growth mindset and self-efficacy, are related, while gauging distinct skills. Growth mindset is the only skill that grew among students from 4th grade through high school, while average scores for the other three declined, starting in middle school. Self-management then plateaued in high school, self-efficacy rebounded slightly in 12th grade, but the mean social awareness scores declined every year after 5th grade.

CORE developed a model that uses the scores on each social and emotional measure of students in a school and compares those to scores of similar students in other schools, taking into account gender, race and ethnicity and other demographic characteristics. The variations in scores may accurately reflect the impact that a school and its teachers had on social and emotional growth. But that assumes that CORE’s model for predicting improvement produces consistent and reliable schoolwide scores.

Variable Results

In its latest study, PACE found that scores vary substantially for the same schools from one year to the next. This “lack of stability” raises doubts that survey-based social and emotional learning measures can consistently identify true differences in school performance over time, the researchers wrote, while acknowledging that a longer study might show more consistency.

Download a PDF of the report, Measuring School Contributions to Growth in Social-Emotional Learning.

Excerpted from “There’s value but also limitations to students’ grading their own interpersonal skills, study finds” in EdSource online. Read the full article.

Source: EdSource | There’s value but also limitations to students’ grading their own interpersonal skills, study finds, https://edsource.org/2019/theres-value-but-also-limitations-to-students-grading-their-own-interpersonal-skills-study-finds/613326 | © 2019 EdSource

Do you need someone to talk to? CHC can help. We invite you to call or email our Care Coordinators at 650.688.3625 or careteam@chconline.org to set up a free 30-minute consultation.