State of Student Aid in Texas – 2018
Average Annual Earnings for Project QUEST Participants and Non-Participants
Given cost pressures at colleges, identifying interventions that can have the greatest impact on student success is vital. Research using random controlled trials provides meaningful insight into the extent to which various interventions are effective in promoting desired outcomes.
Project Quest is a San Antonio, Texas organization aimed at helping low-income residents complete job-focused higher education programs and become gainfully employed. The organization provides students in specific technical and skill-based programs with a comprehensive suite of support and resources including financial assistance, remedial instruction for placement tests, personal and academic counseling, weekly meetings with a focus on life and study skills, and job placement assistance throughout the student’s time pursuing a degree and employment.
In order to evaluate the effectiveness of these resources, Project Quest conducted a randomized trial among students who started services with Project Quest between 2006 and 2008 and followed them for six years. The evaluation focused on those pursuing skilled technical positions in medical fields (e.g., registered nurses, licensed vocational nurses, sonography technicians, etc.). Students who completed the program saw long-term gains in wages and employment, with participants earning over $5,000 annually, on average, and experiencing a 15% higher level of year-round employment compared to the control group. These gains were primarily realized during the last half of the six-year observation period.
Sources: Economic Mobility Corporation, Inc., Escalating Gains: Project QUEST’s Sectoral Strategy Pays Off, April 2017 (https://economicmobilitycorp.org/escalating-gains-project-quests-sectoral-strategy-pays-off/).
InsideTrack Student Coaching Evaluation: Persistence Increases Associated with Individualized Coaching, Compared to Control Group
A Portland-based company, InsideTrack, contracts with higher education institutions in all sectors to provide individualized student coaching to improve student success. The company conducted randomized trials at multiple institutions where they randomly divided students at a school into two groups, providing coaching to one group and not to the other, in order to observe the effect of the coaching experience on retention and graduation. The National Bureau of Education Research evaluated many of those experiments, selecting those from the 2003-04 school year, to enable comparisons with certain national survey data, and the 2007-08 school year, as the most current year available at the time of evaluation.
The evaluation found that, when controlling for covariates (i.e. age, gender, high school GPA, SAT score), individualized coaching was associated with an approximately five percent increase in retention after six months, a 12 percent increase after 12 months, and a 14 percent increase after 24 months compared to the control group. Groups receiving coaching also saw levels of completion four percent higher than groups who did not receive coaching.
Individualized student coaching, in addition to more traditional advising financial and academic advising and counseling, has the potential to increase persistence and attainment rates and was more cost effective, in the case of this InsideTrack evaluation, than other previously studied methods of increasing attainment, such as increasing financial aid.
Sources: Bettinger and Baker, The Effects of Student Coaching: An Evaluation of a Randomized Experiment in Student Advising, March 2014 (http://journals.sagepub.com/doi/abs/10.3102/0162373713500523).
Enrollment Gains from Summer Melt Text Nudges
At times, colleges want to provide outreach and interventions to a large number of students in order to improve certain outcomes, such as retention and graduation, but often have limited resources. Sending text messages with targeted messaging at key intervals, commonly referred to as “text nudges,” offer an empirically tested method for positively influencing students along a variety of outcome variables. Text nudges have been found to be effective in combatting attrition during the summer following initial enrollment, known as summer melt,** for as little as two dollars per student included in the texting campaign. Text nudges can be more easily scaled to a larger student population compared to more robust interventions such as phone calls or advising sessions, which, while typically effective, tend to be more labor intensive.
Researchers from the University of Virginia and the University of Pittsburgh conducted randomized controlled trials in 2012 at three high schools. The students in the experimental groups received text messages reminding them about college-related deadlines or required tasks (e.g. reminders to access important paperwork and register for orientation). These text nudging interventions were designed to lower the number of college bound high school graduates that fail to matriculate in the fall.
The text nudges were found to be effective for students with moderate GPA’s, students who were enrolling in a two-year program, students enrolled in free or reduced-price lunch programs, students with unspecified college plans, and students who had not completed the FAFSA. Two-year programs experienced a statistically significant increase in enrollment with an increase of three percent. This suggests that summer melt text nudges are most effective for groups that may have limited access to other quality college information sources and represent a cost-effective intervention, but may be insufficient when used alone for many groups of students.
*Statistically significant at the 0.05 level (p<0.05).
**Summer melt is a term used to describe the occurrence of students indicating their intent to attend a college in the fall but then ultimately not matriculating. Some have defined this term as only including those who did not matriculate at any college while others have defined it as specific to an institution. Students may indicate their intent to attend college through various activities such as expressly saying so on a form to a counselor, paying college deposits, and registering for classes. The activities used to determine intent depend upon the definition being used for summer melt. For the purposes of this study, the researchers used information on students’ expressly stated intentions to attend college and defined summer melt as “the phenomenon that college-intending high school graduates fail to matriculate in college anywhere in the year following high school.”
Sources: Summer Nudging: Can personalized text messages and peer mentor outreach increase college going among low-income high school graduates? Castleman, B. and Page, L., Journal of Economic Behavior and Organization (2015), (https://www.sciencedirect.com/science/article/pii/S0167268114003217).
Enrollment Change For Two-Year College Students Receiving FAFSA Re-Filing Text Nudges
The sending of targeted messaging via text messages at key intervals is commonly referred to as a “text nudge”. Text nudges have been found to have positive effects in facilitating increased rates of annual FAFSA completion, and in promoting retention and attainment, for a minimal cost to the institution. A series of studies on text nudges used in varying contexts have suggested that text nudges may provide a low-cost alternative or supplement to other more intensive methods of outreach during a student’s academic career, but are limited in effectiveness for some groups of students.
Researchers from the University of Virginia and the University of Pittsburgh used a randomized controlled trial design to examine the impact of text nudges on FAFSA re-filing rates among college freshmen. Text nudges containing information on where to obtain help with financial aid, important deadlines and requirements, and offering assistance related to financial aid and were sent to a randomly assigned group of community college freshmen during the 2012-13 academic year. Outreach took place over the course of approximately seven months with messages approximately every two weeks. Text nudges designed to provide important information and prompting concerning annual refiling of FAFSA have been found to be highly effective among community college students. Freshman community college students who received text nudges were nearly 12 percent more likely to persist into the fall of their sophomore year and were 14 percent more likely to persist into the spring.
Text messages represent a viable cost-effective option and are a valuable tool as part of a set of strategies to impact academic accessibility, persistence, and attainment; however, used alone, text nudges are likely to be inadequate for the overall student population. While impacts are substantial in some cases, effects are consistently limited to specific groups of students, often those with low availability of resources.
*Statistically significant at the 0.05 level (p<0.05).