State of Student Aid in Texas – 2019
Average Annual Earnings for Project QUEST Participants and Non-Participants
Given cost pressures at colleges, identifying interventions that can have the greatest impact on student success is vital. Research using random controlled trials provides meaningful insight into the extent to which various interventions are effective in promoting desired outcomes.
Project Quest is a San Antonio, Texas organization aimed at helping low-income residents complete job-focused higher education programs and become gainfully employed. The organization provides students in specific technical and skill-based programs with a comprehensive suite of support and resources including financial assistance, remedial instruction for placement tests, personal and academic counseling, weekly meetings with a focus on life and study skills, and job placement assistance throughout the student’s time pursuing a degree and employment.
In order to evaluate the effectiveness of these resources, Project Quest conducted a randomized trial among students who started services with Project Quest between 2006 and 2008 and followed them for six years. The evaluation focused on those pursuing skilled technical positions in medical fields (e.g., registered nurses, licensed vocational nurses, sonography technicians, etc.). Students who completed the program saw long-term gains in wages and employment, with participants earning over $5,000 more than the control group annually, on average, and experiencing a 15% higher level of year-round employment compared to the control group. These gains were primarily realized during the last half of the six-year observation period, inspiring hope that the wage gap will continue to grow later in the students’ careers.
Sources: Economic Mobility Corporation, Inc., Escalating Gains: Project QUEST’s Sectoral Strategy Pays Off, April 2017 (https://economicmobilitycorp.org/escalating-gains-project-quests-sectoral-strategy-pays-off/).
InsideTrack Student Coaching Evaluation: Persistence Increases Associated with Individualized Coaching, Compared to Control Group
A Portland-based company, InsideTrack, contracts with higher education institutions in all sectors to provide individualized student coaching to improve student success. The company conducted randomized trials at multiple institutions where they randomly divided students at a school into two groups, providing coaching to one group and not to the other, in order to observe the effect of the coaching experience on retention and graduation. The National Bureau of Education Research evaluated many of those experiments, selecting those from the 2003-04 school year, to enable comparisons with certain national survey data, and the 2007-08 school year, as the most current year available at the time of evaluation.
The evaluation found that, when controlling for covariates (i.e. age, gender, high school GPA, SAT score), individualized coaching was associated with an approximately five percent increase in retention after six months, a 12 percent increase after 12 months, and a 14 percent increase after 24 months compared to the control group. Groups receiving coaching also saw levels of completion four percent higher than groups who did not receive coaching.
Individualized student coaching, in addition to more traditional advising financial and academic advising and counseling, has the potential to increase persistence and attainment rates and was more cost effective, in the case of this InsideTrack evaluation, than other previously studied methods of increasing attainment, such as increasing financial aid.
Sources: Bettinger and Baker, The Effects of Student Coaching: An Evaluation of a Randomized Experiment in Student Advising, March 2014 (http://journals.sagepub.com/doi/abs/10.3102/0162373713500523).
Enrollment Gains from Summer Melt Text Nudges
At times, colleges want to provide outreach and interventions to a large number of students in order to improve certain outcomes, such as retention and graduation, but often have limited resources. Sending text messages with targeted messaging at key intervals, commonly referred to as “text nudges,” offer an empirically tested method for positively influencing students along a variety of outcome variables. Text nudges have been found to be effective in combatting attrition during the summer following initial enrollment, known as summer melt,** for as little as two dollars per student included in the texting campaign. Text nudges can be more easily scaled to a larger student population compared to more robust interventions such as phone calls or advising sessions, which, while typically effective, tend to be more labor intensive.
Researchers from the University of Virginia and the University of Pittsburgh conducted randomized controlled trials in 2012 at three high schools. The students in the experimental groups received text messages reminding them about college-related deadlines or required tasks (e.g. reminders to access important paperwork and register for orientation). These text nudging interventions were designed to lower the number of college bound high school graduates that fail to matriculate in the fall.
The text nudges were found to be effective for students with moderate GPA’s, students who were enrolling in a two-year program, students enrolled in free or reduced-price lunch programs, students with unspecified college plans, and students who had not completed the FAFSA. Two-year programs experienced a statistically significant increase in enrollment with an increase of three percent. This suggests that summer melt text nudges are most effective for groups that may have limited access to other quality college information sources and represent a cost-effective intervention, but may be insufficient when used alone for many groups of students.
*Statistically significant at the 0.05 level (p<0.05).
**Summer melt is a term used to describe the occurrence of students indicating their intent to attend a college in the fall but then ultimately not matriculating. Some have defined this term as only including those who did not matriculate at any college while others have defined it as specific to an institution. Students may indicate their intent to attend college through various activities such as expressly saying so on a form to a counselor, paying college deposits, and registering for classes. The activities used to determine intent depend upon the definition being used for summer melt. For the purposes of this study, the researchers used information on students’ expressly stated intentions to attend college and defined summer melt as “the phenomenon that college-intending high school graduates fail to matriculate in college anywhere in the year following high school.”
Sources: Summer Nudging: Can personalized text messages and peer mentor outreach increase college going among low-income high school graduates? Castleman, B. and Page, L., Journal of Economic Behavior and Organization (2015), (https://www.sciencedirect.com/science/article/pii/S0167268114003217).
Enrollment Change For Two-Year College Students Receiving FAFSA Re-Filing Text Nudges
The sending of targeted messaging via text messages at key intervals is commonly referred to as a “text nudge”. Text nudges have been found to have positive effects in facilitating increased rates of annual FAFSA completion, and in promoting retention and attainment, for a minimal cost to the institution. A series of studies on text nudges used in varying contexts have suggested that text nudges may provide a low-cost alternative or supplement to other more intensive methods of outreach during a student’s academic career, but are limited in effectiveness for some groups of students.
Researchers from the University of Virginia and the University of Pittsburgh used a randomized controlled trial design to examine the impact of text nudges on FAFSA re-filing rates among college freshmen. Text nudges containing information on where to obtain help with financial aid, important deadlines and requirements, and offering assistance related to financial aid and were sent to a randomly assigned group of community college freshmen during the 2012-13 academic year. Outreach took place over the course of approximately seven months with messages approximately every two weeks. Text nudges designed to provide important information and prompting concerning annual refiling of FAFSA have been found to be highly effective among community college students. Freshman community college students who received text nudges were nearly 12 percent more likely to persist into the fall of their sophomore year and were 14 percent more likely to persist into the spring.
Text messages represent a viable cost-effective option and are a valuable tool as part of a set of strategies to impact academic accessibility, persistence, and attainment; however, used alone, text nudges are likely to be inadequate for the overall student population. While impacts are substantial in some cases, effects are consistently limited to specific groups of students, often those with low availability of resources.
*Statistically significant at the 0.05 level (p<0.05).
Sources: Freshman year financial nudges: An experiment to increase FAFSA renewal and college persistence. Castleman, B. and Page, L., Journal of Human Resources (2016), (http://jhr.uwpress.org/content/51/2/389.short)
Enrollment, Retention and Graduation Change For College Students Receiving Need-Based Grants
A metanalysis of ten recent studies by Sneyers and DeWitt examined the effects of need-based grants on enrollment, retention, and graduation. This analysis demonstrated overall positive effects of need-based grants across the three outcomes measured in all studies. While these effects were small, they were strongly significant.
More granularly, the study found that need-based grants had a small positive impact on enrollment across studies in different educational contexts and multiple countries. Enrollment among those receiving grants increased by 2.5 percent compared to the control group. When looking at retention, the study similarly found a small but significant positive impact from need-based grants. When compared to their peers, the recipients of need-based grants were 2.5 percent more likely to be retained. This effect was again seen across all studies included in the analysis.
Finally, the results indicated that need-based grants have a positive effect on students ultimately graduating, but not necessarily within the ideal time to degree, such as four years for a bachelor’s degree. Students who received grants were significantly more likely to graduate when looking at longer time to degree; however, no significant results were found for graduation rates within the ideal time to degree. The authors note that this seems to indicate that the effects of need-based grants may be delayed or may increase over time.
Sources: Eline Sneyers & Kristof De Witte (2018) Interventions in higher education and their effect on student success: a meta-analysis, Educational Review, 70:2, 208-228, DOI: 10.1080/00131911.2017.1300874 (https://doi.org/10.1080/00131911.2017.1300874).
In 2007, the City University of New York (CUNY) launched the Accelerated Study in Associate Programs (ASAP) with the aim to improve graduation rates among low-income students. The program provided services, such as tutoring and individual advising, and financial assistance with public transportation, textbooks, and tuition. An evaluation of the program found that retention rates, credit accumulation, and graduation rates were significantly increased among program participants as compared to a control group. The program group had a graduation rate that was nearly double that of the control group after three years (40 percent vs 22 percent) and was still 10 percentage points higher than the control group after six years (51 percent vs 41 percent), showing that the program both increased the graduation and helped students graduate at a faster rate.
CUNY ASAP: Earned Any Degree
The ASAP model was replicated in 2014 at three Ohio community colleges as a demonstration project. An evaluation of the Ohio ASAP found similar results. Retention rates, credit accumulation, and graduation rates were all significantly higher among program participants as compared to the control groups. As with the CUNY ASAP, the Ohio ASAP saw the graduation rate nearly double among program participants compared to the control group after two years (19 percent vs 8 percent). Though there are costs associated with implementing a program like ASAP, these evaluations demonstrate the real-world benefits and can help inform conversations about costs and the return on investment.