Impact of Online Summer Mathematics Bridge Program on Placement Scores and Pass Rates

An online four-week summer mathematics bridge program was implemented at a Midwest university with historically low pass rates in College Algebra and Remedial Mathematics. Students who completed the four week program significantly increased their mathematics placement exam scores. These students also had a higher pass rate in their initial college mathematics course compared to the overall student population taking the same courses.


Introduction
Introductory college mathematics courses have proven to be a barrier to many undergraduate students, particularly those wishing to pursue STEM, business, health, or paraprofessional careers. Pass rates of these initial college mathematics courses has been shown to be quite low [1,2]. At our institution, the pass rate of our college algebra course is below sixty percent, the lowest pass rate of any credit-bearing course at our university.
Similar to other institutions, many students must complete one or two remedial mathematics courses before even attempting college algebra. The National Council of State Legislators estimates that anywhere from twenty-eight to forty percent of all undergraduates will need to take at least one remedial course [3]. The historic pass rate for these courses is even worse than that of college algebra; the remedial mathematics pass rate has often been below fifty percent. For both college algebra and remedial mathematics, students often attempt the same course for multiple semesters before either passing the course or giving up and choosing a different major or life path.
In response to such low first year pass rates and first year retention; many universities have developed summer bridge programs. Historically, these programs have often been aimed at underrepresented populations [4][5][6][7][8][9]. Many of these programs have not focused only on mathematics. Instead, these programs allowed participants to choose a subject that they needed help with, such as mathematics, writing, or a specific elective and then would supplement this with material aimed at improving study skills, time management, etc. These programs typically lasted between four to six weeks in length and were primarily residential in nature. More recently, some institutions have begun to offer online programs, either as an alternative, or a replacement for residential programs. Preliminary results have indicated that students participating in residential programs saw greater success than those in online programs [8]. However, as technological advances continue and online courses can become more interactive there is increased evidence of their potential [10].
Summer bridge programs that do focus primarily on mathematics have often been designed for engineering or STEM students who need to be placed in calculus their first semester [11][12][13][14]. This is a different target population than our student population. Although like many other summer bridge programs we do have a large number of first generation students and other underrepresented students, our barrier course is college algebra, not calculus.
Despite the abundance of bridge models, there is a notable gap in conclusive evidence as to whether these programs benefit students past their first semester in college because there are so few longitudinal studies of such programs [4,15].
Much like summer bridge programs in general, summer mathematics bridge programs have often not been evaluated in depth or consistently. Longitudinal or ongoing evaluations have been relatively rare in the literature [15]. One exception to this was a longitudinal study done at the University of Alabama examining the effect of their Engineering Math Advancement Program. After three years they observed a twelve percent increase in retention. However, the grades of the participants were lower in their first mathematics course compared to the students who did not participate in the bridge program, which could be due to the fact that the bridge participants began with a lower mathematical skill set than those who did not participate in the bridge program. This led to the suggestion that students who place into the course, not just those who need remediation, should be provided additional supports to increase success in the course [16].
As we developed our program, we examined existing literature, but we were also subject to significant constraints. The program, which received very little institutional funding had to be offered at low cost to participants. Rather than specifically targeting low-income, underrepresented populations, we targeted students based on their mathematics placement score. Due to cost concerns, we created an online course. While historically these have had less impact than residential courses, as technology has developed there have been recent results to suggest that an online mathematics summer bridge program could make a difference [10].

Objectives
Our bridge program was a four-week online summer mathematics refresher program. The ultimate goal of this project was to minimize costs and maximize impact on students' success in their first semester of college mathematics. In particular, this study examined whether the participants' pre and post placement scores increased. Then, we compared the pass rates of bridge program participants to the overall population of students in the applicable math course.

Placement Exam
Before registering for their initial math course, all incoming students must either transfer in mathematics courses or take a mathematics placement exam. The placement exam consists of two parts. If a prospective student passes the first portion of the placement exam, which assesses topics needed for College Algebra, they may proceed to the second part of the placement exam, which assesses knowledge of pre-calculus and trigonometry topics.
The placement exam is used to place students in courses varying between introductory remedial mathematics through the first semester of calculus. A score of 0-6 places the student in an introductory remedial mathematics course; 7-11 allows placement into Intermediate Algebra, our second remedial mathematics course; 12-20 allows placement in College Algebra; 21 or above allows placement in the first semester of calculus.
Normally, students are not permitted to retake the placement exam without advisor approval and significant extenuating circumstances. Participants in the summer bridge program were allowed to retake the exam at the end of the program. All students who scored into College Algebra or below were targeted for the summer bridge program. The range of participants' scores was between 0 and 11.

Program Information
The summer bridge program was designed using pre-existing online modules and lessons. The primary source for this material was through the Khan Academy website. A former high school teacher and current instructor of College Algebra created a redesigned curriculum map following an initial pilot the previous summer and a Khan Academy course was created to track student progress. The program was self-paced and contained an online unit quiz over each unit. In order for participants to proceed to subsequent units, they had to demonstrate proficiency. If participants followed the curriculum map, each unit took approximately one week to complete. The program cost $50 dollars, and if participants completed the program in the designated four weeks, they received a $50 book scholarship. Online material remained available to program participants for the subsequent academic year in case they wanted to review a topic again while enrolled in their college mathematics course.
Online tutoring was provided as part of the program. Participants were able to interact with the tutors through Blackboard Collaborate; which allowed tutors to interact with participants in real time. More traditional communication methods, such as email, fax, and phone calls were also available to program participants.
The retake (post-test) of the mathematics placement exam occurred in a proctored setting. Seats in pertinent mathematics courses were reserved for program participants who improved their placement test score and were eligible for a more advanced mathematics course.

Data Analysis
Participants' progress was tracked during the summer program. All participant data, regardless of progress was analyzed. Then, to better see the impact of the program itself, data from participants who completed the course were analyzed separately. These "program completers" were actively involved in the course for portions of all four weekly units, determined by weekly quiz attempts and the retake of the placement exam.
Not all participants took the placement exam twice. For those that did, pre and post placement scores were compared using a two-tailed paired t-test. This was done twice: for participants who took both placement tests and the smaller population of program completers.
In addition, data was collected regarding participants' fall mathematics class enrollment. Once final grades were submitted, participants' grades were recorded and the success rate of program participants was compared to that of the overall student population in the relevant courses. The same analysis was done for program completers. Successful completion of the mathematics course was defined as passing the course with a grade of C or better. Students who earned a grade lower than C, or withdrew from the course were considered to have not successfully passed the course and are designated in the DFW category.

Participant Information
The participants were incoming undergraduates who scored lower than they wished to on their mathematics placement exam. Incoming students traditionally take the placement exam once before they are allowed to register for their initial mathematics course. While the program was advertised to all students, only students who tested below a 12, the cutoff score to enroll in College Algebra, elected to participate in the program.
Of the 81 participants registered for the program, three never took the initial placement exam, so they were excluded from the pre and post exam calculations. The three participants who failed to take the initial placement exam all self-assessed that they would have received a low score on the initial placement exam. There were also a number of students who took the initial placement exam, but failed to complete the post exam. This occurred primarily due to two reasons, either they were happy with their original score and simply wanted to refresh their content knowledge or they essentially stopped participating in the program and did not show up for the post exam. Therefore, out of the initial 81 participants, only 65 of the participants took both the pre and post placement exams.
Out of the 65 participants who completed both the pre and post placement exams, only 50 of them completed at least eighty percent of the course. Those 50 participants we broke out for additional analysis and labelled them program completers.

Pre and Post Test Comparison
Mean scores significantly increased from an average of 8.32 to 12.62, as can be seen Table 1. A two-tailed paired t-test was administered and the means were found to be significantly different with an extremely low p-value that rounded to .000.
Digging in to the data a little deeper, 42 of the 68 participants, or 61.7%, who took the post-exam raised their score enough that they were able to take a more advanced mathematical course than their original placement allowed. Sig. (Two Tailed) .000 *All participants who completed both the pre and post placement exams, regardless of completing the program.
Of the 50 participants who completed the program, one failed to take the pre-test, so was excluded from this phase of data analysis. The mean placement exam score increased from an average of 8.41 to 13.20 as can be seen in Table 2. A two-tailed paired t-test was administered for this population and again, the means were found to be significantly different with again an extremely small p-value that rounded to .000.
Thirty-five of the 50 (70%) program completers raised their placement score enough to take a more advanced mathematics course. Sig. (Two Tailed) .000 **All participants who completed both the pre and post placement exam and finished at least 80% of the program.

Success Rate of Participants
After completion of the program, participants were tracked as to if they enrolled in a mathematics course and the ultimate outcome in the course. Of the 81 participants, only 9 did not take a math course the following semester. Of the 72 who took a math course, 49 or 68.1%, passed the course.
Data on the program completers was examined. Of the 50 participants who completed the program, 3 did not take a math course. Of the remaining 47 participants, 34, or 72.3% passed their mathematics course.
The majority of students who did enroll in a math course were either enrolled in College Algebra or Intermediate Algebra, a remedial mathematics course, (see Table 3 for detailed information). Thirty participants enrolled in College Algebra and 20 students, 66.7%, passed the course. Thirty-five students enrolled in Intermediate Algebra and 24 of those students passed the course. The results for the 50 students who completed the program as intended are included in Table 4. For those enrolled in College Algebra, there was no significant difference in the successful completion rate between program participants and program completers. Since approximately eighty-seven percent of the participants enrolled in College Algebra had completed the program, this is not a surprising result. However, there was a substantial difference in the number of program participants versus program completers enrolled in Intermediate Algebra. Out of the 30 participants who enrolled in Intermediate Algebra, only 18 had actually completed the program. Out of those 18, only 4 failed, which resulted in a significantly higher pass rate of 77.8% for the program completers compared to the 68.6% pass rate for the program participants enrolled in Intermediate Algebra.
The overall pass rates for that same semester of the overall student population was 57.1% for College Algebra and 53.1% for Intermediate algebra. Therefore the successful pass rates for both College Algebra and Intermediate Algebra was more than ten percent higher for program participants than it was for the general student population. **All participants who completed both the pre and post placement exam and finished at least 80% of the program.

Discussion
The data from the summer bridge program is promising but presents several issues and questions. These issues include outreach, self-selection, drop-off rates, and better tracking of program participants. Significant changes were made to the following summer's program and this model of analysis, reflection and improvement will continue as long as funding for the program is available.
As discussed in the results section, the pass rate of participants is higher than that of the overall student population in both College Algebra and Intermediate Algebra. For Intermediate Algebra, this makes a great deal of sense. After all, it is precisely this material that is being during our program. Participants who took Intermediate Algebra during their first semester had either placed in the initial remedial mathematics course, thus saving themselves a semester of remedial mathematics or had initially tested into Intermediate Algebra. Either way, these students encountered a review of material that was in their first mathematics course. Therefore, it is reasonable that these students might perform better than the average student taking Intermediate Algebra since they had just had a review of the material.
There is also, of course, the issue of self-selection and the difference in motivation levels of participants of the program versus the regular student population. It is reasonable to assume that not all students want to spend part of their summer doing mathematics. This means that either the participants themselves or their parents may have a stronger motivation to pass their mathematics course. There are several factors that could reasonably be attributed to this motivation including participants' level of academic responsibility or their level of math anxiety. It would also be misleading if the parents' role were not also given consideration. Parents are often the motivators behind a student signing up for this program in the first place, but the participants must do the actual work. It seems reasonable to assume that at least some of the participants who do not complete the course would fit into the category of high parent motivation, but low individual motivation. Parents can enroll the participants and drive them to the exam retake, but they cannot compel them to successfully finish the program. However, the data suggests that even being exposed to a small amount of material and completing a retake of the Maple TA exam does increase student success. This may imply that even though student motivation was initially low, they feel increased internal or external pressure to successfully complete their first mathematics course.
The higher performance of participants than the overall student population in College Algebra is rather surprising. These students all initially tested into a lower course. By definition this would mean that they are likely weaker students than those who placed into College Algebra without needing a refresher. On the positive side, 30 students were spared the expense and discomfort associated with taking a Universal Journal of Educational Research 5(6): 1039-1044, 2017 1043 remedial mathematics course. Our success rate was substantially higher than that of the total population of College Algebra students for the same semester. While we had hoped that the success rate would be near that of the overall student population, we had expected it to be near or lower than the overall student population [16].
It is possible that our success rate was merely an anomaly, which future data should bear out one way or the other. It may also be that self-selection by the students participating in the program was a factor [17]. Additionally, providing students with additional material they could refer to during their course may have provided them with external support and reference materials that made a substantial difference. Although it is tempting to believe that this program really could increase the pass rates for the course, there may be other issues related to the course that we have no control over that artificially lowered the pass rate that semester, such as staffing or scheduling. Further research and analysis is needed.
Another challenge is outreach. This program was originally designed to increase the success of students in the College Algebra course. We have demonstrated that it is possible for students placing in a remedial mathematics course to increase their placement score and be successful in a higher level mathematics course. However, we want to convince students who test into College Algebra to spend additional time over the summer reviewing material because we think it may improve their success. The subsequent year's program did include at least some members of this population and we may soon have at least a partial answer to that question.
An additional challenge, one shared with the courses we are preparing students for, is the drop rate. Of 81 participants, only 50 actually completed the four-week program. Obviously, this is not a unique issue for our program, but considering that it is a voluntary program and upon completion of the program, the initial investment of fifty dollars is essentially refunded to, it is still disappointing. A possible factor may be that the primary stakeholder is the parent not the student. It is possible under the current system for a parent to sign up their child without the participant's advanced knowledge or consent. It may also be that the participants were not prepared for the accelerated nature of the program. We need to consider this in the marketing of the program to students. Attempting to find the appropriate balance between encouraging participants to enroll in the program and ensuring they have realistic expectations about the workload will likely be an ongoing concern.
We continue to struggle with administrative issues. There are many stakeholders concerned with the success of incoming students. Each stakeholder has their own agenda and vision of what needs to be done to ensure student success in their first semester. And there remain disagreements about the importance of mathematics in this process. Some stakeholders believe that it would be better for students to take so-called easier courses and wait to take mathematics courses later. However, others believe that taking the first mathematics course while the material is still relatively fresh in a student's mind is best. For STEM majors, at least, it appears that there is some consensus that mathematics is needed as early as possible. We are hopeful that as we obtain additional data we can convince more stakeholders of the utility of our program. In addition, we have begun to collaborate more with the faculty members involved in coordinating developmental mathematics and college algebra and hope that this will also improve outreach and reduce the overall workload for all faculty involved.

Conclusions
The online summer mathematics bridge program provided some promising results. A majority of the participants passed their first semester mathematics course. Mathematics placement exam scores significantly improved and over half of the participants were able to take a higher mathematics course than they had initially been placed into.
There are still substantial areas for improvement. For example, there are still a large number of potential participants who choose not to participate in the program. We hope to achieve additional academic advisor "buy in" so they will encourage more of their new advisees to participate in the program. We also want to raise our completion rate. Similar to the courses we are trying to prepare the participants for, there is a significant drop out after the first few days of the program. We are exploring ways to alleviate this issue.
We remain cautiously optimistic about the program and its outcomes. Enrollment has been steadily increasing and a majority of the participants have received a tangible, positive benefit from the program.