Summary of Results of Hotspot and iPad Impact on Students in Fall 2020
CALIFORNIA STATE UNIVERSITY, FRESNO
OFFICE OF INSTITUTIONAL EFFECTIVENESS
SIMRAN NAGRA
1 Introduction
In Fall 2020, California State University, Fresno (Fresno State) had the DISCOVERe device loner program which provided currently enrolled students with a hotspot and/or iPad if they needed it. With the ongoing COVID-19 pandemic, most of the classes in the Fall 2020 semester were online, thus making this technology a necessity for the entire student population. For students who needed the device(s), they completed a survey where they selected a time to pick up the device(s) at the Henry Madden Library. In order to pick up the device(s), the students needed to have their student ID card or their ID number. The devices would be returned at the end of the semester. The devices were dispersed on a first-come, first-served basis.
With the ongoing pandemic, technology was one the many barriers students faced when completing their education. Was there any impact in student’s academic outcomes, specifically GPA and unit amount, when provided with these device(s)?
Method
Population Demographics
The population size was 22,564 undergraduate students. There was a total of 3,105 students who received a hotspot and/or iPad during the Fall 2020 semester. There were 19,459 students who did not receive any device during the same semester. When running the background statistics of the students who received a device and those who did not, there were observed differences between these groups.
When looking at the demographics (age, student level, Pell grant, underrepresented minority, and first-generation status), those who received a device were more likely to be Pell grant eligible (68.2%), an underrepresented minority (66.9%), and a first-generation student (72.8%). The non-treated group did have students with similar backgrounds, but the percentages were not as high (58.0% were Pell grant eligible, 58.3% were underrepresented minority, and 64.8% were first-generation students).
Match It
I used MatchIt in R (Ho et al., 2011) to create a control group that had similar characters as the students who received a hotspot and/or iPad. The demographic characteristics taken into consideration were: age, student level, Pell grant status, if they were an underrepresented minority, and first generation status. Graduate students were not included in this sample. The population and created sample were all compromised of undergraduate students.
The MatchIt methods that were considered to see the best balanced control and treatment groups were: full, optimal, nearest neighbor, and exact. Nearest neighbor was selected as the best control-treatment group because of the overall improvement in balance and plots from the unmatched sample to the matched sample. Nearest neighbor matches participants based on pairing the closest control participant to the closest treated participant (Greifer, 2021). Nearest neighbor does not account for other pairing, so it is not considered an optimal matching method (Greifer, 2021).
Sample Size and Demographics
In Fall 2020, 870 undergraduate students received only hotspots, 1,275 undergraduate students received iPads, and 960 undergraduate students received both. Within this group, there was a total of 6,210 students in this study, 3,105 in the control group and 3,105 in the treatment group. The control group had students who did not receive any device from Fresno State in the Fall of 2020. The treatment group had students who received a hotspot and/or iPad from Fresno State in the Fall of 2020. After the MatchIt program was run, the treatment and control groups had values in the demographic areas that were close to each other. Average age was 22.4 years for the treatment group and 22.2 years for the control. Student level, a scale that ranged from 1 for Freshmen to 4 for Seniors, was 2.9 for both groups. Pell grant eligibility status was 68.2% of the sample for the treated group and 68.5% for the control group. Underrepresented minorities made up 66.9% for the treatment group and 66.9% for the control group. First-generation college students made up 72.8% of the treatment group and 72.3% of the control group.
Results
Grades and Units
After nearest neighbor match was conducted, a 2-way between-subjects ANOVA was conducted for each of the dependent variables: Units at the Beginning of Term, Units Attempted, Units Earned, and Term GPA, all for Fall 2020. Units Attempted is the amount of units a student attempted for the Fall 2020 semester. Units Earned is the amount of units the student earned at the end of the semester. Term GPA represents their average grade for courses taken in Fall 2020.
For units earned at the beginning of the Fall 2020 semester, there was no significant difference between the control (M=12.23, SD=3.83) and treatment group (M=12.44, SD=3.81) for those who received a hotspot. For the Units in Fall 2020, there was no significant difference between the control (M=12.23, SD=3.83) and treatment group (M=12.13, SD=4.09) for those who received an iPad. There was no significant interaction between hotspots and iPads. There were 176 participants who excluded from the analysis because of missing values. The means and standard errors for each of the groups are displayed below in Figure 1.
Figure 1. Average unit amount and standard errors.
For the Units Attempted, there was a significant difference between the control (M=13.38, SD=3.04) and hotspot group (M=13.68, SD=2.85), F(1, 6206)=22.82, p<.001. Students who received a hotspot attempted a significantly higher amount of units than those who did not receive a hotspot. For the Units Attempted, there was a significant difference between the control (M=13.38, SD=3.04) and iPad group (M=13.79, SD=2.88), F(1, 6206)=26.21, p<.001. Students who received an iPad also had a significantly higher amount of units they attempted than those who did not receive an iPad. There was no significant interaction between hotspot and iPads. The means and standard error are displayed below in Figure 2.
Figure 2. Means and standard error for the groups that had significant difference from the control group.
For the Units Earned, there was a significant difference between the control (M=11.80, SD=4.52) and hotspot group (M=12.57, SD=4.03), F(1, 6206)=22.00, p<.001. Students who received a hotspot earned more units than those who did not receive a hotspot. For the Units Earned, there was not a significant difference between the control (M=11.80, SD=4.52) and iPad group (M=11.89, SD=4.66). There was no significant interaction between hotspot and iPads. The means and standard errors are displayed below in Figure 3.
Figure 3. Means and standard error for the control, hotspot, and iPad groups. Those who received a hotspot earned more units than the control group.
For term GPA in Fall 2020, there was a significant difference between the control (M=3.06, SD=1.00) and treatment group (M=3.30, SD=0.84) for those who received a hotspot, F(1, 6030)=28.44, p<.001. Students who received a hotspot had a higher GPA than those who did not receive a hotspot. For term GPA in Fall 2020, there was not a significant difference between the control (M=3.06, SD=1.00) and iPad group (M=3.08, SD=0.97). There was a significant interaction between hotspots and iPads (M=3.14, SD=0.94), F(1, 6030)= 10.32, p<.01. There were 176 participants who excluded from the analysis because of missing values. The means and standard errors for each group are displayed below in Figure 4.
Figure 4. Average GPA and standard errors for each group.
Retention and Graduation
GPA and unit amounts are not the only way to track student success. Another important factor is if students continues in the same university or graduates. This reflects on the success of the university if the students continued or completed their education, which is the overall goal of every university. This variable is called persistence. If a student graduate or was retained at Fresno State, they were counted as ‘persisted’1. If a student did not enroll in the Fall 2021 semester or/and did not graduate, they did not persist. A logistic regression was conducted in order to see how hotspots and iPads predicted the odds of persisting. Students who received hotspots were statistically significantly 1.8 times more likely to persist than those who were in the control group, B=0.59, SE=0.13, p<.001. There was no significant difference in persistence between students who received an iPad and the control group, B=0.10, SE=0.10, p=.28. Students who received both devices were statistically significantly 1.17 times more likely to persist than those in the control group, B=-0.53, SE=0.18, p<0.01. Generally, when odds are greater than one, this means receiving one or both of the devices means that students were more likely to persist than the students in the control group. However in this analysis, it was only students who received a hotspot and both devices where this odds ratio was significant and observed differences in persistence were unlikely due to chance.
Discussion
Overall, there were some significant differences between the control group and the treatment groups in regard to if they received a hotspot and/or iPad. Before students are starting the semester, there are no significant differences between any of the groups. However, as students start to enroll in classes, students who received a hotspot or an iPad enrolled for more units than those who did not have any devices from Fresno State. At the end of the semester, the students who received a hotspot earned more units than those who did not receive any device from Fresno State. For GPA, students who received a hotspot had a higher GPA than those who did not receive a device. Students who received a hotspot and an iPad had a higher GPA than those who only received an iPad, but they did not have a higher GPA than those who only received a hotspot. Interestingly those who only received an iPad did not have statistically significant higher GPA than the control group.
For cumulative GPA, students who received a hotspot had a higher cumulative GPA. Similarly with students who received an iPad, they also had higher GPA than those without any device received. Additionally, students who received both devices had a statistically significant higher GPA than those who did not; however, those with both devices did not have higher cumulative GPA than those who only received a hotspot. Lastly, for cumulative amount of units, students who received a hotspot from Fresno State had a greater unit amount than those who did not receive a hotspot. In contrast to the rest of the results from this analysis, those who received an iPad a statistically significantly lower amount of cumulative units than those who did not receive any device. Additionally, while there was a significant interaction for those who received both devices, the average units showed that it was the control group that had a higher unit amount than those who received both devices.
In looking at persistence, when a student continued to the next semester or graduated with their degree instead of disappearing from campus records, providing a device has helped student persist in their education. People who received hotspots were more likely to persist, followed by receiving both devices, and lastly iPads.
The most consistent variable that showed a significant different with units and GPA was hotspots. Students who had hotspots attempted more units, earned more units, had higher semester and cumulative GPAs. There are benefits to receiving iPads as well, but receiving a tablet without Wi-Fi does not help students in attending online classes. Interestingly, with the overall units students have earned, it was the control group that had more units than the iPad and the hotspot and iPad groups. While this matched the results where students who received an iPad did not earn more units at the end of Fall 2020 like the students who received hotspot groups, this inconsistency opens lines of questions about the use of these devices in student’s lives. Additionally, even when receiving both devices was significant, the GPA or unit amount did not surpass the hotspot group. Overall, providing hotspots to students supported students more than iPads.
Interesting lines of questions that arose from analyzing this data are: How are students using iPads? Did iPads have everything students need in order for them to complete their classes? Did receiving an iPad and a hotspot replace the function the Henry Madden library provides for students? Limitations for this analysis include that the effects of the pandemic, wildfires, and environmental conditions could not be measured. In Fall 2020, students had to learn from home (or wherever they were sheltering at) therefore, there could have been background variables that could have impacted their academic performance such as strength of the Wi-Fi connection or loud environment that can interfere with learning and studying. These factors could have impacted students throughout the Fall 2020 semester and could have influenced student’s academic performance.
Recommendations based on this analysis is to invest in hotspots so any student who is in need of a hotspot are able to receive them. Additionally, inquiring about tablet use would be beneficial to see how students used the tablets, to see if the iPads are the best for students, and if there are supplemental materials needed with it (e.g., a keyboard or the pen to write on the screen with). In the roll out of the device loner program, students had to complete a survey to pick up the device(s); therefore, it is unknown if all of the students who needed a device received them before the device(s) ran out. If there are more devices available, having a longer roll-out and more advertising may help students in need access the devices. Finally, as the campus is opening up and the COVID-19 pandemic is being controlled with the vaccines, it would be beneficial to see student outcomes during the Spring 2021 semester to see if the impact of receiving devices are consistent across semesters.
References
Greifer, N. (2021, May 26). Estimating Effects After Matching. Matching methods. https://cran.r-project.org/web/packages/MatchIt/vignettes/estimating-effects.html
Greifer, N. (2021, May 26). MatchIt: Getting Started. Matching methods. https://cran.r-project.org/web/packages/MatchIt/vignettes/matching-methods.html
Ho, D.E., Imai, K., King, G., and Stuart, E.A. (2011). MatchIt: Nonparametric Preprocessing for Parametric Causal Inference. Journal of Statistical Software, Vol. 42, No. 8, pp. 1-28.
This HTML version was generated with AI assistance and may contain errors or omissions. The original approved PDF remains the official document of record. An alternative accessible format is available upon request from the document creator in accordance with Title II of the ADA.