Skip to main content Skip to main navigation Skip to footer content

Office of Institutional Effectiveness

Data & Research

Reports, Surveys & Studies

ACT CAAP Reading Test Results
General Education Area IC Courses
California State University, Fresno

April 2005

By Priscilla Chaffe-Stengel
Professor, ISDS and Faculty Fellow, OIE

In Spring, 2005, faculty teaching sections of General Education courses that integrate the arts and humanities (GE Area IC) were encouraged to make one class meeting available in which to give their students ACT’s Collegiate Assessment of Academic Proficiency (CAAP) Reading Test. One section of freshman students enrolled in University 1 was included in the study. In April, the University Test Office coordinated the distribution and collection of examinations with faculty. A set of 695 students completed the examination.

Institutional Sample. Descriptive statistics provided in ACT’s institutional report indicated that approximately 38% of the sample identified themselves as Caucasian, 74% as having English as their first language, and 83% were 25 years of age or less. The sample was considerably younger than the campus student body as a whole, where institutionally less than 53% of the student body is 24 years of age or less. Nearly 29% of the sample identified themselves as seniors, 53% as juniors, 14% as sophomores, and 4% as freshmen.

Overall Performance. The CAAP Reading Test reports a scaled maximum score of 80. Institutional scores on this test ranged from 43 to 73, with an average score of 60.4 and a standard deviation of 5.92. The national average was 62.5 with a standard deviation of 5.3. A quarter of our students earned a scaled score of 63 or better. Compared to the nation, 25% of students earned a score of 65 or better. Whereas 31% of our students scored 56 or below, in comparison 30% of students across the country earned a score of 59 or below. As a whole, our students tended to lag behind the national norm by only two to three scaled points out of the total of 80 available on the CAAP Reading Test. As yet we have not asked ACT to identify the mix of institutions represented in their estimate of the national norm, so we currently do not know the degree to which small, private colleges and large, comprehensive universities participate in that norm. The good news for our campus remains that a modest improvement in student reading scores can move us ahead of the national average.

Gender. Gender differences were not evident in the results of this test (t not significant at P = .42). Men earned an average of 60.2 compared to an average of 60.5 for women. Males comprised approximately 40% of our campus sample taking the test. University figures report approximately 40% of our campus student body is male, so our sample tested for reading is representative of the gender distribution University-wide.

Transfer Students. Transfer students performed significantly better on the CAAP Reading Test score than first-time freshmen (t significant at P = .02). For the 327 students who identified themselves as first-time freshmen to our campus, their average score was 59.8 compared to an average score of 60.9 for transfer students. Transfer students represented approximately 52.5% of the sample taking the test. This contrasts to a general trend of two-thirds of our students campus-wide who transfer in from other institutions. It is not known what bias the under-representation of transfer students introduced into the test results.

Non-Professional Majors. Non-professional majors performed significantly better than professional majors (t significant at P < 10-5). 269 students, or 44% of the sample, indicated they were majoring in non-professional areas of study and averaged 61.7 compared to an average of 59.7 among the 337 students who indicated they were majoring in professional areas of study. Across the campus as a whole, approximately 45% of students major in traditional non-professional areas of study.
Ethnicity. Caucasians outperformed their counterparts with measurable significance (t significant at
P < 10-13). Students who identified themselves as Caucasians scored an average 62.5 on the test compared to non-Caucasians who averaged 59.0. Approximately 45% of our student body campus-wide identify themselves as Caucasian or white, in contrast to approximately 38% of our sample tested.

Full-Time Students. Even though nearly 93% of the students specified themselves as full-time students, there was no statistical difference in their mean performance compared to part-time students (t not significant at P = .49). 644 full-time students averaged 60.36 compared to 60.96 for 51 part-time students. The sample is representative of the campus cohort, where approximately 93% of our campus student body enrolls in classes on a full-time basis.

Non-Professional Majors vs Transfer Status. To test whether the differences in test scores found between non-professional and professional majors was affected by any differences in transfer versus first-time freshman status of students, separate analyses were run within non-professional majors and within professional majors to test for any differences in reading test scores by transfer versus first-time freshman students. Within non-professional majors, differences in reading test scores for transfer versus first-time freshmen were not significantly different (t not significant at P =.20). Within professional majors, differences in reading test scores for transfer versus first-time freshmen were not significantly different as well (t not significant at P =.06). There was not sufficient evidence to conclude that transfer versus first-time freshman status of students had any measurable effect on the significant differences noted between non-professional and professional majors.

CAAP Reading Test Score as a Function of IC Grade and Total GPA. The total grade point average and the individual class grade students earned in their GE IC class were regressed on the CAAP Reading Test score each student earned. The resulting regression model was highly significant (F significant at P < 10-18) with both partial regression coefficients significantly and positively correlated to the reading test score. This indicates that students who perform better on the CAAP Reading Test do better in their grade for their GE IC class, and vice versa. This also indicates that students who perform better on the CAAP Reading Test have higher total GPAs, and vice versa. Approximately 13% of the total fluctuation in the reading test score (R Square) was explained by these two variables. The total GPA was comparatively the more potent of the two measures in two ways: (1) the P-value for total GPA was significant at 10-10 compared to the P-value for the GE IC grade which was significant at 0.013; and (2) the size of the partial regression coefficient indicates that for each increase of one point on a student’s total GPA, the regression model predicts an increase of 3.4 points on the CAAP Reading Test compared to an increase of just 0.8 points on the test for each jump in a grade earned in the GE IC class.

CAAP Reading Test Score as a Function of IC Grade, Total GPA, and Campus Units Completed. The introduction of campus units completed as a third independent variable into the regression model described immediately above reduced the overall significance of the model from P = 10-18 to P = 10-14. More importantly, however, the partial regression coefficient for campus units completed was not significantly correlated with performance on the CAAP Reading Test (t not significant at P =.62). There was not sufficient evidence for us to conclude that the number of campus units students had completed affected their reading score on the test.

CAAP Reading Test Score as a Function of IC Grade, Total GPA, and Age. The introduction of the student’s age in 2005 as a third independent variable into the regression model described above reduced the overall significance of the model from P = 10-18 to P = 10-12. More importantly, however, the partial regression coefficient for campus units completed was not significantly correlated with performance on the CAAP Reading Test (t not significant at P =.73). There was not sufficient evidence for us to conclude that the student’s age in 2005 affected their reading score on the test.

RECOMMENDATIONS

First-Time Freshmen. Campus attention should be paid to improving the reading skills of entering first-time freshmen. Since an estimated 50% of our first-time freshmen enroll in University 1, an early intervention might be mounted in coordination with this class.
Professional Majors. Campus attention should be paid to improving the reading skills of students declaring majors in professional areas of study, including agriculture, business, engineering, and health/human services. To the degree that professional schools have sequenced curricula, intervention might be effectively planned to intercept professional majors early in their upper-division coursework in the major.

Reading and Grades. Students who read better earn better collegiate grades. However, students who have more campus experience and accrue more campus units do not on their own improve their reading skills. As well, the more mature a student is has no effect on how well he or she reads. Put simply, students read well and get better grades, or they don’t read well and they get poorer grades. Improving student reading skills holds the promise of improving student academic performance more broadly. Campus attention should be paid to targeting effective pedagogical practices that break through this self-perpetuating cycle to improve student reading skills. Faculty development in the area would be helpful.

Assessment Grants Relating to the ACT CAAP Reading Test Results

top

Resource Links

Campus Links