Student Ratings Task Force Report_2010

CALIFORNIA STATE UNIVERSITY, FRESNO College of Health and Human Services Department of Kinesiology 5275 N. Campus Drive MIS SG28 Fresno, CA93740-8018 559.278.2016 Fax 559.278.7010 TO: Dr. Michael Caldwell FROM: Ad Hoc Committee (of the Academic Senate Expanded Executive Committee) to Investigate Student Rating Instruments John Constable / Thomas Holyoke Dawn Lewis, Chair1(f Paula Popma . Brian Tsukimura DATE: August 5, 2010 SUBJECT: Committee Report regarding Investigation of Student Rating of Faculty Instruments The Ad Hoc committee’s charge was 1) to research student ratings of faculty instrument options for our campus including external vendors and instruments developed internally by California State University peer institutions, 2) to research and deliberate the potential for on-campus development of an instrument, and 3) to review the finding of the 2008 RFP Committee to aid in the process. Upon completion of this charge, the committee was expected to submit a summary of its investigation to the Expanded Executive Committee (EC). The following report of the committee’s findings includes A) a description of the methods used to evaluate student rating forms, B) a summary of internally developed forms used by California State University peer institutions, C) a review of forms developed by external vendors, D) whether forms and services comply with California State University Executive Order No. 926 Policy on Disability Support and Accommodation, E) a budget summary, and F) the committee’s recommendations for the selection of rating forms. The committee’s full recommendations are given in Section F which are mainly to issue a Request for Proposal (RFP) to appropriate vendors to receive full and accurate description of services and binding cost estimates, to select and implement the forme s) and services of an external vendor that meets the required and desired features outlined in Section F, and to include an expert on accessible technology and services for persons with disabilities on all subsequent committees regarding this charge. THE CALIFORNIA STATE UNNERSITY
2. How were the common questions developed? 3. Can departments add questions? Who develops these questions? How flexible is this? 4. How did you demonstrate reliability and validity? 5. Can the forms be delivered in paper, online, or both? 6. Who processes the ratings forms? 7. How are the results interpreted? How is the data normed? 8. Is there space for written comments? 9. Are the ratings used for summative and/or formative purposes? 10. How did you implement? Was there training? 11. Is the form in compliance with section 508 of the Rehabilitation Act of 1973? 12. What are the costs involved – forms, supplies, equipment, staff?
13. Questions for California State University Peer Institutions
Ad Hoc Committee to Investigate Student Rating Instruments Subcommittee of the Extended Executive Committee of the Academic Senate Summer 2010 Questions for CSU Peer Institutions using a Common Student Rating of Faculty Form 1. We understand you use a common form, is that correct?
See Appendix C External Vendors’ Contact Information IDEA The IDEA Center 211 South Seth Child Road Manhattan, KS 66502-3089 800-255-2757 or 785-320-2400 Fax: 785-320-2424 E-mail: info@theideacenter.org SIR II by ETS Diana C. McNeil Account Manager, West Region Higher Ed Division - College & Graduate Programs MS 40-L - RM# L289-R 609-683-2338 (direct line) 609-683-2040 (fax) E-mail: dmcneil@ets.org Website: www.ets.org/hea The Instructional Assessment System (IAS) Joanna Loss Manager (206) 543-9847 voice (206) 543-3961 fax jmloss@u.washington.edu CourseResponse by Digital Measures Dana Clark Senior Account Executive Digital Measures Phone: (414) 238-6212 Mobile: (262) 951-8355 Fax: (414) 918-4602 Mail: 301 N. Broadway, Floor Four, Milwaukee, WI 53202 dclark@digitalmeasures.com http://www.digitalmeasures.com The SEEQ Herbert Marsh, Ph.D. Department of Educational Studies St Cross College Oxford University herb.marsh@education.ox.ac.uk +44(0)1865274041



































Table 2 Comparison of External Vendors of Student Ratings of Teaching Performance
IDEA Long Form IDEA Short Form IAS SIR II SEEQ Questions Base questions 47 12 2 31 40 32 (developed 1970s & 1980s) Instructor written questions Yes Yes 12 10 9 Questions for Formative / Summative purposes Yes 18 formative / 4 summative (global) Yes 29 Q formative / 3 Q summative Questions about student 12 6 9 10 Questions separated into known teaching dimensions Yes Yes No Yes 8 dimensions Yes 9 dimensions Likert scale type answers Yes Yes Yes no neutral answers Yes no neutral answers Yes w/ neutral answers Questions "validated" Yes Yes Yes Yes Yes Written comments possible? Yes Yes Yes Yes Yes Short form available? Yes --- No but 13 different types of forms available No No












































The committee does not recommend the development of an internal instrument of student rating of faculty at California State University, Fresno.
The committee finds that IDEA satisfactorily meets many of the criteria outlined in Appendix B and should be given additional consideration by the University.
The committee finds that ETS’ SIR II and services satisfactorily meet several of the criteria outlined in Appendix B and should be given additional consideration by the University.
It is this committee’s conclusion that there is nothing to be gained by using Digital Measures CourseReponse because it requires the university to develop its own form(s). Therefore, in summary, the committee does not recommend further consideration of the CourseResponse service for student rating of faculty.
Because administration, analytical and report services for the rating form are not provided by an external vendor associated with the instrument, the committee does not recommend the SEEQ for further consideration by the University.
This HTML version was generated with AI assistance and may contain errors or omissions. The original approved PDF remains the official document of record. An alternative accessible format is available upon request from the document creator in accordance with Title II of the ADA.