The IDEA Center

Form Type: Diagnostic and Short Form
Areas of Best Practice: Faculty Buy-in, Teaching Improvement, Faculty Evaluation, and Student Buy-In.

Faculty Buy-In

The Art Institute of California – San Francisco began using IDEA shortly after the campus was instituted in July of 1998. Since evaluation has always been done, it is a part of the culture and viewed to be an informative process. Results are used in the annual review process and to guide improvement efforts.

Faculty members are asked to use the Diagnostic Form when they are first time teachers and at least once a year if they are continuing faculty. The Short Form is used in other courses. They are encouraged to structure focused questions to guide student comments. This helps ease the concerns of faculty who are resistant to numbers and statistics. In addition, the Director of Faculty Development conducts classroom observations and provides a summary to the faculty member and his or her immediate supervisor. By using multiple sources of information, faculty view IDEA to be one part of a multi-part system, including access to input from a resident expert in teaching, learning, and the IDEA system. This helps faculty balance the results of the IDEA instrument with comments from students and the observations of the Faculty Development office.

One important part of the process that fosters faculty trust is the opportunity to “review and comment” on the classroom observation summary before it is sent to the faculty member’s supervisor. The faculty member has an opportunity to review the summary and suggest edits to clarify the context of the observations. The faculty member can also request that another observation take place on a different day, with the understanding that the summary from the first observation will be destroyed. This review/edit process supports an environment of trust and helps the Director of Faculty Development build rapport with the faculty with whom he then works to improve teaching and learning. It also ensures that the student ratings are not the only source of information used in the annual review process.

The Director of Faculty Development serves as the “resident expert” and consults with individuals, or groups of individuals, to help interpret IDEA results. He was initially trained by his predecessor and also attended a number of conference call workshops where IDEA Center staff provided training on the use of the system.

Teaching Improvement

The Director of Faculty Development reviews all IDEA reports before they are distributed to faculty. In conjunction with a review of student comments and classroom observations, faculty members are identified that may need some additional guidance. Information from IDEA has been used as a starting point to discuss ways to improve classroom performance. Specifically, results are used to identify strengths and to show faculty improvement trends across multiple semesters. Having this information available has helped build the confidence of new and struggling instructors.

The results also have been used to guide the placement of instructors. From an analysis of the trends across courses and semesters, some faculty have been identified to be more successful in certain types of courses (e.g., upper level versus lower level) and are given assignments in which they can be most effective.

Faculty Evaluation

IDEA results are used as part of the annual evaluation. Each quarter, the Director of Faculty Development prepares a graph (will link to an example) that plots results for two summary measures, Progress on Relevant Objectives and Overall Excellence of Instructor, so that supervisors can see at a glance the faculty who are in top, middle, and bottom thirds. This helps administrators deal with a large amount of information in a summarized fashion.

Student Buy-In: Responding Thoughtfully

In order to get students to better understand the importance of the rating process, the College Life Skills course will incorporate a section that introduces to the IDEA system and inform them about the instrument, its use, and the importance of their careful completion of the surveys.

Prior to the administration of the forms, one faculty member reviews some of the items with students. He might point out two objectives that were not relevant to the course (i.e., of minor or no importance) and instructs students that if they rate these items as “5” because they liked him or the course, they are not completing the survey appropriately. Similarly, he points out a relevant (“important” or “essential”) objective and suggests that if they indicate making no progress on that objective because they didn’t like him or the course, they are probably not completing the survey appropriately. This process emphasizes to students that the IDEA system is not assessing how much they “like” or “dislike” the instructor or the course, but encourages them to carefully read and discriminate between the items.