Lawrence M. Aleamoni, PhD
Professor Emeritus, Department of Disability and Psychoeducational Studies, University of Arizona, Tucson, AZ 85721, USA; Tel. (520) 297-5110; (520) 665-9988; Fax: (520) 297-9427; E-mail: aleamonl@email.arizona.edu
Every year academic administrators must evaluate faculty performance for the purpose of making retention, promotion, tenure, and merit pay decisions. Inevitably questions of objectivity, reliability, and validity arise during the processes involved in making these decisions. This article describes the procedure for an academic administrator or faculty committee member charged with designing and implementing a faculty evaluation system, to follow in developing a comprehensive, integrated, organized system for evaluating faculty. The steps described herein will result in the development of a customized faculty evaluation system which meets the specific needs and characteristics of the individual division, department, college or other academic unit.
The model for developing a comprehensive faculty evaluation system described herein has been successfully used by many colleges and institutions to create a system that works best for that individual college or institution. No two colleges or institutions using this approach may necessarily come up with the same system – although similarities will exist, of course, at least to the extent that the assumptions implicit in the model are accepted.
The development of a comprehensive faculty evaluation system is a challenging and time-consuming process. There is no shortcut that will lead to a valid, fair, and useful system although some procedures have been successful in accelerating the process. However, the process of developing a fair and valid faculty evaluation system requires that the administration of the institution be committed to the project and be willing to provide the necessary support for the work that needs to be done. Experience has shown that following the eight steps briefly described below for developing a faculty evaluation system greatly facilitates the process. The faculty evaluation system, developed using the eight steps, will have the greatest probability of acceptance and successful use by the faculty and administrators, because both constituencies will have had early and ample input to its design and construction.
The reason for this is that the design of any successful faculty evaluation system must be predicated upon and reflect the values, priorities, traditions, culture, and mission of the institution. Unless the faculty evaluation system adequately reflects and includes these issues in its design, it is unlikely to be accepted by the faculty or function appropriately from an administrative perspective. Simply adapting or adopting the forms and procedures developed by one institution does not guarantee those forms and procedures will work at another institution.
The process for developing a faculty evaluation system using the eight steps assumes that there is no one best faculty evaluation system that could be successfully applied to any and all colleges and universities. To that extent, then, the eight steps for developing a comprehensive faculty evaluation system may be considered a proven process for developing a customized faculty evaluation system rather than a best practice.
Experience has shown that a necessary part of the process of developing a successful faculty evaluation system is the planned and systematic inclusion of faculty input. In this regard the best approach to developing a faculty evaluation system is to appoint a committee composed primarily of faculty, a few key administrators, and perhaps even a student or two (depending on the institution’s culture and traditions), which is responsible for gathering the information and following the eight steps. Thus, the various steps in the process should refer to the Committee as the operational entity carrying out the process. If the process is carried out primarily, or exclusively, by a single administrator or by an administrative group, the probability of a successful outcome is greatly reduced.
The process of developing a faculty evaluation system involves attending to the technical requirements of good measurement and the political process of gaining the confidence of the faculty. Thus, a well-designed comprehensive faculty evaluation system may be defined as one which involves:
By design, any faculty evaluation system developed using this model interprets all measurement data by means of a predetermined, consensus-based value system to produce consistent evaluative outcomes. For example, a partial list of possible faculty roles to be measured are: teaching, scholarly and creative activities, professional recognition, and service. The possible sources are: students, peers, department chair, self, etc. The measures should take place when there is enough information to reliably and validly characterize the faculty member’s performance.
It should be noted that faculty evaluation and professional enrichment are really two sides of the same coin. Ideally, faculty evaluation programs and professional enrichment programs should work hand-in-hand. If some aspect of faculty performance is to be evaluated, then there should exist resources or opportunities that enable faculty to gain or enhance their skills necessary for that performance. For maximal self-improvement effect, faculty evaluation systems must be linked to professional enrichment programs.
A successful faculty evaluation system must provide 1) meaningful feedback information (in both quantitative and qualitative form) to guide professional growth and enrichment and 2) evaluative information on which to base personnel decisions is presented as an overall composite numerical index representing a summary of the faculty member’s performance based on the steps described below. These two purposes can be well served by one system. The key to constructing a system that serves these differing purposes is in the policies determining the distribution of the information gathered. The general principle to be followed is that detailed information from questionnaires or other forms should be given exclusively to the faculty member (by a faculty committee at the department level) for use in professional enrichment and growth efforts. However, aggregate data that summarize and reflect the overall pattern of performance over time of an individual can and should be used for such personnel decisions as promotion, tenure, continuation, and merit raise determination.
Step 1: Determining the Faculty Role Model
The objective of Step 1 is to have each department identify and define the roles faculty play in the department. This is determined by taking an inventory of the actual activities in which the faculty engages in pursuing their professional responsibilities. In this step faculty can generally easily identify the activities that, for them, define the traditional roles of teaching, scholarly and creative activities, service, and administration or management.
Step 2: Determining Faculty Role Model Parameter Values
The objective of Step 2 is to begin the process of defining the value structure on which the evaluation system will ultimately be based. In this step the department begins to establish and specify the relative importance of each role to the department/institution. Here faculty is asked to determine how much value or weight they believe should be placed on each role in the faculty role model that resulted from their work in Step 1.
Step 3: Defining Roles in the Faculty Role Model
The definition of the specific roles in which faculty engage is the last step in the process of building the faculty role model upon which the evaluation system will be based. As noted earlier, it is assumed that a specially appointed Committee will coordinate the detail work associated with this project. Step 3 involves reaching a consensus on how each of the roles identified and briefly defined in the previous steps are to be completely defined.
Step 4: Determining Role Component Weights
At this point, definitions will have been developed for the various roles in the faculty role model (Step 3). The relative impact or parameter values that the different roles can take in the overall evaluation of a faculty member will have been determined (Step 2). Depending upon the definitions developed for each role in Step 3, one may have also identified specific subsets of performances or components of various roles. For such roles it now becomes important to consider how much weight or relative importance the various components of each role should have in the overall evaluation of that specific role. That is, one must express the proportion or weight that will be given to the performance of each component in the evaluation of the total role.
Step 5: Determining Appropriate Sources of Information
In Steps 1-4, we focused on determining and defining the roles that should be evaluated, how much weight or value should be placed on the performance of each role in the overall evaluation, and how much weight the individual components of each role contribute in the evaluation of that role. This step, then, is to decide who should provide the information on which the evaluations will be based. The most important principle in identifying and selecting sources of information is to make certain that the source identified has first-hand knowledge of the performance being evaluated.
Step 6: Determining the Source Impact Weights
In any well-designed faculty evaluation system, the evaluative judgments concerning faculty performances in the various expressions or components of the roles should be based on information derived from multiple sources. The issue of the appropriateness of those sources is addressed in Step 5. Having determined where this information is to come from, now the issue of the credibility of those sources needs to be addressed. Thus, specify the weight or impact the information from each source will have in the overall evaluation.
Step 7: Determining How Information Should Be Gathered
In this step, we set about determining how the information we have specified in our role definitions is to be gathered from the sources we have identified and agreed are appropriate.
Step 8: Completing the System: Selecting or Designing Forms, Protocols, and Rating Scale
We now arrive at the last step in developing a comprehensive faculty evaluation system – designing the questionnaires and other forms. Constructing valid and reliable rating forms, questionnaires, or other tools needed to implement the data-gathering strategies specified in Step 7 is a complex technical task requiring expertise in psychometrics. It must be remembered that what is being developed are tools to measure, in a valid and reliable way, complex psychological phenomena (e.g., opinions, reactions, observations, rankings, etc.). Even selecting published forms or other commercially available tools requires fairly sophisticated psychometric skills in order to adequately assess their appropriateness and utility for the faculty evaluation system one has designed.
At this point one is ready to begin using the system. The task now is to combine all the data resulting from the system into a usable form. The appointed committee responsible for gathering the information and following the steps described above will have agreed upon a common scale, i.e., 1 to 4, to be used in reporting all information gathered from each source. That is, regardless of whether a questionnaire, an interview schedule, or some other technique has been used in gathering evaluative information from the various sources identified, that data will be reported on the same scale.
Having determined and specified the weights to be assigned to various activities and sources in the overall faculty evaluation system, it is now possible to compute an overall rating for each role that reflects the collective values of the faculty. This rating will be referred to as the Composite Role Rating (CRR) because it will be derived from information from a variety of sources. Each source will provide information concerning various components of each role. The information from each source concerning each component of each role will be weighted in ways that reflect the consensus value structure of the institution. Although, the CRR does not represent an objective measure, the subjectivity involved in computing it has been carefully controlled and prescribed by the values assigned to the sources and role components.
Note: This article reflects my and my colleague’s, Raoul A. Arreola, over 20 years of experience in conducting National workshops on this topic.
10.17140/CTPOJ-7-125
© 2014-2024 I Openventio Publisher is licensed by Creative Commons Attribution, unless otherwise indicated.
