Colorado State University

Worldwide Research Project:

Department of Psychology
Industrial and Organizational Psychology
Fort Collins, CO 80523-1876
Phone: 970-491-5233

" Cultural Values and Assessment Center Practices in the Americas, Europe, and Asian Countries "

Dear Assessment Center Expert,

Colorado State University is conducting a study in cooperation with the International Task Force on Assessment Center Guidelines. We cordially invite you to participate in this study. The objective of this project is to gain knowledge about the conception, implementation, and evaluation of Assessment Center (AC) practices in organizations in the U.S., Europe, and Asia. We ask you to describe one present Assessment Center practice in your organization. Furthermore, we aim to ascertain to what extent differences in organizational culture and general national cultural between countries are manifested in current Assessment Center procedures. Your information will be handled confidentially and anonymously. It will be examined only from a scientific perspective. The results will not be made public in any way that identifies individual persons or organizations.

We tried to keep the questionnaire as short as possible, and so we have two parts:
The first part of the questionnaire is about one Assessment Center in your division or organization.
The second part is about general cultural values in your country.

In addition to your answers it is important to get information from someone else in your organization. This second person could be a HR manager who will answer questions about the organizational culture in your division and general cultural values in your country. This additional information will help to understand the AC procedures. Could you give the second questionnaire to someone else in your department?

We believe it is not feasible to translate the questionnaire in all different languages. Therefore we are presenting it in English. We recognize that it may be difficult for some persons whose second/third language is English to answer some of the questions. If you need help, please feel free to contact us (thornton@lamar.colostate.edu or dkrause@lamar.colostate.edu). We would be happy to inform you about the results of our study if you provide us with a contact address. The final results will be presented in future professional conferences and journals.

Please return the completed questionnaire as soon as possible. Thank you very much for your cooperation.

Dr. George C. Thornton III, Professor Dr. Diana E. Krause, Visiting Research Scholar


How Is the Questionnaire to Be Completed?

When you are filling out this questionnaire, please think on the most important AC.
If several ACs are used in your organization, pick the one which is used most frequently in your organization. PLEASE RELATE ALL THE FOLLOWING ITEMS TO THAT AC.
If you like to describe more than one AC, please fill out another questionnaire.

For most questions, check one response. For those questions marked by an asterisk (*), multiple responses are possible. In the category "Other" you can make your own additions. In order for the study to be complete, it is necessary that you answer all questions.
If no Assessment Centers are used at this time in your organization, please answer only the first question.

I. The Use of Assessment Centers

General Information (GI)
GI 1. Are Assessment Centers currently used in your organization? Yes No
GI 2. If Yes, pick one AC and describe that AC. In what part of the organization is this AC used (organizational unit)?
AC for the whole company AC for a specific division
GI 3. In what country does this AC take place? (*)
Argentina Chile France Ireland Netherlands Sweden
Austria China Germany Italy Portugal United Kingdom
Belgium Columbia Greece Japan Peru United States
Brazil Korea India Luxemburg Switzerland
Canada Denmark Indonesia Mexico Spain Other:
GI 4. Where was this AC originally developed?
In own country In another country
GI 5. If it was developed originally for another country, FROM which country TO which other country was it transferred (e. g., FROM U.S. TO Indonesia)?
From U. S. From U. K. From South Africa From Germany Other: From
GI 6. How many employees work at the division where this AC takes place?
Number of employees:
GI 7. How many employees work in the whole corporation?
Up to 500 501 to 2000 2001 to 5000 5001 to 10,000 10,001 to 20,000 More than 20,000
GI 8. Which economic sector does the division belong to, where the AC takes places?
Services Electronic, gas, sanitary services Agriculture, forestry, fisheries Transportation, public utilities Manufacturing Construction
Mining Wholesale trade Retail Trade Finance, Insurance, real estate Communications Government
Other *Please do not use ' & or * in your response.
GI 9. In what year was this AC implemented for the first time? For example, 1995 Year
GI 10. If your AC started MORE THAN FIVE YEARS AGO: How does the frequency of AC usage in the last five years compare to the period before?
Not applicable: AC started less than 5 years ago Decreasing More or less unchanged Increasing
GI 11. What is the main objective of this AC? (Please select only one)
Personnel selection of external applicants Personnel promotion of internal employees to the next level Personnel development of internal employees
GI 12. If the main objective is personnel development, which of the following sub-goals are important? (*)
NOTE: when you see an asterisk (*), select all answers that apply.
Diagnoses for personnel development or training needs Promotion of organizational cooperation/ Team building Stabilization of the leadership culture/Connect the AC to general HR culture
GI 13. To what degree is the successful participation in this AC a prerequisite for advancing one's career?
Not at all Somewhat To some degree Highly important Essential
GI 14. In what functions are you involved in this AC procedure? (*)
Evaluator Moderator Observer Developer Other:
GI 15. Are you employed as a member of the organization or are you an external expert?
Member of organization External expert
GI 16. If you are a member of the organization, what is your function within the organization?
Executive level Managing Director Head of Department Chief Department Head
Division Manager Group Leader Employee/ HR expert Other
Development of the AC (DE)
DE 1. Were the Guidelines and Ethical Considerations of Assessment Center Operations considered in the development of this AC? Yes No
DE 2. How has this AC procedure been tailored to the division?
Use of standard AC developed elsewhere Adaptation of a standard AC to this division Development entirely according to this division's own needs
DE 3. How long is an average AC procedure? (in days)
Less than 0.5 1 2 3 4 More than 4
DE 4. How often has the AC procedure been revised systematically in the past years?
Every 7-10 years Every 4-6 years Every 2-3 years Yearly Never
Job Analysis and Job Requirements (JA)
JA 1. Was there a need-oriented personnel planning prior to implementing this AC? Yes No
JA 2. Was there a job analysis prior to specifying the job requirements/ dimensions and the exercises used in this AC? Yes No
JA 3. If yes, which data sources were used to analyze the job's specific requirements/dimensions? (*)
Job description Interview with job incumbents Questionnaire to job incumbents Interview with supervisor Questionnaire to supervisor
Critical incident technique Observation of job incumbents Workshops or teamwork Existing competency model New competency model
JA 4. How many job requirements/dimensions are observed and evaluated per exercise?
1 2-3 4-5 6-7 8-9 More than 9
JA 5. How many job requirements/dimensions are observed and evaluated per AC?
Fewer than 3 4-5 6-7 8-10 11-15 More than 15
JA 6. To what degree were the job requirements/dimensions systematically deduced from a job analysis?
Not at all To a small degree To an average degree To a high degree
JA 7. Which job requirements/dimensions are observed and evaluated in each AC? We present a standard set of job requirements/dimensions. Do your best to fit your dimensions into the following list. (*)
Communication Consideration/Awareness of others Drive
Influencing others Organizing and planning Problem solving
JA 8. To what degree are the job requirements/dimensions defined in a concrete and job related way?
Not at all To a small degree To an average degree To a high degree
Participants (PA)
PA 1. What is the average number of participants each time an AC is conducted?
2-4 5-7 8-10 11-13 More than 13
PA 2. What groups do the participants belong to? (*)
Non-managers 1st line managers 2nd line managers Executives External managers External entry-level
PA 3. What is the position to which the participants are going after this AC?
Trainee 1st line Manager 2nd line Manager 3rd line Manager Other:
PA 4. What is the function to which the participants are going after this AC?
Sales Marketing Customer employee Manufacturing employee Other:
PA 5. On what basis are the participants of this AC selected? (*)
Self-nomination Supervisor recommendation Personnel ratings
Observer (OB)
OB 1. What is the ratio of participants and observers?
1 : 1 1 : 2 1 : 3 4 or More
OB 2. What groups are represented by the observers? (*)
Line managers Internal HR experts External HR experts Works council
A participant's direct supervisor Company officer for minority affairs Internal Psychologists External Psychologists
OB 3. Which of the following criteria are considered in selecting the pool of observers? (*)
Race Ethnicity Age Sex
Organizational level Functional work area Education level Other
OB 4. Which aids do the observers have in evaluating behavior? (*)
None Qualitative aids; e.g., handwritten notes of the participant's behavior Quantitative aids, such as certain forms and systems of observation
OB 5. Which quantitative systems are used by the observers in evaluating behavior?(*)
Behaviorally anchored rating scales (BARS)
(Distinctly defined behavioral indicators of a given dimension are listed for levels of a Likert scale.)
Behavioral checklists
(A behavioral checklist provides a rater with a series of descriptive statements of job-related behaviors. The assessor decides in each case whether the participant has demonstrated the defined operationalization of the construct.)
Realistic behavioral descriptions
(With realistic behavioral descriptions, the behaviors are rated on a scale whose poles are defined according to verbal content, not in numerical terms.)
Graphic rating scales
Points on the Likert scale are defined with general adjectives.
Other
*Please do not use ' & or * in your response.
OB 6. Sometimes a rotation plan is used which has the function that each participant is seen by each observer. Is this type of rotation plan used in this AC? Yes No
OB 7. If yes, is each participant seen by more than one observer? Yes No
Observer Training and Qualification of the Assessor (OT)
OT 1. Is there an observer training? Yes No
OT 2. If yes, how long is the observer training? (in days)
Less than 0.5 1 2 3 4 More than 4
OT 3. Which concrete contents are conveyed in the observer training? (*)
Knowledge of the organization Knowledge of the target job Knowledge of the job requirements (definitions, demarcations)
Knowledge of the exercises Knowledge to observe each job requirement independently Knowledge to distinguish between the various job requirements
Knowledge and sensitizing for errors of judgment Relationship between the dimensions and job performance Method of behavioral observation
Use of behavioral systems including correct scale interpretation Ability to observe, record, and classify the participant's behavior in job requirements Instruction to focus on the various job requirements for which the exercise has been designed
Knowledge about the data integration process Knowledge and practice regarding diagnostic standards Ability to give accurate oral or written feedback
Consistency in role playing Forms of group influences in the data integration process Methods of forming judgments and ratings (statistical, non-statistical)
Professional behavior with the participants during the AC Limits of the AC method Other
OT 4. Which methods are used in the observer training? (*)
Lectures Discussion Video demonstration/Camera
Observe other assessor Observation of practice candidates Other
OT 5. Are the qualities of observational and rating skills of each observer evaluated after the observer training? Yes No
OT 6. Do the observers receive a certification after completing the observer training? Yes No
Exercises (EX)
EX 1. How many exercises are used in this AC procedure?
Fewer than 3 4-5 6-7 8-9 10-11 More than 11
EX 2. In your estimation, what is the percentage of exercises specifically developed for your organization (in contrast to " off the shelf " exercises) in relation to the total number of exercises?
Less than 25 % 26-50 % 51-75 % More than 75 %
EX 3. Which exercises or simulations are used? (*)
In-basket Role playing Presentation Background interview Situational interview
Case study Group discussion Sociometric devices Fact-finding Planning exercises
EX 4. If group discussions are used, are these conducted with a designated leader? Yes No
EX 5. If group discussions are used, is the content of discussion a current topic of interest to your organization (real life problem)? Yes No
EX 6. If group discussions are used, are these conducted with assigned roles? Yes No
EX 7. If one-on-one talks are simulated, who plays the role of the other person?
Another participant An observer A role player A professionally trained actor
EX 8. Are the exercises separate and distinct from each other, or related and integrated (" day in the life ")?
All separate and distinct Some exercises are related All integrated
EX 9. Is there a pretest of the exercises before they are implemented in the AC? Yes No
EX 10. What other procedures are used to diagnose participants in this AC? (*)
None Biographical questionnaire Intelligence tests (GMA) Personality tests
Skill/ability tests Knowledge tests Work sample tests Graphology
EX 11. Are computers used in your AC? Yes No
EX 12. If yes, how are computers used? (*)
Presentation of exercises Behavioral checklists Classification of behavior Scoring of dimension Writing reports Other:
EX 13. Can some exercises be completed on the web? Yes No
EX 14. Do the participants rate each other in the AC (Peer-rating)? Yes No
EX 15. Does each participant rate his or her own performance in the AC (Self-Rating)? Yes No
Participant Information (PI)
PI 1. What information do participants receive prior to the AC procedure? (*)
No information Tips for preparing Objective of the AC Kinds of exercises
How individuals are selected for participation The staff and role of observers How the results of the AC will be used How feedback will be given
The storage and use of the data What recommendations will be made by assessors The possibility of reassessment Other:
PI 2. Are the job requirements/dimensions to be assessed in the individual exercises explicitly communicated to the participants before the exercises start? Yes No
PI 3. Are the expected behaviors for each dimension explicitly communicated to the participants before the exercises start? Yes No
Data Integration (DI)
DI 1. What is the process of data integration?
Assessor discussion to consensus Statistical aggregation Combination of discussion and statistical Voting Other:
DI 2. Do the observers complete a report for each participant before the integration process begins? Yes No
Feedback (FE)
FE 1. In what form do the participants receive feedback regarding their results in the AC?
None Oral Written Oral and written
FE 2. When do participants receive feedback regarding their results in the AC?
Directly upon completion Up to one week after the AC More than one week after the AC
FE 3. Who gives the feedback? (*)
Observer Direct superior Employee of personnel department External expert Other:
FE 4. What information is included in the feedback?*
Ratings on each dimension in each exercise Behaviors observed on each dimension in each exercise
Overall dimension ratings across the exercises A final overall assessment rating across all dimensions
FE 5. Who is notified of the AC results of the participants? (*)
Participant Head of department Direct superior Personnel file Other:
FE 6. Do the participants have the possibility of reassessment? Yes No
FE 7. If yes, how long is the time period until participants with inadequate results have the opportunity to repeat the AC? (in years)
Less than 1 1-2 3-4 More than 4
Evaluation of the AC (EV)
EV 1. Was there a systematic evaluation of this AC? Yes No
EV 2. If yes, do you have written documents describing the evaluation of your AC (e.g., research reports, publications)? If you have this, please send us a copy. Yes No
EV 3. If yes, is a third party individual (someone other than the developer or the AC administrator) involved in this evaluation? Yes No
EV 4. Several evaluation criteria are listed below. Indicate if statistical calculations have been used to evaluate each criterion, and, if available, please write the numerical results in the fourth column. In the last column, indicate if expert judgments have been used to evaluate each criterion. (*)
Statistical Calculations Correlation Coefficient Expert Judgments
Objectivity (Inter-Rater-Agreement)
(This is shown if several observers come to the same result.)
Yes
No
Yes
No
Reliability
(This expresses the extent to which the AC measures in a consistent manner.)
Yes
No
Yes
No
Criterion validity: Predictive validity
(This expresses the extent to which the AC's results are empirically related to the participant's later performance on the job.)
Yes
No
Yes
No
Criterion validity: Concurrent validity
(This expresses the extent to which the AC's results are empirically related to performance which is measured at the same time.)
Yes
No
Yes
No
Content validity
(This expresses the extent to which the sample of exercises used in the AC procedure is representative of the job-holder's later activity)
XXXXXX XXXXXX Yes
No
Construct validity
(This tells whether the exercises actually measure the intended job requirements. The evidence that shows the AC results are related to other measures of the same dimension in the AC.)
Yes
No
Yes
No
EV 5. Is there an evaluation of participants' reactions? Yes No
EV 6. If yes, which criteria are the participants asked to evaluate? (*)
Face validity Fairness Usefulness of the AC Transparency of the general AC procedure
Transparency of the appraisal process Communication of results Competency of the observers Controllability of the situation
Initial expectations Feelings during the AC procedure Stress perception during the AC Consequences for the self-concept
Predictability of the results Consequences for motivation Accuracy of the feedback Acceptance of the Feedback
Acceptance of the AC Acceptance of the company Commitment to the organization Other
EV 7. In your estimation, what is the degree of satisfaction of the AC procedure by different groups?
Very
Dissatisfied
Dissatisfied Mixed Satisfied Vey
Satisfied
Executive boards
Line managers
Personnel department
Work council/Unions
Participants
II. Questions Concerning the General Cultural Values in Your Country
Next are some questions pertaining to the national cultural values in your country. Please think about the beliefs held by most of the people in your country. Please DO NOT think about what you personally believe or what you think other people should believe. Indicate how accurately the following statements describe what most people in your country think.
It matches
not at all
It matches
to a small
extent
It matches
to a large
extent
It matches
completely
1 2 3 4 5 6 7
1.

In our country most people believe that the interests of individuals prevail over those of the group.
2.

In our country being accepted as a good group member is more important than trying to stand out as an individual.
3.

In our country obligations to one's family and one's relatives are very strong.
4.

In our country individually-oriented rewards are more important than rewards the group earns.
5.

In our country resources and rewards are shared by groups.
6.

In our country most people generally accept the situation that some people have more power than other people.
7.

In our country most people believe that some members of society should exercise authority over others.
8.

In our country it is accepted that leaders give instructions to followers because they have the higher rank to do it.
9.

In our country it is not accepted that more powerful members deserve privileges and status.
10.

In our country most people believe there should be a small distance between higher and lower status individuals.
11.

In our country most people prefer leading highly structured lives with few unexpected events.
12.

In our country most people are generally very tolerant of uncertainty in life.
13.

In our country most people believe " What is different, is dangerous ".
14.

In our country most people do not like the unpredictability of future events.
15.

In our country most people experiment a lot with new ways of doing things.
16.

In our country most people think they have the ability to determine their outcome in life and how their life will turn out.
17.

In our country most people believe that one's future is not strongly controlled by the individual.
18.

In our country most people have the attitude that individuals can influence what happens to them.
19.

In our country most people think fate plays a major role in events.
20.

In our country most people think they can control many of the forces that affect their lives.