I/O Psych Chapter 3: Selection Assessment

- Constructs: psychological characteristic that a predictor is intended to measure: examples are intelligence, personality, and leadership - Tests, Interviews, SJTs, and Assessment Centers are all just methods - Use methods to tap: Cognitive ability Personality Specific skills Background, experience, and knowledge

- \"An objective and standardized procedure for measuring a psychological construct using a sample of behavior.\" - This definition encompasses a number of types of tests E.g. intelligence, personality, interests, assessment centers, work samples, etc - The word \"objective\" in the definition refers to scoring and not how the test was made

- Speeded Tests: Give test-takers a time limit when taking a test with easier items i) Many test-takers will not finish ii) If the time limit is not long enough, the test may be measuring others abilities than the one it is designed to measure Example: Given really easy math problems (2-2) and seeing how many you can do in a short time- Power Tests: Although a time limit may still be used, the time is usually sufficient to complete the test i) Some test-takers will still not finish ii) Designed to assess ability more clearly Example: - Questions difficult but given adequate time to complete -Less people do not finish

- Paper-and-Pencil tests are more commonly used - Electronic Page Turners (EPTs) are paper-and-pencil tests delivered electronically i) EPTs can provide more efficient and accurate scoring than traditional paper and pencil tests - Paper-and-pencil tests almost equivalent to computerized tests (Mead & Drasgow, 1993) i) Speeded test are less equivalent

Benefits: 1. Tests can be scored quickly and accurately 2. Novel stimuli can be presented a. Test-takers can interact with the test (e.g. video based questions, dynamic pictures and graphs, and auditory stimuli b. Reaction time can be assessed 3. Computer-based tests are more dynamic a. Content is easily changed relative to paper-and-pencil tests 4. Can use a wide variety of administration methods (proctored, unproctored, Internet-based, computerized adaptive)Disadvantages: 1. Computerized tests can have a high initial cost but may be cheaper in the long run 2. Computing experience may affect performance

- Heavily debated in I/O psychology - Proctored is more commonly used for employee selection - But unproctored testing has a number of advantages - Unproctored testing can be used to reduce the applicant pool before moving to more expensive proctored tests or interviews - Not a lot of work has been done on cheating on unproctored tests - Although personality is generally thought to be unaffected by the \"proctoredness\" of the test, cognitive or other abilities are considered highly affected by this administration method - 76% of college students have reportedly asked friends for answers to an exam (Stern & Havlick, 1986)

- After answering an item, the next item given to the respondent matches the current estimate of their ability level - As a result, different examinees will see a number of different items - Process continues until a certain number of items are administered (GRE) or until a desired level of accuracy is reached (Nurse Licensure Exam)

• Advantages of CATs: 1. Reduces the opportunity for cheating because different examinees are administered different items 2. Ability estimates are more accurate • Disadvantages: 1. A large pool of items is necessary 2. Data can be difficult to analyze and requires advanced psychometric knowledge

- Structured i) Source of the questions is a job analysis ii) All applicants asked the same questions iii) Standardized scoring key - Unstructured i) Interviewers ask anything they want - Structured interviews are more reliable and valid

- Skill-level determiner • Taps an interviewee's level of expertise • \"Several months after installing a computer network, the client calls and says that nothing will print on the printer. What could be going wrong?\"- Past-focused/behavioral • Interviewee is asked to provide an example of their past behavior • \"When you are dealing with customers, it is inevitable that you are going to get someone angry. Tell us about a time when a customer was angry with you. What did you do to fix the situation?\"- Future-focused/situational • Ask an applicant what he/she would do • \"Imagine that you told a client that you would be there at 10:00am. It is now 10:30 and there is no way you will be finished with your meeting until 11:30. How would you handle this situation?\" - Organizational fit • Assesses how well the applicant would fit in • \"What type of work place is best for you?\" - Clarifier • Allows the interviewer to clarify info in the resume, cover letter, etc. • \"I noticed a 3 year gap between 2 of your jobs. Could you tell me about that?\" - Disqualifiers • If the interviewee gives a particular answer, they are disqualified from the job • \"Can you work overtime without notice?\"

- Higher Validity i) even when over the phone ii) Most structured: .56 validity iii( Least structure: .20 - More favorable in court - Much less adverse impact than cognitive ability - Higher Reliability (average around .8) - Behavioral (past-oriented) may be more predictive than situational (future-oriented) questions - All provide incremental validity over g (Schmidt & Hunter, 1998) - BUT, it might be hard to write good behavioral items for inexperienced job applicants

- Cognitive ability - Personality - Social Skills (Rundquist, 1947) - Interests and Preferences - Applicant Fit- Structured Interviews may measure constructs more related to job performance (Huffcutt et al. (2001)

- The questions are not job-related - Many questions interviewers ask are illegal • Are you married? • Do you have health problems? - Interviewers make their decision in the 1st 3 minutes (Dessler, 2002; Otting, 2004) • No relationship between interview length and prediction accuracy - Contrast effects (can be reduced by training) - Negative-information bias - Interviewer-interviewee similarity - Interviewee appearance matters - Nonverbal communication i) Eye contact accounts for 80% of the variance in interview scores

- Right/wrong approach - Typical answer approach i) SMEs rate typical answers to the question - Key issues approach i) SMEs create a list of key issues that should be included in the perfect answer

- SJTs measure more contextually embedded reasoning than cognitive ability tests i) Tacit/Practical Knowledge ii) Social Competence or Emotional Intelligence iii) Conflict Management Skills iv) Leadership - Can be administered via pencil-and-paper, but really interesting when using video/computer i) Pencil-and-paper also tends to be confounded with g and with verbal ability- Present participant with a description of a relatively complex situation i) Example: two subordinates in conflict ii) Example: decide between multiple options proposed by a team - Participant makes a decision from a set of options (see, still multiple choice!)

- Empirical Keying i) Give to SMEs so they can generate options ii) Which option correlates best with outcome? - Rational or Theoretical Keying - Expert ratings i) Give to SMEs so they can generate options ii) What would the experts do? - Hybrids i) e.g., combine empirical and theoretical keys

Here is a situation: You work with other three classmates are working on a class project. One team member doesn't seem to care about the project. He is always late for group meetings, and always tried to find excuses for not working on the portion assigned to him. It seems like he is trying to do as little for the project as possible.As the deadline is approaching, you and the other two team members are worried about the project. What would you do?A. Go to the professor and report the problematic group member. B. Get together with the other two people and divide the work to make sure it all gets done. C. Get together with the other two people, do only the work you were originally supposed to do, and include a note when you turn in the project that explains why it is incomplete. D. Confront the problem group member and ask him/her to start contributing to the project.

- Can be confounded with g - McDaniel, et al (2001) identified 102 validity coefficients, and classified the SJTs as High-, Medium-, or Low-g • High-g: .41 • Medium-g: .18 • Low-g: .34 • Overall: .34

Characteristics: 1. Based on job analysis 2. Behaviors classified into dimensions 3. Linking of dimensions and behaviors and assessment techniques 4. Multiple Assessments 5. Simulations 6. Multiple Assessors 7. Assessor Training 8. Systematic recording of behavior 9. Data Integration

• Oral communication • Planning and organizing • Delegation • Control • Decisiveness • Initiative • Tolerance for Stress • Adaptability • Tenacity

• In-Basket • Leaderless Group Discussion • Role Play • Simulations/Business Game • Mock Presentations • Interviews, paper and pencil tests, and related measures

- Job Performance: .37 - AT&T's Management Progress Study: Of those predicted to reach middle mgt. 48% had by year 8 and 89% had by year 16 - Validity is highest when trained psychologists and/or managers serve as raters - .67 correlation with cognitive ability; .50 with openness; .50 with extraversion - Schmidt & Hunter (1998) found that ACs showed only small incremental validity above cognitive ability

collection of procedures for evaluation that is administered to groups of individuals; assessments are typically performed by multiple assessors