question

Evaluation

answer

-An Ongoing process -Provides a baseline of an individual's skills, performance, abilities on which progress is measured

question

OT role:

answer

Work with individuals, groups & organizations to enable them to participate in the occupations of everyday life

question

OT Goal:

answer

-Improve occupational performance -Must learn about the client's physical, cognitive, neurobehavioral, psychological skills as well as the activities, tasks, roles that the client defines as important

question

Therefore OT needs to be able to:

answer

-Select, administer, interpret standardized & non-standardized tests & assessments -Collect additional data through skilled observation, client & family interview -Recommendations -Must understand principles of tests & measurements

question

Why professionals need measurement knowledge:

answer

Evidenced-based practice: Helps client to make an informed decision about services

question

Evidenced-based practice:

answer

-Documentation/measurement of client's status, competencies, and issues enables OT to plan & document effectiveness of intervention -Sound measurement strategies can allow OT & client & family to work together to select compatible & effective interventions.

question

Evaluation:

answer

Process of obtaining & interpreting data for understanding the individual, the system or situation & to make plans for intervention Includes documentation of process, results & recommendations

question

Assessment:

answer

Specific tools, instruments or interactions used during the evaluation process. Component of evaluation

question

Evaluation Vs. Assessment

answer

According to IDEA: the definitions are reversed Bottom line: terms are often used interchangeably. Accurate documentation of data is what is important!

question

Screening:

answer

Process of reviewing available data (Use of general observation, use of screening tools) to determine strengths & limitations of an individual or group in order to decide whether a thorough evaluation is appropriate.

question

Initial evaluation:

answer

first encounter with client. Baseline data

question

Re-evaluation:

answer

on-going process of evaluation to track progress

question

Re-assessment:

answer

re-testing of specific tools

question

Purpose of Evaluation

answer

To help OT obtain information about the client that is: Objective Factual Verifiable Applicable for practical use in planning intervention To help OT's ability to: Make an OT diagnosis (identify issues relevant to client & OT practice) Select or classify individuals into categories (WNL, delayed, dependent, etc.) Perform research Determine intervention planning Assess the effectiveness of intervention (evidenced-based practice) Establish rapport with client & family

question

Completing a thorough evaluation involves:

answer

-Comprehensive knowledge of human development & individual variations, occupational roles & performance -Ability to interact with others & elicit information -Ability to judge performance and quality of skill performance -Understand the principles of testing & measurements -Ability to administer a variety of assessment tools; understand the strengths & limitations of those tools; be able to select appropriate tool for specific situation -Ability to put the data into the context of the client's life, roles, environment

question

When planning measurement strategies (assessment tools), OT practitioners need to think about the following questions:

answer

-Will the measurement process generate consistent information? -Would everyone agree about what you are measuring? -What are the most appropriate measurement parameters?

question

Differing Roles of COTA & OTR in evaluation

answer

-COTA: Can assist in data collection & evaluation under supervision of OTR -Experienced COTA may do standardized testing after he/she has demonstrated competencies under supervision of OTR -OTR: Selects assessment tools, administers & interprets results, formulates intervention plan

question

Effective intervention planning starts with Client-centered measurement:

answer

-Concepts of client-centered practice [Law (1998) Client-centered occupational therapy, Slack] -Respect for clients & families and the choices that they make -Clients & families have ultimate responsibility for decisions about daily occupations & OT services -Emphasis on person-centered communication. Provide information, physical comfort, emotional support -Facilitation of client participation in all aspects of OT service -Flexible individualized OT service delivery -Enable clients to solve occupational performance issues -Focus on person-environment-occupation relationship

question

Use of the International Classification of Function (ICF) of the World Health Organization (WHO) and the OT Practice Framework in the process of evaluation:

answer

Adapted from: Chapter 1 in Measuring Occupational Performance by Law, Baum, Dunn (Slack, 2001)

question

ICF

answer

Body function & Body structure: Activity: Participation in Community/Society: Environmental Factors:

question

Body function & Body structure:

answer

physiological & psychological function

question

Activity:

answer

Performance of activities, roles, & tasks that person views as important

question

Participation in Community/Society:

answer

how that person can be involved in situations/activities within his/her community/society

question

Environmental Factors:

answer

features, aspects, attributes of objects, structures, human-made organizations, service provision, & agencies within the physical, social. Attitudinal environment in which a person lives & conducts his/her life

question

Gather a database:

answer

Interviews (client, family, relevant individuals) Observation Review of records Informal testing/assessment (non-standardized tools) Formal testing/assessment (standardized tools)

question

Methods of data collection need to be appropriate for:

answer

Age Gender Education Cultural background Socioeconomic, medical & functional status

question

Interpret findings

answer

-Comprehensive picture of client's interests, needs, strengths/abilities, and limitations/barriers to function -Client's perception of his/her abilities

question

Use findings to establish recommendations

answer

-Communicate findings in a clear, concise manner that is understandable by client & family & other professionals (no jargon!) -Refer to other professionals as necessary Use client-centered approach to develop goals

question

Considerations to keep in mind during the measurement process (use of assessment tools)

answer

-Will measurement process generate consistent information? -Is what you observe or record likely to be the same under various circumstances? -Would everyone on the team agree about what you are measuring? -Validity -Did you conduct your assessment in context of daily activity? -Example: Measuring ROM and MMT, but can you conclude that client can feed or dress himself?

question

What are the most appropriate measurement parameters?

answer

What is your criterion for measurement? What are the skills or target behaviors that you are looking for?

question

What is your frame of reference?

answer

-Need a clear relationship between the assessments you use & your intervention approach -Sensory & behavioral issues? Sensory integration -Body structure issues? Biomechanical & Rehabilitation approach -Psych issues? MOHO

question

Methods of Data Collection, cont.

answer

-All assessment tools have biases; know strengths & limitations of assessment tools (gender, socioeconomic, cultural, geographic, medical status): -Choose appropriate assessment tool to fit your client -Use several tools to cross-reference data -Know your assessment tools! -Misuse of assessment tools & results is a form of malpractice -Select appropriate tools that are relevant for your client's needs & interests -Be aware of possible examiner bias (conscious or unconscious): -Be aware of person prejudices so they don't interfere with your professional judgment -Moral issues to consider; your results could determine: -What freedoms your client will or will not have -What roles your client may or may not be able to do -What activities your client may or may not be able to do -What benefits or resources the client may or may not be able to receive -Maintain confidentiality of results: -Understand HIPPA rules

question

Accurate OT evaluations are critical to your practice as an OT

answer

Critical for proper intervention for clients Critical for you: establishes you as a competent, worthwhile member of the health care team

question

General Tips for Interviewing

answer

1.Prepare yourself, client, environment: Prepare yourself: Read preliminary information Formulate in your mind where to start asking your questions, based on referral information Revise your questions as client starts to identify areas of concern Ice breakers: examples (introduce yourself) Prepare Client: Schedule first appointment at a time that is best for client (if possible) Clarify your role & what can OT offer Explain the purpose of the interview Explain how the information will be used Prepare Environment: Private space, comfortable temperature in room, have tissues, water available Position yourself in chairs, facing each other at an angle, 3 - 4 feet apart If child: have toys available, arrange room to be enticing & non-threatening; try not get between child and care-giver; allow child to be in close proximity to care-giver (have a barrier between you & child if necessary), sit lower than child if you can

question

Interview Process

answer

2. Questioning & Responding: Questioning Our objective is to find out what matters in the client's life Open-ended questions are more likely to yield more information than a closed, yes-no, question; your goal is to find out what the person needs & wants to do and we need to find out about the contexts they need to do these goals in example: "Please tell me about what problems you are having with your hand" instead of: " Do you have any problems with weakness in your hand?" Narrative questions: information about events & perceptions about events "Tell me about..." Factual questions: descriptive information -- "What do you do for..." factual questions (more structured) work better for clients with disorganized thought process (due to cognitive issues or feeling overwhelmed)

question

Interview Process, cont.

answer

Be cognizant of questions that are making client feel uncomfortable Avoid putting the client on the defensive (avoid questions that begin with "Why..") Ask one question at a time; be non-judgmental (avoid an opinion or advice or judgment ) It's okay to wait a few seconds and compose your thoughts between questions than to keep talking

question

Avoid leading questions:

answer

-You cut your finger on a ketchup bottle. Are you having any numbness or tingling? -Your doctor said you broke your arm on a jet ski. Are you having pain in it right now? -I was told that Johnny is unable to write. He is having trouble with numbers as well as letters, right?

question

Types of responses

answer

-(helps you convey that you are listening and that you are concerned, helps you organize your thoughts): -Content responses: when you want to clarify facts; paraphrase what you heard ( "So, you're saying that ... or "This is what I believe you mean.." or "Have I understood you correctly?") -Affective responses: helps you understand the feeling or tone that the client is trying to communicate: " It seems like you're feeling...."

question

Active Listening

answer

allows you to respond appropriately: Try not to be distracted (think about where you need to be next, or prematurely think about solutions, make snap judgments): paraphrase example: "My left wrist hurts whenever I swing a bat & miss the ball" you could say...: example: A Teacher says, "Jason is driving me crazy. He never seems to be able to stay in his seat, he wanders around the classroom disturbing the other students. I am at my wit's end... you know, I have responsibilities to the other children in the classroom... I'm not sure he belongs in a regular classroom." you could say...:

question

Avoid potential traps

answer

- giving advice that could contradict the doctor or another professional -example: A client asks: "I think the doctor should go ahead and do a tendon transplant, what do you think?" -When asked what you think about their condition, ask what they have been previously told & what do they think? Better to answer the question with a question

question

find out about context

answer

(environment, supports, are there any barriers to performance): person may have the skill but context may be barrier to performance example: distractions (TV, study hall), social pressures & expectations

question

Observe behavior during the Interview:

answer

-Affect: emotional level, energy level, non-verbal behaviors, eye contact, tone of voice -Cognitive skills: comprehension, memory, concentration, thought organization (oriented to time & place), perception of self -Social skills: appearance, interpersonal behavior -Client's posture -Is person defensive or resistant during interview? Are you doing something during the interview?

question

4. Structuring the Interview:

answer

-Opening: Introduce yourself, describe your role Describe purpose of interview and what types of questions that you will be asking and how you plan to use information -Body of Interview: Questions to find out the client's story Use pre-structured interview assessment tools if available & appropriate for client Follow interview format set up by place of work (agency) Closure: Allow time to summarize information Identify important themes Address how you both will work together on client's goals Set up next appointment and discuss what will happen at the next appointment or go on to evaluation of performance components if time allows Thank the client

question

5. Let's practice with some case studies:

answer

Observational Information: "JP" is a large man, 6 feet and weighs 220 pounds. He is 56 years old. He comes to the appointment in dirty jeans and a t-shirt. His hair is dirty, but his hands are clean. He seems pleasant, but is a little nervous. He holds his right hand as if he is protecting his thumb from injury. He grimaces when he accidentally bumps his hand when he sits down. Referral Information: John caught his right hand in an electric roller (safety guard had been pulled off) while on the job as a city bus maintenance worker (accident covered by working man's compensation). Extensive lacerations severed the digital nerves and flexor pollicus longus on the right thumb. He had surgical repair to right thumb involving nerves, tendons. Hand has been casted for 6 weeks to allow for tissue repair. Has limited ROM of thumb MP & IP, no subluxation or dislocation. Has not returned to work. He was referred to OT from his hand surgeon following removal of cast and to be fitted for a hand splint.

question

Interview John and find out:

answer

Personal goals: (Consider: values, beliefs, spirituality) Functional limitations in: ADLs IADLs Rest & Sleep Work Leisure Social participation Work history: Social history: Leisure interests: Daily Schedule (Performance Patterns: habits, routines, rituals, roles): Context: (physical, cultural, social, personal, temporal, virtual)

question

Standardized Assessment Tools/Main Purposes of Evaluation

answer

Obtain descriptive information Obtain Diagnostic or Predictive Information Prepare for Intervention Planning Outcomes measurement

question

Assessment Tools: Basic information

answer

-Assessment categories: Screening Specific in-depth assessment -Types of Assessment Data: Quantitative Qualitative -Assessment formats -Measurements: define behavior

question

Critique Usefulness of Assessment Tools

answer

-Is tool defined? -Does tool measure what is says it measures? -Are scores reliable & valid? -How does tool fit into constraints of practice? Time constraints Facility procedures Therapist experience Performance vs Capacity

question

Standardized tests: Definition

answer

-An evaluation tool that has gone through rigorous development to establish norms for performance, given a comparative sample -Uniform and consistent administering procedures provided -Comprehensive, clear instructions for administration & scoring provided -Includes an exact list of materials to be used -Includes exact oral instructions, demonstration procedures, time limit, preferred testing conditions -Psychometric characteristics (reliability & validity) available -Norms: expected responses

question

Standardized Tools = Standardized Procedures

answer

-Clearly delineated, step-by-step instructions for administering & interpreting tool -Standardized procedures remove variance in data -Uniformity: scores obtained from different evaluators on the same client should come out the same or comparable -Manual should clearly describe testing conditions and modifications (if allowable)

question

Evaluator Can Influence Results: Avoid These Pitfalls

answer

-Background bias: -Rating too strictly or leniently -Tendency to rate everyone average & avoid extreme scores -Observer expectations: evaluators personal interests resulted in test results -Experience with tool administration -Client may change behavior just by having evaluator present

question

Client Can Influence Test Results

answer

-Comfort level: Anxiety & stress; health -Fatigue -Mood & affective behavior -Client beliefs regarding assessment:

question

Terms Used in Test Psychometrics

answer

-Raw scores converted into scores and interpreted based on a statistical distribution -Normal distribution: Bell shaped curve -Mean, Median, Mode -Standard Deviation: -Correlation: degree of relationship & association between 2 variables -Correlation coefficient: numerical range of relationship between +1 & -1 -Positive = high correlation; negative = low correlation

question

"True Score" and Testing Errors

answer

-Error of measurement Item bias Rater Bias Client errors/issues Environment/Context issues -Standard Error of measurement (SEM)

question

Psychometrics: Reliability

answer

-Definition: accuracy & stability of the measure; consistency of scores obtained by the same person on difference occasions with equivalent sets if items and testing conditions -Test-retest reliability: degree of variance of scores when same test is given to same individual with significant time that individual cannot remember their response to the items or have learned. .90 = high; .80 = moderate; .70 = low; .60 = unacceptable -Inter rater reliability: degree of agreement between two evaluators for the same individual & conditions. .85 or more is good -Intrarater reliability: consistency of measurements by same evaluator when 2 similar testing situations are correlated -Alternate form reliability: if using equivalent forms of the same test (different questions to test the same performance areas), how equivalent are the scores (>.80 preferred) -Internal consistency: do several items measure the same construct area (>.80); high internal consistency means that test items measure a homogeneous construct

question

Psychometrics: Validity

answer

-Definition: What is the meaning of the test? What construct, behavior, or trait does the test scores measure? Does test measure what it says it measures? -Content validity: do items represent the construct that is being tested? -Face validity: from appearance only, without statistical measures, do items appear to address the purpose of the test? -Concurrent or congruent validity (or criterion-related validity): extent of which 2 tests agree that measure the same content area; -Construct validity: does data from the test represent the theory it was designed to measure? -Factor analysis: correlations among items of a test to be able to identify patterns in the correlations

question

Response Methods for Test Items

answer

-Raw Score -Rating Scales Likert Scale Guttman Scale- Pain 1 to 10 Checklist Scale Semantic Differential- verbal type of likert scale Q-Sort

question

Norms & Understanding Test Scores

answer

-Norms: performance of normative sample or fixed reference group (group of individuals) --Measures of the average performance of the reference group --Raw score is only relevant when compared to other similar individuals --Specific demographics of normative sample need to be reported in the manual (ages, region, rural, urban, gender, ethnicity or other relevant characteristics) -Target population (the group that tool was developed to measure) needs to be stated in the manual --Does your client match the target population? --Normative sample needs to be relevant to the target population -Norms should be recent; data can become dated

question

Conversion of Raw Scores into Standardized Formats

answer

Permits comparison of results with other professions who are testing the same individual

question

Types of Standardized Scores

answer

-Percentile equivalent or rank: -Standard deviation scores: Z-score: T-score: Stanine: -Developmental Age or grade-equivalent scores: scores compare individual's performance to that of a typical individual of the same age or grade -Profile: individual scores across several sub tests

question

Item Analysis

answer

Item Discrimination Rasch Analysis Cross-Validation

question

Testing Approaches & Interpreting Results

answer

Norm-Referenced Tools -Uses population parameters to describe a performance -Interpretation of test results in terms of norms related to norm group -Norm = average performance -Use of a standardization normative sample -Used to establish degree of ability or disability, function or dysfunction of client in comparison with normative sample -Advantages: Provides a target range of typical or expected performance abilities Can measure variability or differences among clients Can be used to measure an client's performance over time to assess change in function if tool is designed for this -Disadvantages: Can only use if client is similar to normative sample

question

Criterion-referenced Tools:

answer

-Determine whether client can accomplish objectively defined criterion, defined standard of performance or skill mastery -Use content & functional levels to measure performance -Find out what clients can do; not how they compare with others -Skills are clearly delineated in behavioral terms, in sequencing performance steps -Mastery is used to determine if client has reached a certain skill level Can demonstrate an all or none score: Can demonstrate mastery or not Usually a cut-off score to indicate sufficient mastery -Advantages: Test of essential, fundamental skills in a highly structured situation Determine degree of competence in a given area Identify specific components of skill that need to be developed Used to determine skill improvement following intervention -Disadvantages: looks at skills, but not individual differences. Does not assess quality of skill performance

question

Ecological Assessment:

answer

-context-specific evaluation to evaluate the client within the context he/she needs to perform the skills -Uses observation, interviews, checklists and questionnaires to gather data -Advantages: Assess skills in context -Disadvantage: Has potential for evaluator bias due to subjectivity

question

Cultural Issues in Standardized Testing

answer

-With increasing diversity of client population is tool appropriate? Is Diversity represented in the norm sample? Check the manual -Language issues: --What if language of assessment tool is not the main language of the client? --Is there a translation available? --Can an interpreter be present & how might that affect results? -Behavior expectations vary between cultures -Perception of time & speed can vary between cultures -Test content labels may differ between cultures

question

Canadian Occupational Performance Measure(COPM)

answer

...

question

Occupational Performance

answer

COPM Manual, Pg 3: "The unique contribution of OT is a result of its focus on occupational performance." AOTA, 2002 - "Engagement in everyday activities that have meaning and value to individuals and culture." Asher, Pg 31 - "Performance of everyday activities supports participation in the various contexts of a person's life." How would you explain occupational performance to a client that you are evaluating? How would you define it as a construct?

question

Authors

answer

Canadian OTs: Mary Law Sue Baptiste Anne Carswell Mary Ann McColl Helene Polatajko Nancy Pollock

question

Purpose or Objective of Tool

answer

Use of a semi-structured, standardized interview format as an outcome measure to measure changes in self-perception of occupational performance (performance areas) over time among clients with a variety of disabilities and across all developmental stages COPM Manual, Pg 1

question

Standardized:

answer

-Collection of data is objective and quantifiable -Facilitates communication -Scientific generalization explaining human behavior

question

Outcome Measure:

answer

Results of intervention Effectiveness of the intervention for the individual based on the client's self-perception.

question

Purpose or Objective of Tool

answer

Identify problem areas in occupational performance Provide a rating of the client's priorities in occupational performance Evaluate performance and satisfaction relative to those problem areas Measure changes in a client's perception of his/her occupational performance over the course of occupational therapy intervention (COPM Manual, Pg. 1)

question

Test Development

answer

Originally published in 1990 2nd edition in 1994 3rd edition in 1998 4th edition in 2005 The format of the 4th edition is unchanged from the 3rd edition. 4th edition includes new information about research studies using the COPM

question

Test Development Cont

answer

Developed as a response to a task force in 1980 between Canadian Department of National Health & Welfare and the Canadian OT Association to develop quality assurance guidelines for practice of OT. Outcome measure for occupational performance Focus on client's own environment Focus on what is important for the client Measures client's perception of performance areas and context.

question

Outcome measure for occupational performance Considerations for Focus on client's own environment Focus on what is important for the client Measures client's perception of performance areas and context.

answer

Client's developmental stage Client's life role Client's motivation

question

Canadian Model of Occupational Performance (CMOP)

answer

The interaction of the person, environment, and occupations = Occupational Performance

question

Population

answer

"Occupational performance is a feature of humans regardless of age, gender, or disability." Not diagnosis specific Any age May not be appropriate for clients with cognitive impairments Family or caregivers can be used as respondents

question

Materials and Supplies

answer

Test manual Recording form

question

Cost

answer

DVD Kit: $225.45 Manual: $48.83 Manual and form kit (100 forms): $52.45 100 forms: $18.38

question

Time Needed to Administer

answer

~30-40 minutes

question

Administration Conditions and Procedures

answer

Interview format with quiet space for interview Interview client to identify those occupational performance areas that the client wants, needs, or is expected to do Identify occupational performance areas that the client feels are unsatisfactory Individualized for each client

question

Administration Conditions and Procedures Cover Page

answer

Demographic Information Allows for dialogue regarding disability and impact on daily functioning Extra information can be recorded on back cover

question

Administration Conditions and Procedures

answer

Interview about occupational performance What daily activities does the client want to do, need to do, or are expected to do in a typical day.

question

Administration Conditions and Procedures 3 Areas of Occupation:

answer

Self-care Productivity Leisure

question

Administration Conditions and Procedures Facilitating the Interview

answer

Occupational Profile and History Typical Day Themes of Meaning

question

Scoring Criteria & Interpretation

answer

After client has identified all areas of occupational performance: Rate Importance 1 to 10 scale (1 is Not Important; 10 is Extremely Important) Scores do not have any inherent meaning in themselves, but are used as a therapy focus or guide Client identifies priority areas (top 5 or less)

question

Scoring Criteria & Interpretation Rating performance

answer

"How would you rate the way you do this activity now?" 1 is Not Able to Do It at All 10 is Able to Do It Extremely Well

question

Scoring Criteria & Interpretation Rating Satisfaction

answer

"How satisfied are you with the way you do this activity now?" 1 is Not Satisfied at All 10 is Extremely Satisfied

question

Scoring Criteria & Interpretation

answer

Add the PERFORMANCE scores Divide by the number of identified problems =Total Performance Score

question

At reassessment:

answer

Scores are compared A difference of 2 or greater between initial and reassessment scores is considered a significant and satisfactory change in skill perception

question

Evaluators:

answer

Occupational Therapists Training is required to ensure that the COPM is administered in a reliable and valid manner. Manual, Self-Instructional Programme, Video, Workshops

question

Psychometric Properties

answer

Standardized vs. Non-Standardized Standardized - There are specific methods for administering and scoring the test. Norm-referenced vs. Criterion-referenced Criterion-referenced Client rates the skill in terms of ability, importance, and satisfaction. Occupational performance areas are conceptualized and determined by the client.

question

Reliability

answer

How accurate and stable is the measure? Consistency of the scores obtained by the same person on different occasions with equivalent sets of items and testing conditions. Test-retest reliability: Degree of variance of scores when the same test is given to the same individual with significant time that the individual cannot remember their response to the items or have learned.

question

Test-Retest Reliability Research

answer

3 studies Stroke: 0.89-0.88 = Moderate Schizophrenia: 0.84- 0.85 = Moderate COPD: 0.90 - 0.92 = High All within an acceptable range.

question

Validity

answer

What construct does the test measure? Occupational performance Does the test measure what it says it measures? Yes

question

Content Validity

answer

Supported when experts agree that the full domain or content of the construct has been measured.

question

Criterion Validity

answer

Supported when scores on the measure of interest correlate positively with other well-accepted measures of the same construct.

question

Construct Validity

answer

Supported when the measure of interest correlates positively with measures of other concepts that are expected to be theoretically related.

question

Validity Studies

answer

8 studies Settings: Neurorehabilitation, Home Care, Mixed Rehabilitation Conditions: Upper limb disorders, Rheumatoid Arthritis, Schizophrenia, Hemophilia, Orthopedic, Stroke

question

Validity proof

answer

Content: Compared the COPM to a variety of other measurement tools that ranged from functional measures to psychological and social Functional Independence Measure (FIM) Life Satisfaction Scale (LSS) Criterion: Compared with a variety of other well-accepted measures of the same construct Health Assessment Questionnaire (HAQ) Klein Bell ADL Activity Subscale Wisconsin Quality of Life- Client Questionnaire (WQL) Construct: compared with 2 other measures that were conceptually similar to the COPM Satisfaction with Performance Scaled Questionnaire (SPSQ) Reintegration to Normal Living Scale (RNL)

question

Utility

answer

8 studies Responsiveness to change Ease of administration Time to complete Ability to communicate aspects of occupation Wide variety of clinical settings Different languages Different cultures

question

Utility Usefulness

answer

Useful as an interview guide Useful for goal-setting Useful for establishing goal priorities Useful as an outcome measure of client satisfaction

question

Strengths of the Tool

answer

Client-centered Helps to focus goal priorities Motivating and holistic Easy to learn and administer Excellent interview guide and helps therapist to develop rapport Helps to clarify OT role in intervention Helps to provide a framework for the initial evaluation Can be used with a variety of clients, conditions, and ages Can be used as a satisfaction measure

question

Limitations of the Tool

answer

Difficult to use with clients who have cognitive impairments -Can the individual understand the methods of rating? -Can the individual understand realistic goals? COPM does not measure the actual abilities, only the perception of those abilities If the client is in the early stages of recovery of a traumatic condition (TBI, SCI, Stroke), the client may not understand the full and long term consequences of the condition.

question

Considerations

answer

"How can I take the time to do a COPM with my clients? I have a really high caseload and just don't have the time." "Some of my clients have trouble with the rating scale. How can I help them?" "What if the client identifies problem areas to work on which I believe are inappropriate?" "What if a person's perception of what they do is 'distorted'?" "How do I use the total scores?" "Do you have to do the scoring?" "What am I supposed to do with all the other assessments that I normally do with my clients?" (COPM Manual, 20-21)

question

Uses of Evaluation Data

answer

1. Use of data for outcomes measurement 2. Use of data to support evaluation processes 3. Use of data collected from evaluations to support evidence-based practice 4. Use of data for program development and evaluation

question

Use of data for outcome measurement

answer

What are OT outcomes? -According to OT Practice Framework-II,"outcome is to support health and participation in life through engagement in occupation" -Framework lists 9 categories of outcomes (areas of occupation)

question

Implementing an outcomes-oriented approach

answer

-Evaluation process is individually focused as well as population-focused -Data are kept not only on individual clients but also to determine the needs of the population of which the client is part (i.e. older adults with hip replacements, 4th graders with handwriting difficulties, etc.) -Some good tools for gathering individual as well as population data are: FIM(Functional Independence Measure), COPM, PEDI(Pediatric Evaluation of Disability Inventory), VMI, SFA(School Function Assessment) -Use of standardized measures is critical in outcomes-oriented approach in order to be able to interpret and generalize data to the same clinical population

question

Use of data to support evaluation processes

answer

Selecting evaluation tools a. Identify the best tool based on hierarchical evidence b. Look for studies on tools' validity and reliability c. Good tool = good data

question

Use of data collected from evaluations to support evidenced-based practice:

answer

Think about how you document the outcomes of your OT intervention program a. Involves writing measurable goals b. Use of appropriate assessments to gather baseline data c. Description of program/intervention d. Documentation of progress e. Use of appropriate assessments to gather outcome data f. Good initial evaluation and re- evaluation data results in accurate and measurable account of treatment effectiveness (i.e. create your own evidence)

question

Use of data for program development and evaluation

answer

Designing a new program/new practice area A new program can only be implemented with data to support the new initiative Description of potential clients: Who are they? Needs assessment of potential clients and client populations b. Clearly define performance outcomes of your program: What observable skills/behaviors/abilities will the client present to indicate a successful outcome? c. Clearly define the performance measures that will be used to measure the outcomes of your program Appropriate selection of assessment tools Review validity & reliability of these assessment tools

question

Program evaluation of existing program

answer

Questions to ask about your program: a. Are interventions provided by our facility improving the status of our clients or prevention further limitations? b. Are our treatments effective and efficient (cost effective)? c. Are our clients better because they received these services? -To answer these questions: appropriate use of assessment tools that are valid & reliable -Outcomes that are measured: changes in performance in areas of occupation, client satisfaction, improved role competence, adaptation to condition, health & wellness, prevention, quality of life

question

Use of Case Studies and Case Reports to gather data

answer

-Within the confines of clinical practice, it is difficult to conduct large randomized controlled trials -Case studies and case reports offer the clinician the ability to gather and report data obtained in clinical practice -Note: Case studies (prospective, case study n=1 experimental), -Case study/report (retrospective, non-experimental)

question

Case Report

answer

-Non-experimental research, retrospective (often reports knowledge that was gained serendipitously) -No systematic collection of data, difficult to replicate -Identify interesting case -Describe full medical, developmental, and/or behavioral history -Describe presenting issues, symptoms, prior treatment, relevant social and demographic factors -Use of information from assessment data -Describe treatment/intervention plan. Cite information from literature review that supports rationale for intervention -Describe intervention process -Describe outcomes using information from assessment data -Discuss results (discuss at state & national meetings or study groups, write article, etc.) -Case reports are level V evidence

question

Single-Subject research (Case Study)

answer

-Quasi-experimental research, prospective -Defined methodology, repeated measurements, more reliable -Identify a question of interest -Identify baseline behavior, description of program, and proposed outcomes -Literature review -May need Institutional Review Board (IRB) approval -Collect baseline data using appropriate assessment methods Institute phases of your study & collect data -Analyze data -Repeat with multiple subjects if possible -Discuss results (discuss at state & national meetings or study groups, write article, etc.) -Case studies are level IV evidence

question

Who is interested in the data?

answer

-Clients, families, caregivers -Providers (therapists, other health care professionals) -Payers and regulators (insurance companies, state & federal regulatory groups that set standards and policies for reimbursement)

question

Use of EBP principles for:

answer

-Selection of best evaluation tools -Selection of best treatment approach -Documenting outcomes in a consistent manner, using reliable and valid assessments and documentation

question

Introduction to Evidence-Based Practice

answer

Historical Development of Evidence-Based Practice The term 'evidence-based medicine' was created at McMaster University Medical School in Canada in the 1980s to describe the process of problem-based clinical learning and teaching. • Over time, the term has evolved to be called 'evidence-based practice' as other disciplines have embraced the concept. • In the 1990s, EBP found its way into health care and social work practice. • EBP in OT has been discussed since the mid to late 1990s.

question

What is Evidence-Based Practice (EBP)?

answer

Canadian Association of Occupational Therapists, 1999, p. 267: "Client-centred enablement of occupation, based on client information and a critical review of relevant research, expert consensus and past experience."

question

EBP is the process of integrating

answer

Clinical expertise Best available clinical evidence Patient preferences and values

question

Why do we need EBP?

answer

-To improve our clinical expertise -To provide the best patient care -To keep up to date with changing evidence -To ensure reimbursement for services -To properly educate student clinicians -To be able to advise clients who conduct their own literature searches but who are often overwhelmed by the material

question

Not All Evidence Is Created Equal

answer

Types of Research Quantitative Qualitative Levels of Evidence Hierarchy of Evidence

question

What is Quantitative Research?

answer

-Provides statistical evidence to assess the effectiveness of a certain assessment or treatment approaches -Research approaches: Experiments Randomized controlled trials (RCTs) Surveys Case Reports

question

Examples of Quantitative Research Topics

answer

-Effectiveness of constraint-induced movement therapy on upper extremity function in clients with strokes -Reliability of the Test of Gross Motor Development in detecting motor delays in pre-schoolers

question

Assessing the Evidence

answer

Evidence is ranked in hierarchical order based on the type and rigor of the study with lowest level of evidence (level V) on the bottom and highest level of evidence (level I) on the top

question

Levels of evidence (Liebermann & Scheer, 2002).

answer

Level I - Systematic reviews, meta-analyses, randomized controlled trials Level II - Two groups, nonrandomized studies (i.e. cohort, case-control) Level III - One group, nonrandomized (i.e. before and after, pretest/posttest) Level IV - Descriptive studies that include analysis of outcomes (single-subject design, case series) Level V - Case reports and expert opinion that include narrative literature, reviews and consensus statements

question

Assessing the Evidence, cont.

answer

-It is important to note that low level evidence does NOT equal 'bad evidence' -Based on the topic of a study, retrospective case reports may be the best available and/or the only obtainable evidence

question

Current Issues in Occupational Therapy

answer

-What is "Good Evidence" in OT? -Quantitative research? Perceived to be too impersonal, not client-centered -Qualitative research? Perceived to be of less rigor

question

OT profession is struggling with implementation of EBP at various levels

answer

Profession Institutional Individual

question

Profession

answer

-Uncertainty regarding discipline specific definition of EBP -Uncertainty regarding consensus of what constitutes 'good evidence' within occupational therapy practice -Levels of Evidence hierarchy does not encompass all types of research, i.e. one size does not fit all -Profession is still transitioning from arts based to science based discipline

question

Institution

answer

-Lack of time and access to resources -Perceived lack of relevance to clinical setting

question

Individual

answer

--Lack of knowledge of EBP and its implementation -Lack of time -Perceived threat to client-centered practice, i.e. too much focus on quantitative results

question

What is Qualitative Research?

answer

-Explores the meanings and phenomena as they occur in the natural setting -Explores the 'shared experiences' -Explores the complexities of human behavior -Adds 'the human touch'

question

What is the role of qualitative data in outcomes-oriented approach?

answer

It can identify client needs and expectations for therapy It can provide data for subjective outcomes such as satisfaction and perceived quality of life Provides 'quality of life' information that is important for consideration in OT assessment and intervention =>informs the holistic approach to intervention

question

Qualitative Research Approaches

answer

-Phenomenology - focuses on the experience of individuals -Ethnography - examines a culture -Grounded Theory - explores a specific topic with the intent to generate theories -Research methods: interviews, focus groups, observations, written materials

question

Examples of Qualitative Research Topics

answer

-Occupational therapists' perceptions of evidence-based practice -Experiences of individuals living with spinal cord injury -Experiences of couples post-stroke -Impact of home-based therapy services on daily life: families' perspectives

question

How to evaluate qualitative evidence?

answer

No hierarchy until recently that ranks the type and rigor of the research Concepts like validity, reliability, generalizability are not applicable to qualitative research

question

The Role of Qualitative Research in Evidence-Based Practice, cont.

answer

-Research methodology is different and uses techniques such as participant observation, triangulation of data, auditable trail of analysis, peer checking, for example -Authenticity => reliability -Transferability => generalizability -Therapist needs to determine whether the article addresses his/her clinical question in a meaningful way (i.e. does the situation described closely relate to the client ) -Identify researcher bias (ideally, this should be addressed in the article's 'methods' section) -'Conclusion' section of article should present evidence from the study outcomes and existing literature to support the conclusions drawn -Level V evidence on the 'traditional' hierarchy

question

Visual Information Processing Frame of Reference

answer

-Vision is one of the primary senses in humans, the ability to see & interpret visual stimuli is important for task performance -Vision is a dominant sense: provides brain with 80% of what it knows

question

What is good vision?

answer

-The ability to see 20/20 -The ability to see clearly -Having healthy eyes and not needing glasses -Being able to pass vision screening test at school or doctor's office

question

Three component model of vision

answer

Visual integrity Visual efficiency Visual information processing

question

Visual integrity

answer

-Visual acuity (measuring resolving power of the eye, i.e. 20/20) -Refraction -Eye health

question

Visual efficiency

answer

Ability of the visual system to clearly, efficiently and comfortably allow a person to gather visual information from environment

question

Visual efficiency skills:

answer

Accomodation/focus Binocular vision Eye movements

question

Visual information processing skills/visual perceptual skills

answer

Visual spatial skills Visual analysis skills Visual motor integration skills

question

Visual Integrity

answer

Good Vision Visual Efficiency Visual Information Processing

question

Perception

answer

Developmental Theories of Piaget, combined with concepts from developmental psychology, education, & ophthalmology led to development of Visual Information Analysis Frame of Reference The process of receiving and analyzing sensory information -This process leads to a cognitive decision regarding behavior An intermediate step in information processing between sensation and cognition (Hammel, Pearson & Voress, 1993) -Sensation: awareness of sight, sound, touch -Perception: interpreting & organizing physical elements of stimulus -Cognition: reasoning, reading, responding

question

Perceptual Process involves 3 steps

answer

Detection Analysis Response to stimulus/stimuli

question

Detection of stimulus:

answer

-Preattentional state: ability to be ready to attend to stimulus. -Scanning ability: ability to attend to meaningful stimuli and not attend to non-meaningful stimuli -Alertness: ability to focus attention -Orienting: ability to turn toward stimulus -Regulation: ability to sort out stimulation to maintain interest & attention without going into sensory overload

question

Analysis:

answer

make a decision about the information (cognitive)

question

Attending:

answer

active process that creates encoding of information by gathering data (explore using various senses), differentiating between features of an object or event

question

Judgment:

answer

decide what to do: continue to attend or move on?

question

Memory:

answer

stimulus is identified as being novel or familiar

question

Response:

answer

-observable behavior or action as a result of the stimulus -Ignoring stimuli is a conscious choice

question

Types of responses:

answer

-Habituation response: individual has acclimated to stimuli, stimuli no longer novel; stop attending or responding to it -Sensitization response: aware of stimulus and individual behaviorally responds

question

Visual Perception Definition

answer

-The ability to interpret what one sees in order to determine behavior or a course of action (combination of interaction of visual & cognitive skills) -Visual stimuli are related to other sensory input from other senses -Vision rules and often over-rides the other senses

question

Visual Information Analysis:

answer

-Use of cognitive skills to extract & organize visual information from the environment & the ability to integrate information with other senses and previous experiences -Vision, integration of visual input with other senses, and cognitive abilities influence how a person can interact & act on what he or she sees. -Visual system has interconnections with all of the other sensory systems -Purpose of vision is to guide & direct movement & learning Interconnections between brain centers: pathways developed through experience

question

Theoretical Base for Visual Information Analysis

answer

-To cognitively interpret what is seen visually, you need to be able to: --Rely on your ability to receive visual input accurately through the visual system. To do this you need to: -Have functional visual acuity and ocular-motor skills -Be able to integrate visual input with other senses (Vestibular, tactile & proprioception) -Be able to process information: selective attention, visual memory, visual discrimination -Be able to learn & recognize relationships between characteristics (cognitive function): matching, categorization -Be able to store & retrieve information for later use (cognitive memory skills)

question

Processing: Visual Analysis Skills

answer

-Visual attention: alertness, selection of relevant visual input (screening out irrelevant information); ability to maintain focus & attention -Visual memory: ability to store information; recall information; integrate visual input with previous experiences -Visual discrimination: ability to detect & organize features -Developmental process: Size, shape, color: simple to complex Series & gradation Labeling by color, shape, size -Visual Figure-ground: ability to distinguish the object from its background -Visual Position in Space: spatial relationships of objects in relation to self or other objects (on, under, in, behind, larger, smaller, etc) -Visual Spatial Relationships: similar forms & their orientation can have different meaning: Example: d & b -Visual Form constancy: forms & objects can have the same meaning even when seen in various environments, positions, sizes, and configurations -Example: A = a = A = ɐ = A -Visual closure: Ability to understand, recognize the object or shape when only a part of it is seen

question

Responses: Observable Behaviors and Skills

answer

-Perceptual-motor skills: writing, reading, drawing, computation, eye-hand coordination for sports or expressive arts or music -Visual recognition for understanding and completing matching tasks, categorization, construction, sequencing tasks, whole-part relationships (construction and sequencing) -Affective behaviors: likes, dislikes, fears, approach, withdraw, enjoy, etc.

question

Role of Visual Perception

answer

-Develop the process of learning tasks -Perform tasks (ANY task from ADL to work to leisure to mobility) -Potential performance Issues: -Issues with visual clarity and ocular motor skills can affect an individual's motor performance and perceptual skill development -Issues with visual perception can impact eye-hand coordination, reading, writing, organizational skills

question

Visual Perceptual Difficulties

answer

-Visual perceptual issues may be due to: -Visual acuity -Ocular-motor control -Sensory to motor control -Sensory integration, sensory modulation and regulation -Cognitive and perceptual analysis (attend, focus, memory, categorize, discriminate, etc.) -Inter-connections between brain centers (LD, ADHD, etc.)

question

OT Evaluation of Visual Perceptual Skills

answer

-Some conditions place individuals at risk for visual perceptional issues (TBI, CP, ADHD, LD, Spina Bifida, Stroke) -OT needs to determine if issue in task performance is due to: -Problem with visual perception (without motor component) -Problem with motor coordination (without visual perception issue) -Problem with integration between visual to motor performance -Problem with sensory detection, attention, regulation, modulation -Through the use of various evaluations, you discover what may be the core feature of the issue in task performance -After evaluation, you can design your intervention plan appropriately

question

Visual Perception Assessment Instruments

answer

Various assessment tools used by OTs to assess visual perception: -VMI (6th ed.): can assess visual perceptual to motor integration, visual perception, and motor coordination but does not discriminate between types of visual perceptual skills -Test of Visual Perception-3 (TVPS-3): assesses components of visual perception, without a motor component -Motor-Free Visual Perceptual Test-3 (MVPT-3): assesses components of visual perception, without a motor component -Developmental Test of Visual Perception (DTVP-2): assesses components of visual perception, without a motor component -Test of Visual-Motor Skill (TVMS-3): assess visual motor skills -Parts of the Peabody Developmental Motor -Scales - 2 (PDMS-2): assess eye-hand coordination and visual-perceptual-motor skills -Parts of the Bruininks-Oseretsky Test of Motor Proficiency - 2 (BOT-2): assess eye-hand coordination and visual-perceptual-motor skills

question

Data Collection using Non-Standardized Assessment Tools

answer

Optimal method of data collection: Standardized and Non-Standardized Assessment Tools Standardized Assessment Tools Limited in number Used in sterile or artificial conditions Provide little usable information related to the complex nature of the domain of concern of OT

question

What are Non-Standardized Measurement Tools?

answer

Definition: Data collection that does not involve a specific, prescribed method or set of procedures.

question

Why are Non-Standardized Tests Used?

answer

Used to collect data on areas that are not covered by standardized testing. Helps you gather information that is specific and unique to the individual. Ipsitive Intrapersonal comparison

question

Non-Standardized Assessments

answer

May not be uniform in administration or scoring May not have full psychometric data Allow for flexibility and individualization Natural environment Requires strong clinical reasoning Sensitive and responsive to individual change

question

Evidence-Based Practice and Non-Standardized Assessment

answer

Empirical Research Experiential Evidence Theoretical Knowledge

question

OT Practice Framework

answer

Areas of Occupation Client Factors Performance Skills Performance Patterns Contexts and Environment Activity Demands

question

World Health Organization

answer

Participation or Involvement Activity Limitations or Challenges Participation Restrictions or Challenges External and Internal Influences

question

Methods of Non-Standardized Assessment

answer

Consider Person Environment/Context Occupation/Task (CMOP) Theory-base Observation Interview Self-report Questionnaires Performance tests Scales Checklists Protocols Screening

question

When to Use Non-Standardized Assessments

answer

Initial information and gathering stages Intervention planning Intervention Re-Evaluation Screening

question

Observation

answer

Definition: Systematic examination of some type of phenomenon. Most common procedure for collecting knowledge

question

Observation

answer

1.Formality (Semiformal vs. Informal) 2.Structure (Structured vs. Unstructured) 3.Qualitative and/or Quantitative All of the above = systematic approach More systematic and routine = More semiformal and semistructured

question

Observation Collection Methods

answer

Event Recording Duration Recording Rate Recording Time Sampling

question

Performance Tests

answer

Definition: Activity analysis as a means of collecting information about how a client carries out a task in context. Observation is the primary tool

question

Example

answer

Barthel Index Feeding 0 = unable 5 = needs help cutting, spreading butter, etc., or requires modified diet 10 = independent Bathing 0 = dependent 5 = independent (or in shower) 10 ADL items scored on a scale of 0-100 Describes a person's performance in terms of independence

question

Example 2

answer

Physical Performance Test Put on and remove a jacket 2 seconds or less = 4 points 2.5 -4 seconds = 3 points 4.5 -6 seconds = 2 points 6 seconds or more = 1 point 9 items; 36 points total Measures observation of physical function Performance test that uses the observation collection method of duration recording

question

Example 3

answer

Klein-Bell ADL Scale Socks -Grasp sock -Reach sock to R foot -Reach sock to L foot -Pull sock over L toes -Pull sock over R toes -Pull sock over R foot with heel to heel -Pull sock over L foot with heel to heel -Pull sock up to full extension on R leg -Pull sock up to full extension on L leg A check is placed next to each item that person can perform Each item is given a weighted number given by the test developers This tool is more sensitive to change in the person related to ADL items

question

Format for Observation

answer

-Describe the setting -Judge the emotional/social tone -Activity Analysis -How would society label the on-going activities -How long observation occurred and time of day -Summarize findings -Interpretation and actions if needed

question

Interview

answer

Definition: Process of inquiry during which an individual asks another person one or more questions Common procedure used in OT Open-ended vs. Closed-ended Questions "Describe how you are doing." "Tell me how it has been going." "Tell me more about that." "And what did you do then." "What do you do when you first wake up in the morning?" "How is your pain?"

question

Interview

answer

Formality Structure Qualitative and/or Quantitative

question

Interview

answer

Clinical reasoning process: Interactive Narrative Conditional

question

Questionnaire

answer

Definition: Client's self-report measure Paper-pencil Electronic device

question

RAND SF-36 Example

answer

The following are about activities you might do during a typical day. Does your health not limit you in these activities? If so, how much? Lifting or carrying groceries Yes, Limited a Lot (1) Yes, Limited a Little (2) No, Not Limited at All (3) Health related quality of life questionnaire Available in 36 (SF-36) or 12 items (SF-12)

question

Standardized Tools Description

answer

Prescribed methods Criterion-referenced Norm-referenced Group standards Quantitative Scores Less dependent on clinical reasoning for administration & interpretation Formal Strong external validity

question

Standardized Tools Advantages

answer

Get scores to determine level of function or diagnosis Tends to be more objective Can compare scores with other professionals Can delineate client's strengths & weaknesses Has reliability & validity data Can be used as a research tool or program evaluation

question

Standardized Tools Disadvantages

answer

Client needs to fit into norm sample Can be expensive May require special training; always requires practice Does not evaluate quality of skills or reason why a skill is deficient Does not allow for variability in clients or conditions May not be able to generalize to context May not represent skill ability in a functional context

question

Non-Standardized Tools Description

answer

Client-centered Individualized Naturalistic Qualitative Less formal Less structured Requires strong clinical reasoning for assessment and interpretation Strong internal validity

question

Non-Standardized Tools Advantages

answer

Easy to administer Portable Low cost Can use in context Looks at quality of skill performance Can be individualized Tends to be more relevant for the client Allows for cultural differences Appropriate for any age, population, disability Very adaptable/flexible

question

Non-Standardized Tools Disadvantages

answer

No scores, unable to make score comparisons Experience issues May tend to be more subjective; judgment-based Not diagnostic; difficult to make outcome predictions Limited reliability & validity data May have reimbursement issues without the "numbers" in documentation

OT Testing Final – Flashcards

Unlock all answers in this set

Haven't found what you were looking for?

Search for samples, answers to your questions and flashcards