Teaching Essays – Student Performance
Measuring Student Performance in a Distance Learning Environment: A Survey of the Literature
Over the past two decennaries, the outgrowth of enterprising engineering has made it possible for the universe to hold larning chances at its fingertips. Specifically, the World Wide Web has unleashed a powerful entree medium, tapping into a educable market some likely thought unapproachable. Educational establishments, peculiarly higher educational establishments, rapidly gravitated toward this booming, technological phenomenon as a means to supply direction to untypical pupils and, thereby, educate a coevals who may non hold had such an chance to prosecute in meaningful acquisition activities through normal schoolroom scenes. While higher establishments of acquisition had become comfy in offering onsite, night-time and evening instructional environments, engineering offered more and could make the multitudes. Increasingly, untypical pupils became mainstream pupils, and colleges, universities, and biennial educational systems began to provide to a more diverse pupil population ( Jones and Jo, 1999 – 3 ) .
Consequently, distance acquisition is the most proliferate signifier of national and planetary instruction. With direction being delivered via the Internet, telecasting and other media, it is expected to go an even more dominate force in educating pupils of all ages ( McIsaac and Gunawardena, 1996 – 1 ) . While that intelligence may be promoting, distance acquisition pedagogues are still perplexed as to how to measure pupil public presentation accurately and reasonably, given that pupils are spread out geographically and category sizes are frequently big. Additionally, teachers are challenged to keep and guarantee a balanced acquisition environment merely as they have in synchronal larning environments. This paper is non an effort to supply unequivocal solutions to those two issues, but it is an effort to show what the literature reveals about measuring pupil public presentation in an asynchronous acquisition environment.
For pupils in an online acquisition environment, grounds of cognitive and affectional acquisition is valuable, but even more valuable for policy shapers of grownup pupils is grounds of results such as improved occupation public presentation. ( Johnston, 2002 -67 ) Although, cognitive growing is an “enabling” result for the existent end of improved public presentation, some grownup instruction plans that assist those who desire to better life conditions do so with the purpose to supply accomplishments educational development for the benefit of improved occupation public presentation one time a occupation is secured. Johnson describes several technology-based instruction undertakings as holding three countries of appraisal for grownup scholar results: ( a ) workplace accomplishment development and occupation attainment, ( B ) academic accomplishment development, and ( degree Celsius ) orientation to literacy.
First, in the workplace accomplishment development country, the instruction undertakings seek to assist pupils go gainfully employed, better pupil public presentation in an bing occupation, or assist pupils in happening a better occupation. Adult pupils are taught workplace behaviour accomplishments such as squad edifice, interpersonal communicating, and employee/employer outlooks. In add-on to integrating the literacy constituents of reading and math into its course of study, the instruction undertakings besides include occupation application and choice procedures, interview techniques, and restart composing every bit good. ( Johnston, 2002 -9 ) While Johnson champions the thought that a primary step of success of this type accomplishment development would be for pupils to achieve a occupation and keep employment for a lower limit of 12 months after finishing the plan, he concedes that measuring command of these type accomplishments is hard. Second, the academic accomplishments discussed in Johnson’s research that are connected to a major country of measuring in grownup acquisition are those associated with the General Education Development ( GED ) trial. The GED assesses composing accomplishments, societal surveies, scientific discipline, literature and humanistic disciplines, and mathematics. Measuring the public presentation of pupils enrolled in GED plans is more touchable ; nevertheless, Johnson implies that assurance and anxiousness degrees of grownups who question their ain acquisition accomplishments could impact results and engagement. Finally, what starts with adults’ deficiency of assurance in academic accomplishments follows every bit with their uncertainness in literacy and schooling ( Johnston, 2002 ) . What is most unfortunate about the shyness of grownups in this respect is that literacy orientation plays a critical function in workplaceandacademic accomplishments ; it is the 1 accomplishment grownups need in order to be successful in the others. Johnson contributes this uncertainness of grownup scholars to old school failures, embarrassment possibly, and unknowingness of their lacks.
Although a pupil population in the online environment dwelling chiefly of grownups requires, possibly, different attending to class bringing and engagement, cognitive growing among all pupils in a distance acquisition environment is of main importance to pedagogues, chiefly because how to mensurate cognitive growing, peculiarly higher-order cognitive procedures such as job resolution, critical thought, creativeness, and cognition transportation presents a challenge. ( Johnson, 2002 ) It presents an even greater challenge in the asynchronous environment. Customary methods such as standardised trials have been relied upon to a great extent to uncover degrees of pupil public presentation in basic accomplishments but the trials are non true steps of higher-order cognitive procedures ( Cassady, 2001 ) . Even in mensurating basic accomplishments, variableness in disposal of standardised trials and the graduated tables upon which they report the consequences are non clear indexs of pupil public presentation ( Cassady, 2001 ) . Cassady advocates that trim trials and reliable appraisals are better appraisals than traditional standardised trials, but they, excessively, fall short. Tailored trials evaluate unequivocal results of peculiar intercession methods. These type trials are developed to specifically fit what may be a alone acquisition environment. The trials can supply a clearer appraisal of pupil public presentation if they are devised with attachment to basic trial development criteria—that is, guaranting accurate cogency, dependability, and graduated table of describing tonss. Authentic appraisals are straight related to systematic observation ( Cassady, 2001 ) . This attack requires more in-depth analyses of single scholars because it involves direct scrutiny of learners’ work quality, detecting learner public presentation, or measuring learner work merchandises. ( Cassady, 2001 ) In the asynchronous environment, two of these facets is rather accomplishable, but one, observation of public presentation, may non be every bit simple. Nevertheless, for some pedagogues, public presentation observation may non needfully be what one literally sees in a typical schoolroom scene ; it can be what one views as participatory observation when measuring pupil public presentation in distance acquisition.
Affectional sphere steps do offer quality information about the impact of larning on one’s emotional and motivational province, but affectional variables are excessively wide and diverse to see them singularly when measuring scholar results ( Cassady 2001 ) . On the other manus, since it is more recognizable that affectional sphere procedures be included in rating schemes to measure learner emotional and motivational province, and the efficaciousness of the plan under which acquisition is delivered, Cassady asserts that judges can see a conceptual model under which affectional concepts can be used. The rating design proposed by Cassady employs those constructs which focus on attitude, motive, and self-perceptions of ability and accomplishment. Each conceptual class reveals features of scholars that determine involvements and desires, achievement motive, and forms of public presentation, severally. Just as with cognitive sphere appraisal processes, the development of affectional sphere appraisals should adhere to acceptable design criterions of rating. Additionally, harmonizing to Cassady, judges should see seven rules of measurement design: ( a ) sphere sampling, ( B ) societal desirableness, ( c ) response prejudice, ( vitamin D ) developmental rightness, ( vitamin E ) lucidity of linguistic communication ( f ) dependability and cogency, and ( g ) triangulation. Again, Cassady emphasizes that the wide and diverse nature of affectional sphere measuring make it hard to happen a step that meets the specific demands of an rating ; in such instances, following established rules as those does let the turning away of jobs that could restrict converting informations.
Since distance, clip, age, and gender no longer impede one’s chance to prosecute an instruction, a alone planetary community has been emerging, and the civilization of instruction has changed significantly ( Jones and Jo, 1998 ) . As a consequence, the research suggests that flexibleness in design of on-line direction is a primary factor to see to consequence successful scholar results, while deficiency of face-to-face contact—or interactivity—may or may non hold every bit large an impact. Stansfield, McLellan, and Connolly suggest that flexibleness allows scholars to pull off their ain acquisition by accessing class stuffs in whatever sequence they choose and finding whether in-depth or superficial perusal is needed based on anterior cognition. Conversely, although Stansfield, McLellan, and Connolly, acknowledge the value of interactivity in the online environment, they imply that the quality of on-line direction is far superior to its face-to-face opposite number. DeVries & A ; Wheeler ( 1996-21 ) difference this deduction, mentioning the deficiency of face-to-face contact as a major barrier in distance instruction. Interactivity promotes understanding of constructs and deficiency of it in the online environment could do jobs of learner abrasion, failure or dissatisfaction ( Dennen, Darabi, and Smith, 2007 -12 ) . Martin ( 1999 -21 ) [ besides mentions the deficiency of face-to-face interactivity as an obstructor to distance acquisition that could hold an impact on scholar results. Despite the facts of research that support both sides of the interactivity statement, pupil public presentation is influenced if instructional schemes for prosecuting scholars of different features is non considered in the course of study development procedure, and strong, consistent interactivity is particularly needed between facilitator and pupils throughout the length of the category, plan or seminar ( Jones and Jo 1999 – 3 pg 2 ) .
Because of the increasing demands of distance acquisition, on-line instruction plans are designed with all pupils in head, developing assorted appraisal methods and techniques as pupil demographics dictate. And, although the literature does uncover variables interior decorators need to see when covering with assorted pupil population groups, the literature is besides in understanding that standard instructional design rules are necessary to positively impact learner results of all pupils. Therefore, the Naval Air Warfare Center offers a tool or rubric for measuring the quality of web-based direction that could be used to measure a figure of audiences. Debatably, the tool may possibly be one of high involvement to distance learning teachers because appraisals in the online larning environment should be able to mensurate that pupils are larning what they are suppose to larn. Inevitably, so, appraisals may sometimes be more important in web-based direction because teachers are unable to set course of study or proving methods as often or as freely possibly as schoolroom teachers. Consequently, it is critical for interior decorators of distance acquisition to hold a quality and structured rating method or tool for non merely effectual public presentation appraisal but besides for effectual feedback to scholars. ( Hayes, Stout, and Ryan-Jones, 2005 # 20 pg 5 ) . The undermentioned summarizes cardinal points from the Naval Air Warfare Center’s treatment on public presentation appraisal:
- Appraisals should be relevant to what the scholar cares about.
- Should be linked to class aims.
- Should measure what the aim says it will measure.
- Should mensurate accomplishments and cognition required on the learner’s occupation.
- Should mensurate what pupils canmakewith what they know.
Relevance appears to be a repeating subject throughout the literature. Research workers are careful to stress the connexion between what is stated to be taught, what is really taught, and the impact of both on scholar cognition and behaviour. Research from Virginia Tech University and Middle East Technical University in Turkey provinces that the content has to be related to the field of survey and be relevant to rules and criterions ( Yilmax and Cetinkaya, 2002 – 10, pg 81 )
- Appraisals should be larning activities themselves.
- The type of appraisal determines how the pupil surveies ( and how much ) .
- Students are more likely to utilize “surface larning approaches” ( callback ) to analyze for multiple pick tests, but apply more meaningful larning methods when fixing for essay assignments.
Rote memory is purported to be lesser of an attempt than elaborative dry run when easing meaningful acquisition ( J.E. Ormrod, 2008 ) . As with the writers of this study, the author supports the sentiment that higher-order thought requires assessment steps beyond simple callback of facts.
- Appraisals should hold clear intent and vary in design
- Design appraisals to reflect corporate information of class content.
- Appraisals monitor pupil advancement better when they are a portion of a go oning procedure and non done periodically
- Students need aid in puting realistic ends that meaningful, come-at-able, and based on their experience
- Students respond to better when appraisals are more piquant, interesting, and applicable
Regardless of what type appraisal is being designed, much of the literature supports a design scheme that employs unequivocal methods of design rules. Some pedagogues suggest making an even more structured rubric such as single electronic portfolios for pupils to measure their ain work. The portfolios contain students’ coursework or independent surveies brought together in the electronic environment ( Yilmax and Cetinkaya, 2002 -10 pg 80 ) . It is noted besides that the design principles adhere to theoretical rules of course of study development.
In the sentiment of this author, any reappraisal of the literature relation to measuring pupil public presentation in the distance acquisition environment would be derelict if it did non include what some pedagogues would likely state is the benchmark in educational evaluation—Kirpatrick’s Four Levels of Evaluation. Aleckson and Martinelli apply Kirkpatrick’s Four Levels of Evaluation to an asynchronous distance larning environment.
Level One ( learner’s reactions ) measures the reactions of pupils to the plan in which they are enrolled ( Aleckson and Martinelli, 2005 -7 pg 1 ) . It attempts to reply inquiries related to relevance and likability of the plan. In an online environment, automated cyberspace study tools can roll up informations for degree one rating. Positive reactions may non needfully vouch the transportation of cognition through meaningful acquisition, but negative reactions more or less reduces the likeliness of either.
Level two rating ( larning ) now attempts to measure the extent pupils have advanced in accomplishments, cognition, or attitude. It can be computer-generated and should be done earlier, during, and after direction. ( Aleckson and Martinelli, 2005 -7 pg 1 ) In a web-based bringing environment, instructional interior decorators could supply computer-generated pretests ( to place countries in which pupils already run into the criterion or countries in which pupils need excess direction ) and in-course rating instruments.
Level Three ( transportation ) rating steps occupation public presentation and application of information learned ( Aleckson and Martinelli, 2005 -7 pg 1 ) . In the web-based environment, Internet studies can roll up feedback from directors about whether the pupil can use class cognition to occupation undertakings. Formal study instruments, associating specific larning aims to occupation undertakings, could study supervisors about the students’ on-the-job public presentation. Some educational establishments may even see carry oning face-to-face ratings, VTCs, or telephone interviews and focal point groups. In those cases, this author has found that the chance of roll uping richer, more telling feedback rises.
Although more appropriate for measuring the “program” and non student public presentation, level four measuring is a direct consequence of holding improved pupil public presentation through the first three degrees. Level four measures the impact of developing on organisational aims. Harmonizing to Aleckson and Martinelli, “Because of the disbursal and clip, measuring on this degree is seldom done at the individual eCourse level.” Evaluators are looking for such touchable consequences as increased production, improved quality, decreased costs, reduced frequence of accidents, increased gross revenues, and even higher net incomes or return on investing. From a concern and organisational position, this sort of high-ranking rating occurs as portion of accreditation reappraisals, organisational effectivity surveies, and the similar.
Concerned, excessively, about pupil public presentation, the American Association for Higher Education ( AAHE ) , a national organisation dedicated to bettering the quality of higher instruction in the United States, puts forth nine rules that it believes to be best patterns for measuring pupil acquisition ( AAHE, 2001 ) . Although these rules are non specific to distance acquisition, they do hold specific deductions for measuring pupils in distance acquisition environments:
- The appraisal of pupil larning Begins with educational values.
- Assessment is most effectual when it reflects an apprehension of larning as multidimensional, integrated, and revealed in public presentation over clip.
- Assessment works best when the plans it seeks to better hold clear, explicitly stated intents.
- Appraisal requires attending to results but besides and every bit to the experiences that lead to those results.
- Assessment works best when it is ongoing non episodic.
- Assessment Fosters wider betterment when representatives from across the educational community are involved.
- Assessment makes a difference when it begins with issues of usage and illuminates inquiries that people truly care about.
- Assessment is most likely to take to betterment when it is portion of a larger set of conditions that promote alteration.
- Through appraisal, pedagogues meet duties to pupils and to the populace ( AAHE, 2002 ) .
So popular are these best patterns, in her research, this author discovered them used by a figure of universities and academic appraisal offices. Principles one and seven are particularly worthy of farther treatment. AAHE advocates that appraisal should be driven by the educational mission and values. This rule suggests that appraisal should be more closely connected to on-the-job public presentation, a largely agreed upon factor when measuring grownup scholar results. AAHE farther declares that appraisal should turn to issues that people truly care about. It “starts with the inquiries of decision-makers… [ and ] involves them in the assemblage and interpretation of data.” This rule suggests that the assessment procedure should reply inquiries stakeholders have about pupils. Assessment procedures should be peculiarly sensitive to the concerns of instructors, decision makers, and school and legal guardian board functionaries. They should be involved in finding how to measure pupils and construing assessment informations.
Regardless of how well interior decorators of course of study devise rating appraisals and what structured methods, tools are best patterns they use, pupils still play a important function in their ain public presentation. While teachers can plan the best appraisals, acquisition is still a undertaking online pupils have chosen to partake in and hence must keep some answerability in their ain result. Therefore, there is a demand to see the procedure of larning as it relates to types of pupils ( Stansfield, McLellan, & A ; Connelly, 2003 ) . Educators can non disregard this important factor. Stansfield, McLellan, and Connelly identify three attacks to acquisition by which pupils approach their ain acquisition procedure: ( a ) deep, ( B ) surface, and ( degree Celsius ) strategic. In the deep attack, pupils take on acquisition as a personal development and desire ( Stansfield, McLellan, & A ; Connelly, 2003 ) . These pupils are interested in the acquisition, and their behaviour alterations are apparent. Surface attack pupils are at that place to make full in a square ( Stansfield, McLellan, & A ; Connelly, 2003 ) ; they merely prosecute in rote memory to carry through scrutiny demands. Finally, strategic scholars set up a program that leads to obtaining the best classs possible ; their end is master the appraisals ( Stansfield, McLellan, & A ; Connelly, 2003 ) . ( 2, pg 174 ) Each of these types of pupils has established his or her ain procedure of acquisition and each procedure can act upon learner results. In the distance acquisition environment, it appears that scholars determine their public presentation ; instructors merely assess the engagement.
Principles of Good Practice for Assessing Student Learning. American Association for Higher Education: 1996. Retrieved January 18, 2008, from hypertext transfer protocol: //www.fctel.uncc.edu/pedagogy/assessment/9Principles.html
Kirkpatrick’s New Online Approach to Evaluating Asynchronous eCourses.”Proceedings from the 20ThursdayAnnual Conference on Distance Teaching and Learning. Jon Aleckson and Julie Martinelli. University of Wisconsin ( 2005 ) . hypertext transfer protocol: //www.uwex.edu/disted/conference/Resource_library/proceedings/04_1366.pdf
barrier meven argue that it is non merely flexibleness in scholar entree that is of import but besides interactivity—interactivity between facilitators and pupils, and between pupils and other pupils. [ In discoursing the importance of interactivity, Additionally, facilitator-learner resonance is peculiarly of import in the online acquisition environment because there is a sensed positive impact on pupil public presentation and satisfaction ( 12, pg 1 ) Other research, nevertheless, ( Stansfield # 2 ) disputes this statement, stating that the quality of direction is far superior to synchronal direction that involves individual to individual interaction ( 2, pg 175 )
Additionally, facilitators who teach several on-line categories or learn one or two from semester to semester must non confound any of the categories with assorted, out-of-date, equivocal or unrelated category instructions. Traditional-aged pupils may non take note, but grownups will. Each of these rules is of import for making the affectional sphere via the e-Information Highway, which, in bend, could impact motive and later motive.
Technology is here to remain. The continued and extended and powerful growing of the World Wide Web has affected the educational universe forever. Instructional interior decorators and facilitators of larning are called upon to be more originative, various, and adaptable than of all time. In making so, they must non