Validity in relation to your subject Essay Example
Validity in relation to your subject Essay Example

Validity in relation to your subject Essay Example

Available Only on StudyHippo
  • Pages: 5 (1209 words)
  • Published: July 15, 2017
  • Type: Case Study
View Entire Sample
Text preview

In the instructional assessment realm, there are two main areas: "summative assessment" and "formative assessment," also known as "Assessment for Learning." Summative assessment is the traditional method used in Scottish Secondary Education, such as end-of-year exams. Its aim is to measure and record an individual's achievement (Scots Qualifications Authority, 2009c, p.5). According to the Scots Qualifications Authority (SQA), summative assessment is conducted at the end of a learning activity or program to make a judgment on overall achievement. It serves the purpose of recording and often grading performance in relation to learning objectives (Scots Qualifications Authority, 2009c, p.106). This type of assessment is typically used for high-stakes qualifications where success depends on the outcome of a single examination, like Higher Mathematics qualification.

On the other hand, formative assessment is gaining prominence as the primary method in evolving Scottish Seconda

...

ry Education. According to SQA (2009c, p.98), formative appraisal provides developmental feedback to students so they can plan their future learning. It is not recorded externally.Formative appraisal, or "Assessment for learning," is a critical component of Learning and Teaching Scotland's appraisal development program known as Appraisal is for Learning (AifL). The University of Luton CAA Centre has provided an overview of the advantages and disadvantages of various types of appraisal. Summational appraisal is beneficial for external parties, such as employers, educational institutions, and courts that require information to make decisions. However, it may not effectively communicate complex data about a student's abilities, like strengths in algebra but weaknesses in concretion. Conversely, formative appraisal allows students and interested individuals to gain a more comprehensive understanding of their abilities. This knowledge can be utilized to guide further study efforts an

View entire sample
Join StudyHippo to see entire essay

enhance overall performance. Personal experience shows that summational appraisal can be highly stressful since it solely relies on performance on examination day without considering overall course performance. In the past, formative appraisal did not receive much emphasis until Black & Wiliam published "Inside the Black Box" in 1998. This publication marked a significant turning point in Scottish education and potentially served as a catalyst for change with the introduction of 'Curriculum for Excellence' (CfE).According to CfE principles, learners should actively participate in planning and reflecting on their own learning through formative appraisal, self-evaluation, peer evaluation, and personal learning planning. 'National Qualifications' under CfE will transition from traditional summational appraisal to a combination of assessment types. The Scottish Qualification Authority emphasizes the importance of validity in assessments regardless of their form. Validity refers to how accurately an assessment measures intended abilities and if interpreting and using the results are appropriate. For example, a driving test that observes candidates driving has higher validity than a trial where a candidate describes their approach to driving. Therefore, ensuring high validity is crucial in any assessment. There are different types of validity including face validity, criterion-related validity (concurrent and predictive), content validity, and concept validity according to Black & Dockrell (1980), CAA (2002), Deale (1975). Face validity is defined by Deale (1975) as the perception that the test accurately measures its intended purpose. Criterion-related validity refers to the relationship between test scores and other standards such as teachers' estimates or external examination results as described by Deale (1975). This can be either concurrent or predictive.Deale (1975) emphasizes the importance of content validity for teachers, as it evaluates how well a

test covers the intended syllabus area. The CAA (2002, p.11) also highlights the significance of good concept validity in measuring how closely an assessment aligns with the desired domain being measured. Deale (1975) further suggests that good content cogency requires a test to reflect both the covered content and the balance of learning leading up to it.

The SQA publishes 'arrangement papers' for each level of Higher Still Mathematics, which outline the curriculum and what should be taught by instructors for students to pass the final examination. If instructors follow this curriculum, then theoretically, the exam should have high content and concept cogency. To ensure integrity and quality, experienced mathematics educators review and approve test paper inquiries before publication. The SQA has a responsibility to closely relate exam questions to the content of arrangement papers.

Previous years' test papers and marking instructions are available on the SQA website for reference during instruction. These resources provide valuable insights on how marks will be given or deducted, enhancing examination validity when effectively utilized.However, if similar questions are repeated every year, there may be a potential drawback. This could lead instructors to solely focus on teaching students how to answer specific questions, potentially limiting their understanding of the subject matter if those questions don't appear in the current year's examination.

To prepare for the final end-of-course exam, students must complete at least three end-of-unit National Assessment Bank (NAB) tests. All three NABs are required for sitting the terminal exam. Each NAB assesses a specific unit and does not reassess previous units. The NAB evaluates multiple subjects covered in each unit.

Students must meet or exceed a threshold mark per subject to pass

the pass/fail NABs. Some view Higher Mathematics NABs as a minimum competency test with low predictive validity for the final exam, while music NABs are seen as an indicator of high competency and likely success in the final exam according to a music teacher (Scots Qualifications Authority, 2006; Scots Qualifications Authority, 2009c; Webster, 2009).

Some schools/teachers may require an additional internal extension test called an A/B test to better evaluate students' progress towards passing the final class examination, ensuring high predictive criterion-validity.

If students fail an internal extension test (NAB), they have one opportunity to retake the failed portion. However, failing again would result in no further chance for retaking and therefore exclusion from sitting for the final class examination.
During the placement, students are given multiple opportunities to pass the NABs on their first attempt. They were provided with three practice NABs that they attempted either in class or at home. The entire class would then review these practice NABs in the next lesson with their instructor.

The only distinguishable difference between these practice NABs and the actual ones is that some variables (letters), numbers, and units of measurement within the questions have been altered. This raises concerns about consistency across all schools; however, I do not possess information regarding whether this is a standard practice universally adopted in all institutions.

The Higher Mathematics final class test consists of two papers: "paper one" and "paper two." "Paper one" is a non-calculator test with a mix of multiple-choice questions in Section A and extended response questions in Section B. On the other hand, "paper two" is a calculator test that only includes extended response questions.

In 2009, the 'paper one' test

consisted of 20 multiple-choice questions and four extended response questions. The 'paper two' test had seven extended response questions (Scots Qualification Authority, 2009d).

Before finalizing the documents, meetings are held to determine the minimum passing grade, which is equivalent to a 'C' grade (Scots Qualification Authority, 2009b; Scots Qualification Authority, 2009c).In the 2009 paper (Scots Qualification Authority, 2009b), grades were categorized as follows:
- Total available marks: 130
- Upper A Grade: >=106 (~81.5%)
- A Grade: 93-105 (~71.5%)
- B Grade:77-92 (~59.2%)
- C Grade:62-76 (~47.7%)
- D Grade:54-61 (~41.5%)
- No Grade:<54 The papers are externally marked by SQA approved markers who have at least three years of teaching experience at the relevant level based on finalized marker instructions (Scots Qualification Authority, 2007b; Scots Qualification Authority, 2009a). Quality assurance procedures are in place to monitor markers' performance (Scots Qualification Authority, 2007b; Scots Qualification Authority, 2009c). Overall, appropriate measures and controls are implemented to ensure the validity of the 'Higher Mathematics' course. However, assessing mathematics in a CfE inter-disciplinary project presents unique challenges that must be addressed for accuracy. The CfE booklet outlines the experiences and outcomes for each of the eight course of study countries and the three duties of all practitioners - Health and wellbeing across learning, Literacy across learning, and Numeracy across learning. These provide flexibility for teachers but require moderation to establish comparison standards between assessors for validity, reliability, and practicality of the assessment.In order to effectively evaluate the mathematics portion of a CfE inter-disciplinary project, it is recommended that a leader within the mathematics department is designated. This leader will be responsible for identifying relevant CfE experiences and outcomes pertaining to mathematics. The text emphasizes the importance of including

desired outcomes and measurements within the scope of the project, while also ensuring consistency through moderation in this section. Schools are encouraged to engage in direct communication and utilize resources such as 'Glow' and LTS national assessment resources to discuss and share best practices with other schools. The moderated output will be shared with the interdisciplinary project team, who will then select members to create comprehensive lessons covering all areas of the curriculum. These lessons should include success criteria, formative assessment strategies, and supporting evidence. It is expected that Assessment is for Learning (AifL) strategies will be used for assessment purposes. Once the lessons have been created, they will be reviewed by the math lead to ensure coverage of the math 'experience and results'. The math department will assess evidence collected from students throughout the project in order to determine if desired 'experience and results' have been achieved. If not, appropriate actions will be taken to address any identified gaps, with feedback provided to the project team. This process ensures high validity.
In Section 2 - Formative Appraisal in Practice, Ollerton (2006, p.218) highlights the importance of continuous assessment in mathematics and the reliance on feedback from students. According to Teacher TVs (2008), creating learning environments where children can discuss their ideas is crucial for obtaining feedback. Effective questioning by teachers encourages these discussions, while self-assessment and peer-assessment help children understand how to produce high-quality work and improve their skills. The main components of formative appraisal include effective questioning, feedback through marking, peer assessment and self-assessment, and using summative tests formatively (Teachers Television, 2008). These elements are encompassed in AifL approaches (See Appendix 1).

To understand the

practical impact of formative appraisal, an action research project was conducted during placement one using an AifL approach. Action research is a practical method of evaluating one's own work to ensure it meets desired standards (McNiff 2002, p.6). McNiff (2002, p.11) explains that the action plan involves reviewing current practice, identifying an aspect to investigate, envisioning a way forward, implementing it, and assessing the results. The modified action is then monitored, reviewed,and evaluated.However,due to time constraints in this research study only the first five steps were followedDuring observations of an S2 class preparing for their National Assessment Test (NAT), it was noted that only a few students were willing to volunteer answers when the teacher asked questions. The majority of students who did not raise their hand were also hesitant to provide an answer when directly asked.

Identify a facet that we want to investigate

The main focus of the action research project is to examine if the number of students willing to volunteer an answer to question(s) can be increased by using another Assessment is for Learning (AifL) scheme.

Imagine a way forward

The class teacher (CT) has previously used different AifL schemes with this class, such as 'Learning purposes' and 'Traffic lighting'. However, the CT has not yet tried the 'Using incorrect answers' AifL scheme with this class. Consequently, it has been agreed that this scheme will be implemented in this class over one week's worth of lessons. The aim is to enhance students' confidence and create a more supportive atmosphere by allowing the entire class to explore areas that lead to these incorrect answers and expand their range of experiences (Clarke, 2005a; Clarke, 2005b;

Harris, 2007; Learning and Teaching Scotland, 2008).

Give it a try

Before implementing the 'Using incorrect answers' scheme, it is important to establish a baseline measurement of evidence in order to assess progress (McAuliffe, 2009).After implementing the AifL scheme, it is important to gather evidence to determine its impact. As part of the S2 alteration, students were required to attempt six mental maths inquiries per lesson. Once all six inquiries were asked by the instructor, students had the opportunity to volunteer their answers by raising their hands. This provided a suitable time to collect measurable evidence before, during, and after the AifL scheme.

The original research plan involved observing a small number of randomly selected students over a three-week period divided into three stages. Stage one aimed to establish the baseline measurement before implementing the "Using incorrect replies" scheme. Stage two focused on implementing this scheme, while stage three implemented both "Using incorrect replies" and AifL schemes.

However, due to limited contact time with this class because of in-service days and other factors, adjustments were made for additional data collection. Each stage was extended from one week to four lessons for phase one and phase two. The final phase was shortened to three lessons.

As concerns arose about potential negative effects on student self-esteem if abruptly discontinuing the "Using incorrect replies" scheme in the final phase approachedThis concern arose from the observation that the AifL scheme had a positive impact on student self-esteem, as mentioned in the 'Take stock of what happened' section. After consulting with the class teacher (CT), it was decided to continue with the 'Use of incorrect replies' scheme, but at a reduced rate for

the rest of the research period, specifically with two inquiries per set.

Observing the Effects:

The analysis of findings will be discussed in the following section (refer to Appendix 2 for raw data). The study consisted of three phases: Phase 1 - conducted before implementation; Phase 2 - carried out during the "Using incorrect replies" scheme; and Phase 3 - implemented at a lower rate compared to Phase 2.

In Phase 1, two students attempted to answer all inquiries willingly, while three students refused to answer any inquiries. During Phase 2, four students were willing to answer most inquiries and one student answered approximately 20% of them. In Phase 3, one student answered all inquiries, three students answered most inquiries, and one student answered about one third of them.

Considering each individual student and looking at the class as a whole, there were observations made about their behavior before and after implementing the AifL scheme. Pupil 1, Adam initially displayed high confidence and capability but did not raise his hand on four occasions during Phases two and three.The reason behind this change in behavior is uncertain, but it is possible that the student felt neglected by the teacher as other students became more involved. Further investigation would be required to confirm this assumption. Another student, Brian, lacked confidence initially and rarely participated in class discussions. However, based on collected data, it appears that the AifL scheme had a positive influence on him. Initially sitting at the back of the classroom with another student, he never raised his hand during phase one. Although hesitant at first during phase two, he gradually started raising his hand more frequently by the end

of this phase. This trend continued throughout phase three, indicating increased confidence and willingness to contribute to class discussions. Improvement was also observed in his mental math score.

On the contrary, Charley initially showed disinterest in class activities. She would often write in her notebook without making eye contact with the teacher unless directly addressed. Additionally, she frequently engaged in conversation with her seat partner whenever given an opportunity. Similar to Brian, Charley was seated at the back of the classroom and did not raise her hand during phase one. However, she raised her hand five times during lesson five of phase two and again at the beginning of lesson nine in phase three. It is worth noting that on the day Charley raised her hand during phase two, her seating partner was absent.
This implies that Charley's relationship could be a factor in their lack of engagement during lessons. Further investigation is needed to confirm this hypothesis.

During phase three, specifically lesson nine, which focused on team challenges, all students participated by raising their hands. However, Debbie was confident and enjoyed being in class but had a tendency to interrupt or distract others by shouting out. She sat alone at the front and eagerly answered questions throughout the research period, except for lesson seven when she remained quiet. Overhearing her heated discussion with another student before class may explain this sudden change in behavior. Nevertheless, more research is required to identify other factors contributing to it.

Student 5, Edith, struggled with mathematics and also sat alone at the front of the class. Similar to Brian's experience mentioned earlier, Edith did not raise her hand during phase one but

showed improvement in phases two and three where she actively participated. This suggests that the Assessment is for Learning (AifL) scheme had a positive impact on Edith as well. Additionally, her mental math score improved.

Although there is no concrete data available to support it, it seemed that the AifL scheme had a positive impact on the entire class collectively. Student participation noticeably increased during the research period. Initially, only a few students would raise their hands;However, most students were willing to attempt answering questions by the end of phase two. Therefore, it was decided to continue with this approach in phase three, albeit at a reduced rate. One interesting observation was that implementing the "using incorrect answers" strategy created a supportive atmosphere where students felt comfortable sharing their problem-solving methods for mental math questions. The research project aimed to observe how students responded to these questions. Initially, approximately 50% of students scored four or less out of six. However, by the end of the project, every student in the class achieved scores of four or higher.

It is important to note that drawing definitive conclusions based on this research is challenging due to time constraints and limited sample size among other variables. Nevertheless, it seems that implementing the "Using incorrect answers" AiFL strategy has encouraged more students to volunteer their answers. It should be noted though that one student named Adam may have been negatively affected by this approach.

The significance of considering individual learning needs when implementing teaching strategies is emphasized. This assignment has prompted deep reflection on professional development in relation to the eight benchmarks of Initial Teacher EducationThe benchmarks outlined by the University of

the West of Scotland (2009) cover various aspects such as Curriculum, Education Systems and professional duties, Principles and positions, Teaching and acquisition, Classroom administration and direction, Pupil appraisal, Professional contemplation and communicating, and Professional values and personal commitment. Each section of this assignment addresses multiple benchmarks. For instance, in the Assessment theory section, relevant professional literature has been evaluated (benchmark 2.4.1) to gain a better understanding of assessment theory. This involves recognizing the advantages and disadvantages of summative and formative assessment as well as their impact on students' evaluation (benchmark 2.3.1). While comprehending the theory is important, implementing it into practice as an initial teacher education novice requires considering diverse perspectives from other teachers' experiences/views/opinions for supporting personal development (benchmark 3.2). The Validity of national summative appraisal section provides updated information on current national summative assessments. Through research, I have gained a thorough understanding of the SQA's procedures and protocols for ensuring validity in these assessments (benchmark 2.4.1). Although I acknowledge the necessity of a national summative assessment for high stakes qualifications at present, I question its true worth.
Relying solely on assessments to make decisions about students may not provide a comprehensive view; it simply confirms meeting specific standards at the time of assessment (benchmark 3.1). The validity of math assessments in an inter-disciplinary project within CfE presents opportunities for collaboration and establishing relationships outside one's department. This collaboration can enhance everyone's experiences (benchmarks 2.1 & 3.3) and support curriculum development throughout the school (benchmark 2.4.3). From personal experience, I have participated in diverse inter-disciplinary teams with different cultures, and I strongly believe that the benefits outweigh any perceived drawbacks. These experiences have taught me

the importance of valuing everyone's opinion, regardless of their background.

Regarding the "Formative Assessment in Practice (Action research)" section of this assignment, it has provided me with a deeper understanding of the advantages of research in education (benchmark 1.3.2) and how formative assessment can impact students' achievement (benchmark 2.3.2). Initially, I had doubts about conducting an action research project due to its short timeframe and small sample size, but I underestimated how much formative assessment can influence a student's learning and confidenceIn summary, this assignment has provided me with valuable learning opportunities. I have come to realize that I initially underestimated the time required for research and reflection. This experience has taught me not to make the same mistake in the future. As a new type of assignment for me, I have dedicated significant hours to reflecting on the best approach. Through this assignment, I have acquired new skills such as accessing and evaluating relevant literature for my profession (benchmark 2.4.1). Additionally, I have learned how to take responsibility for my own professional development (benchmark 3.2), express my opinions to improve teaching practices, and appreciate the significance of reflection. The submission of this assignment was made through 'Turnitin'.

Get an explanation on any task
Get unstuck with the help of our AI assistant in seconds
New