If you want assessments to reach their full potential, their quality should be as high aspossible. Three (quality) requirements are important in this respect: reliability, validity and transparency (Van Berkel et al, 2014).To work on reliability you can design a model answer –also known as a response model, correction key or correction prescription - which allows you to assess as accurately and objectively as possible.When there are several evaluators for the same exam a model answer is highly recommended: this creates a consensus on the criteriathat must be used to assess and makes sure that everything is evaluated from the same point of view.
What is a model answer?
A model answer indicates which elements you should focus on during your assessment.Based on theseresponse elements or assessment criteria,you indicate whichstudents’ answers get which grades. The model answer also includes the maximum score for each question and (where appropriate and possible) the division of the maximum scores intosubscores(Van Berkel & Bax, 2014).Decide in advance whether you want every response element or assessment criterion to carry its own weight or whether there are several different elements that will make upone rating point and whether there are certain elements with greater weight.
You can also record in the answer model how you will deal with, for example:
- Partially correct answers and answers that you didn’t think of in advance,
- Recurring errors (an earlier error that makes the rest of the answer wrong),
- Identical mistakes,
- Language or spelling errors,
- Whether there is the possibility that a score lower than zero can be obtained.
A model answer isintendedas a guideline for the assessor and is best made during the formulation of the exam questions. Based on the students’ responses you can, if necessary, refine or modify the model answer.It may be useful to put the model answer to the test at the start of the assessment by, for example, taking a sample of some exams and checking them with the key.In case it is necessary, you can makeadjustments in time.This is highly recommended, especially when you are assessing as ateam.
Examples
In practice, there aredifferent kinds ofmodel answers.Which type is best, depends on the type of assessment (and its questions) that you wish to set.For some exams correct (response) elements or assessmentcriteria can be formulated, the so-called 'model answers'.For other exams, e.g. in a paper (essay question) and design projects, it is more about describing those various aspects which should be taken into account throughout the assessment and the respective weights assigned to them, the so-called 'rubrics'. Importantly, the assessment criteria used should include unambiguous formulations, should not overlap with one another and should be based on the learning objectives of the course.
Written exam
In a written exam with open questions the model answer consists of a model answer with clearindications of the essential components that should be included in the answer.
For example:
Exam question
What is the difference between a negative reinforcement and a negative punishment?
Model answer
| Maximum score: 3 points | |
| Framing and definition(for 2 points) | |
|---|---|
From the answer it should be clear that:
For every correct answer: 0.5 points | |
| Examples (for 1 point) | |
For every correct example: 0.5 points |
Points are awarded based on the number of correct elements, which are in accordance with the model answer thathas been drawn up in advance.In the example above, a student who only knows the definition of the concepts required (negative reinforcement and punishment) only gets 1/3.The student does not connect the concepts of negative reinforcement and negative punishment with instrumental conditioning, nor does (s)he employ names like Skinner and Thorndike, so (s)he cannot achieve half of the points on offer.A student who does mention the namesalready gets 2/3.If the student gives an example to demonstrate his/her understanding of the definition using an own application, this will allow him/her to earn the maximal three points.
You can create gradations in points by:
- Distinguishing between response elements that must be minimally present and additional elements -such as nuances, links to other curriculum components, examples, or critical comments - that the student could use to distinguishhim/herself from the others.
- Allowing for space in the distribution of points in order to also assess the quality of the response, for example, by varying the scores (1/2, 1.5/2, ... depending on the quality with which the student describes the response element).
- Providing a penalty (e.g. subtract points as done in the example) when something in the answer should certainly not be there or in case of an intolerable error.
Oral exam
When you give an oral exam, you should also think in advance about which assessment criteria you want to use or about possible right or wrong answers. Again, it is important to make the link with your learning goals: if you want to know whether a student can not only explain certain knowledge but can also analyse or apply a particular theory, then there must be sufficient, explicit attention paid to your model answer for these elements.
For example:
Exam question
What is the difference between a negative reinforcement and a negative punishment?
Model answer
| Content | Concepts are explained and framed correctly | Framing
Concepts
| Framing (max. 1 point) 0,5 points per correct theory Concepts (max. 2 points)
|
|---|---|---|---|
| The difference between the two concepts is illustrated with examples |
| Examples (max. 1 point)
| |
| Extra questions | The student answers the extra question(s) sufficiently | What is the big difference between positive and negative reinforcement? Explain. | 1 point per correct answer on an extra question (max. 2 points) |
Your model answer in an oral exam may consist of you providing a page for each student in which some core items (see column 2 in the example) can be displayed that you expect to hear in the student's answer, possibly supplemented by some more overall skills that you will take into account (think of an answer’s originality, fluency in speaking about the content, defending a position, etc.).On this page, you can take notes during the exam and assign a provisional grade (possibly by adding an extra,blank column in which you can take notes).
Points are granted based on the correct components mentioned by the student.Note here as well that a student who only knows the theory can only get half of the points.
Another option is to work with a rubric (see category ‘papers and assignments’).
Papers and assignments
In a paper or (artistic and freely selected) assignment, it is difficult to formulate a conclusive answer in advance.However, it is also important here to point out the criteria by which you will assess the student.In this case, you probably will have to work with some broader assessment criteria, which have been based on the course’s learning goals.These criteria can be completed with the implementation of levels or standards (Van Berkel et al, 2014), which indicate the number of ways in which the criterion can be fulfilled (e.g. when a paper is good, very good or inadequate or when these have not been met, etc.) and how you will assess the overall performance of the students. This combination of assessment criteria and levels/standards are shown in rubrics.Rubrics can also be used to evaluate a bachelor's or master's thesis.
While assessing,you judge how well a student has met a certain criterion (or set of criteria). In other words, you weigh which (qualitatively defined) standard best matches the student’s actual performance and hereon based you grant a grade.
| Criteria | A (Excellent) | B (Very good) | C (Fair) | D (Poor) |
|---|---|---|---|---|
| Depth and focus | Responds to prompt with appropriate depth and focus | Appropriate focus, although could be in more depth | Some attempt to focus | Not at all focused and/or very superficial; may not follow prompt given |
| Structure | Clear introduction, smooth transitions between topics, and thoughtful conclusion | Introduction, transitions, and conclusion present, but could be clearer or smoother | Evident which topics are being discussed, but no introduction, conclusion of transitions | Unclear which topics are being discussed and when; transitions non-existent |
| Content and application of concepts | Concepts correctly interpreted; own applications given for each concept discussed; applications are reasonable | Concepts correctly interpreted; own applications given but may be unreasonable | Some concepts interpreted incorrectly; few applications given or applications are ill-explained | Most concepts interpreted incorrectly; no applications given |
| Sentence structure | Sentences well-formed and appropriately varied in length and style | Most sentences well-formed, with occasional awkwardness | Some sentences poorly constructed but generally understandable | Many sentences poorly constructed, incomplete, and/or awkward |
| Mechanics | Few if any spelling or grammatical errors | Some spelling and grammatical errors, but paper is still understandable | Some spelling and grammatical errors, making paper difficult to understand in places | Many spelling and grammatical errors, which present significant barrier to understanding |
Bron: University of Waterloo
Figure 3 shows a rubricwhich contains five assessment criteria (left column) and four different levels at which these can be realised.The example does not specify how the criteria relate to one another. For example, does content carry more points than sentence structure does?It’s best to also fix these aspectsquite in advance so that you will know how criteria are related to one another or what exactly is decisive for achieving a grade.
In Figure 4 scores are matched with a given level andeach criteria is divided in different items.
Figure 4: Rubric with scores for the levels matching the item of a criteria
Figure 5 demonstrates arubric on meta-level by giving a rubric of a rubric.
Figure 5: Rubric of a rubric
The added value of assessment and model answers
Working with a model answer and having clear assessmentcriteria has several advantages:
- By thinking ahead about assessmentcriteria, and by using them during the assessment, you make yourassessment process more transparent and explicit as a teacher.Colleagues within your team or faculty have a better insight into your ways of assessment and establishing your grades.
- Criteriaensuretransparency by making clear, in advance, the aspects upon which your students will be assessed (Sadler,2009) so students know what is important in learning and taking an exam/assignment.When your assessment criteria (e.g. for a paper or master’s thesis) are communicated to students in advance, what is expected of them will be clearer, which in itself has an impact on their study process.In this way, the criteria are also able to provide the student feedback in advance (‘feed forward’).Note, transparency about the assessment does not immediately result in better scores for students.There is a difference between being aware of the important substantive criteria and also taking them into account at every stage when making your exam.
- Reviewers using criteria ensure greater reliability in their assessment, increasing the chance of delivering afairer and more consistentassessmentprocess.
- If several evaluators are involved, this can bring them more into line with each other (possibly after meetings based on the criteria). Thus, with a model answer you can avoid differences in terms of severity,e.g. where one assessor judges an answer as correct only if the student employs a term literally while another evaluator assigns the full score simply in the description of the term.Or, you avoid a situation in which assessors opt for a defensive strategy in their assessments (giving a score which is neither too high nor too low, so that they do not stand out).By thinking together, beforehand, about which elements should be taken into account (preferably even putting these to the test ahead of time) a consensus on the assessment criteria can be arrived at and all the assessments can be carried outfrom the same point of view.
- As an individual assessor you can create more consistency over time, i.e. by assessing all exams in a similar way, on the basis of the same criteria.
- Criteria provide avalid assessmentwhen they coincide with the course’s learning goals and can, therefore, also be an instrument in developing a new assessment, assignment, …
- Criteria also make itfaster and easierto provide feedbackto students and to refer to the learning goals.During an after-exam discussion, you can clarify by referring to the model answer how assessment was undertaken specifically. For students, this enhances the feeling that they are being fairly treated.
- Novice teachers or inexperienced assessors (e.g. doctoral students who are engaged in the assessment) feel more confident with guidelines as asupport.
Bibliography
- Teaching with rubrics: The Good, the Bad and the UglyCollege Teaching, 53(1), 27- 30. See more at:http://www.facultyfocus.com/articles/educational-assessment/rubrics-the-essentials/#sthash.j3KM887d.dpuf
- Bloxham, S. & Boyd, P. (2007)Developing Effective Assessment in Education Higher: A practical guide. Open University Press.
- Clement, M. & Laga, E. (2006).Doceerpraktijk index cards. Guarantee
- CitoTestguide: A guide to key manufacturers with information about the eight steps of the test construction processto lookthrough. Http://toetswijzer.kennisnet.nl/html/toetsconstructie/home.htm
- Dochy, F., Heylen, L. & Van Laer Moss, H. (2002).Assessment in education.Utrecht: Lemma.
- Dunn, L. et al. (2004).The Student Assessment handbook. New direcMtions in traditional and online assessment London:Routledge Falmer.
- Teelen Knowledge (2004).Test development in practice: how do I make good questions and buttons.
- Van Berkel, H., Bax, A. & Joosten-ten Brinke, D. (Eds.) (2014).Keys in Higher Education. Wood: Bohn Stafleu of Loghum.
- Sadler, DR (2009). Indeterminacy in the use of preset criteria for assessment and grading.Assessment & Evaluation in Higher Education, 34(2), 159-179.
