What would doubting Thomas do? Some thoughts on assessment

Dr Mel Duffy, Lecturer in Sociology & Sexuality Studies, School of Nursing & Human Sciences

____________________________________________________________________

Grading examinations or assignments has become problematic in recent years.  There was a time where grading was perceived as being the stepping stone enabling students to become better communicators through writing on their field of study but more importantly on a subset within that field of study.  It was not based on the banking method of learning but on the fluidity of knowledge whereby what was learned in one subset directly and indeed at times indirectly influenced the next subset. Within this system students produced work by an allotted time, it was marked and in return the student received feedback.  Upon receiving a lower grade than expected the student would have been permitted to resubmit taking into consideration the pointers that had been discussed.  This however would appear today to be the stuff of nostalgia.  No longer do academics have the time to accept re-submissions rather they are now termed as re-sits.  The language surrounding students’ ability has moved from reality to that of the rarefied world of potentiality without due regard to the authenticity of that potential in the first place.   Therein lies the problem.  Do rubrics really encourage ongoing learning in the student? Are they really a system of empowerment for the lecturer?  Or is the true reality they exist, that is rubrics, as the Berlin wall between the academy and the legal system.

If we are to assume that rubrics are aids to the development of a robust curriculum then are we suggesting that the tried and tested methods of the past produced less than robust results.   In actual fact there appears to be little questioning of the validity for the usage of rubrics particular in the constant chatter in the educational setting that increasingly has gone public that is the ‘dumbing down’ of our degrees.  If we are constrained by the rubric for assessment are we failing to recognize the creative, once in a life time brilliant mind before us as the rubric is too tight to allow for that?  We are ruled by convention, teaching students to produce in a format that sits with the ‘high impact’ output requirement of academia failing to recognize that within the journal of such ‘high standards’ rarely do more than 10 actually read the article but it makes the producer and by extension institution look good.  Marx suggests that new knowledge is the pushing of an already existing idea into a direction never thought of before.  Within this way of thinking will we the academic fail the person who does just that as it does not sit within the confines of a particular box.

What emanates from this is my real fear of failing the student, the one or ones that do not fit the mold but rather sit outside the box looking in and discovering that there is no place for their thought processes.  My concern arises from my experiences of dealing with 1st year students who struggle with the idea that we ‘kinda like to know what they think’ and then confine them in their writing by our grading tools.  Therefore I am the doubting Thomas of the rubric who may in reality only be converted upon its usages or indeed be able to adequately answer the statement posed through reflecting on their usage.

Editor’s Note: Anyone interested in the whole idea of rubrics and whether there are better alternatives might like to read this fascinating article on comparative judgement.

 

References

Panadero, E.; Tapia, J. A. & Huertas, J. A. 2012. Rubrics and Self-Assessment Scripts Effects on Self-Regulation, Learning and Self-Efficacy in Secondary Education. Learning and Individual Differences Vol. 22 pp.806–813

Rezaei, A. R. & Lovorn, M. 2010.  Reliability and Validity of Rubrics for Assessment through Writing.  Assessing Writing Vol. 15 pp.18–39

James, P. W. 1997. What’s Wrong–and What’s Right–with Rubrics. Educational Leadership, Vol. 55 No.2 p72-75

Advertisements

How to motivate students to read prior to class: RATS!

Author: Catherine Corrigan Lecturer School of Nursing and Human Sciences

________________________________________________________________

Getting students to come to class prepared can be a challenge!

One way to encourage engagement with pre-reading activities is the use of Readiness Assessment Tests (RATs). I have been using RATs over the past 6 years and can attest that this method really works, albeit attaching a grade to the activity, which is always a good motivator.

RATs are short (10 question) quizzes that test the students’ knowledge about the assigned readings1. The items can be multiple-choice questions (MCQs), open-ended items, or a combination of the two. The questions are written at the remembering and understanding levels of Bloom’s Taxonomy and are taken directly from the assigned pre-readings.

RATS

Frequent quizzes increase the likelihood that students will read prior to class1 and RATs are ideal to motivate this activity. A small percentage (2%) of the final grade can be awarded towards each RAT, meaning that a total of 10 – 12% can become part of the final evaluation, although as much as 60% has been awarded for the 10-item quizzes2.

Use RATs in class or online
RATs can be administered face-to-face or online, synchronously or asynchronously. Synchronous RAT administration requires that the instructor display each item for an allotted time (typically 1.5 minutes), with the advantage of the computer automatically calculating the score and including it in the grade centre. Asynchronous online RAT items are written at the more difficult application and analyses levels of Bloom’s Taxonomy, because the students 1) are allowed to use their resources and 2) are provided more than one attempt to take the RAT. Instructors can allow up to three attempts for an asynchronous online RAT – this process provides students the opportunity to increase their score. Most platforms such as Moodle and Blackboard can be set up to either accept the most recent score or accept the highest score, to facilitate instructor preference. The questions can include short essay questions that ask the student to explain the meaning of a topic in their own words for instance, as well as some MCQs and/or open-ended items. The student can access the assigned reading material up to three times, meaning that reviewing and revision is occurring, facilitating the development of declarative knowledge3.

Individual RATs and group RATs
When the student has completed their individual RAT (iRAT), they join a (pre-assigned) group to complete the same 10 questions in a group RAT (gRAT) in less time (1 minute per item). The gRAT provides the opportunity for students to accrue extra points; if they are successful in their iRAT, they are awarded the average of the two scores (iRAT and gRAT); however they can only get the benefit of the gRAT score if they ‘pass’ their iRAT, meaning that students cannot ‘pass’ an iRAT on the addition of the group score. If the gRAT score is less than the iRAT score, students get to keep their higher score. The objective of the gRAT is to facilitate another exposure to the material within a short space of time, as well as the learning that occurs through students’ discussing the RAT items, such as providing rationale for why they chose one answer over another. Scratch and win multiple choice cards4 (similar to lottery scratch off tickets) can be used for the gRAT, where students can get immediate feedback on which answer was ‘starred’ correct. An alternate method of review is displaying the questions for the class and discussing each question’s answer as a learning opportunity, in particular by discussing the MCQ distractors as well. Frequent assessment and feedback helps students learn1 moreover timely feedback is paramount to the learning process.

The use of resources
It is up to the discretion of the instructor whether to allow the students to use notes for their face-to-face iRAT. The objective of allowing notes during the iRAT is not to make the test easier, rather to encourage students to read and take hand-written notes (not typed) prior to coming to class. Additionally, allowing the use of notes reduces the pressure of test taking. Pre-reading, note taking, iRATs, gRATs and an in-class review follows Marzano’s recommended four exposures to new material3 to facilitate learning.

Try it!
There are lots of advantages besides encouraging pre-reading! Unannounced quizzes motivate students to read prior to all classes because the quiz scores effects their grade5. For instance, RATs can be administered at the start of class sharp (to encourage timely attendance at a face-to-face or synchronous online webinar), anytime during class (to energise a fading group of students), or on any given day of class throughout the module. Avoid letting students know what day and at what time RATs will be administered – this practice encourages students to read prior to every class, attend class and be on time.

Online RATs completed prior to class provide data for the instructor that can be helpful for class preparation with an emphasis on areas that the students are finding challenging6. Face-to-face RATs can also be used as a means of formative assessment and the instructor can use the data on challenging concepts to prepare for future classes.

References

  1. Weinstein, S. and Wu, S. 2009. Readiness assessment tests verses frequent quizzes: Student performance, International Journal of Teaching and Learning In Higher Education, 21(2), pp. 181-186.
  2. Critz, C. and Knight, D. 2013. Using the flipped classroom in graduate nursing education, Nurse Educator, 38(5), pp. 210-213.
  3. Marzano, R. 2007. Art and Science of Teaching. Virginia: ASCD.
  4. Sibley, J. 2013. Team Cohesion-Readiness Assurance Process. Available at: https://www.youtube.com/watch?v=l_Nzj-QU5Dk (Accessed: 11 April 2016).
  5. Vandsburger, E. and Duncan-Datson, R. 2011. Evaluating the study guide as a tool for increasing students’ accountability for reading the textbook. Journal of College Reading and Learning, 42(1), pp. 6-23.
  6. Heiner, C., Banet, A., and Wieman, C. 2014. Preparing students for class: How to get 80% of students reading the textbook before class. American Journal of Physics, 82(10). pp. 989- 996.