ASSESSING How We ASSESS Learners (Part One)
Tagg, 2003 One of the things I am often asked on my travels around Turkey is:
What can we do to improve the way we assess student learning? I love this question – it shows an “improvement-orientated ethos” and a recognition of just how important assessment is to “learning”. After all, and as Lauren Resnik so powerfully put it, “What you assess is what you get; if you don't test it you won't get it.” BUT, I usually answer the question with a question (yes, my “big little girl” always hated this habit of her daddy):
What principles are your assessment systems and tools built on? I often get a lot of “blank” looks. Many respond by talking about reliability, validity or “professional levels of quality”. To be sure, these are important notions – but my question is digging a little deeper. As teachers, educators and assessment experts we all discuss “quality issues” in assessment but we forget one thing: The REAL assessment “experts” are STUDENTS!
what they define as important;
when they tune in and tune out during formal learning opportunities;
what they study in their own time and how much they engage with their subject matter; and,
how they come to see themselves as students, graduates and people in later life In a nutshell, and to borrow the words of John Cowen, “assessment is the engine which drives student learning” – and it determines not only what students learn but how they go about learning it.
If assessment is so important, why are we more interested in issues of reliability and validity than say the guiding principles that help us imagineer the type of assessment systems, protocols and tools that can really make a difference to the way our students learn? The fundamental challenge for our schools, colleges and universities is to stimulate the “right kind of learning” – this challenge brings together our beliefs and assumptions, our purposes and missions, the way we plan - and also highlights the importance of “walking one’s talk”. Educators have little difficulty in appreciating the power principles in education - guiding and informing practice across the wide spectrum of learning and teaching activities we carry out on a day-to-day basis. However, as we scan the websites and documentation of our institutions we discover that many schools, colleges and universities do not explicitly state their values and principles on learning – let alone assessment. Of those that do, sadly, many fail to live in alignment with their espoused principles. But, there is hope! Nichols tells us that for principles to be effective, they should capture a “guiding light” (preferably linking explicit assumptions and beliefs to research evidence) and they should be “doable” (so as to guide practitioners towards effective implementation). So, what are some of the principles that could perhaps act as an effective guiding light? As far back as 1991 (that’s 20 years ago), Chickering & Gamson summarised the “secrets” of good practice in education:
Encourages contacts between students and faculty,
Develops reciprocity and cooperation among students,
Uses active learning techniques
Gives prompt feedback,
Emphasizes time on task,
Communicates high expectations
Respects diverse talents and ways of learning. In 1992, the American Association for Higher Education (AAHE) also developed a set of Principles of Good Practice for Assessing Student Learning. This was heavily influenced by the ideas developed by Alverno College in the 1970s. The principles developed by AAHE made deeper and more explicit connections between values, the type of learning envisioned and guidance to practitioners as to how to make this happen. The set of nine principles started at the level of values and sought to link the choices higher education institutions make about assessment to the values that these institutions should subscribe to – they are relevant to primary and secondary education as much as they are to tertiary level learning and… They are classic bedtime (or snowtime) reading: 1) The assessment of student learning begins with educational values. Assessment is not an end in itself but a vehicle for educational improvement. Its effective practice, then, begins with and enacts a vision of the kinds of learning we most value for students and strive to help them achieve. Educational values should drive not only what we choose to assess but also how we do so. Where questions about educational mission and values are skipped over, assessment threatens to be an exercise in measuring what's easy, rather than a process of improving what we really care about. 2) Assessment is most effective when it reflects an understanding of learning as multidimensional, integrated, and revealed in performance over time. Learning is a complex process. It entails not only what students know but what they can do with what they know; it involves not only knowledge and abilities but values, attitudes, and habits of mind that affect both academic success and performance beyond the classroom. Assessment should reflect these understandings by employing a diverse array of methods, including those that call for actual performance, using them over time so as to reveal change, growth, and increasing degrees of integration. Such an approach aims for a more complete and accurate picture of learning, and therefore firmer bases for improving our students' educational experience. 3) Assessment works best when the programs it seeks to improve have clear, explicitly stated purposes. Assessment is a goal-oriented process. It entails comparing educational performance with educational purposes and expectations – those derived from the institution's mission, from faculty intentions in program and course design, and from knowledge of students' own goals. Where program purposes lack specificity or agreement, assessment as a process pushes a campus toward clarity about where to aim and what standards to apply; assessment also prompts attention to where and how program goals will be taught and learned. Clear, shared, implementable goals are the cornerstone for assessment that is focused and useful. 4) Assessment requires attention to outcomes but also and equally to the experiences that lead to those outcomes. Information about outcomes is of high importance; where students "end up" matters greatly. But to improve outcomes, we need to know about student experience along the way -- about the curricula, teaching, and kind of student effort that lead to particular outcomes. Assessment can help us understand which students learn best under what conditions; with such knowledge comes the capacity to improve the whole of their learning. 5) Assessment works best when it is ongoing not episodic. Assessment is a process whose power is cumulative. Though isolated, "one-shot" assessment can be better than none, improvement is best fostered when assessment entails a linked series of activities undertaken over time. This may mean tracking the process of individual students, or of cohorts of students; it may mean collecting the same examples of student performance or using the same instrument semester after semester. The point is to monitor progress toward intended goals in a spirit of continuous improvement. Along the way, the assessment process itself should be evaluated and refined in light of emerging insights. 6) Assessment fosters wider improvement when representatives from across the educational community are involved. Student learning is a campus-wide responsibility, and assessment is a way of enacting that responsibility. Thus, while assessment efforts may start small, the aim over time is to involve people from across the educational community. Faculty play an especially important role, but assessment's questions can't be fully addressed without participation by student-affairs educators, librarians, administrators, and students. Assessment may also involve individuals from beyond the campus (alumni/ae, trustees, employers) whose experience can enrich the sense of appropriate aims and standards for learning. Thus understood, assessment is not a task for small groups of experts but a collaborative activity; its aim is wider, better informed attention to student learning by all parties with a stake in its improvement. 7) Assessment makes a difference when it begins with issues of use and illuminates questions that people really care about. Assessment recognizes the value of information in the process of improvement. But to be useful, information must be connected to issues or questions that people really care about. This implies assessment approaches that produce evidence that relevant parties will find credible, suggestive, and applicable to decisions that need to be made. It means thinking in advance about how the information will be used, and by whom. The point of assessment is not to gather data and return "results"; it is a process that starts with the questions of decision-makers, that involves them in the gathering and interpreting of data, and that informs and helps guide continuous improvement. 8) Assessment is most likely to lead to improvement when it is part of a larger set of conditions that promote change. Assessment alone changes little. Its greatest contribution comes on campuses where the quality of teaching and learning is visibly valued and worked at. On such campuses, the push to improve educational performance is a visible and primary goal of leadership; improving the quality of undergraduate education is central to the institution's planning, budgeting, and personnel decisions. On such campuses, information about learning outcomes is seen as an integral part of decision making, and avidly sought. 9) Through assessment, educators meet responsibilities to students and to the public. There is a compelling public stake in education. As educators, we have a responsibility to the publics that support or depend on us to provide information about the ways in which our students meet goals and expectations. But that responsibility goes beyond the reporting of such information; our deeper obligation – to ourselves, our students, and society – is to improve. Those to whom educators are accountable have a corresponding obligation to support such attempts at improvement. Gibbs and Simpson highlighted the practical implications of these principles when they emphasised the conditions that best support student learning:
Assessed tasks capture sufficient student time and effort
These tasks distribute student effort evenly across topics and weeks
These tasks engage students in productive learning activity
Assessment communicates clear and high expectations to students Central to all these principles and recommendations for implementation is the notion of “feedback” – both its quantity and quality. And, the remaining conditions are:
Sufficient feedback is provided, frequently enough & in enough detail
The feedback is provided quickly enough to be useful to students And that this feedback is:
<