If students generate feedback by making comparisons against comments,

why should comments be the only comparator? [David Nicol, 2020]

This website and the arguments herein do not just represent an addition to the literature on feedback, some ideas about how to make current feedback practices more effective. Rather the ideas herein are a challenge to the notion of feedback itself as currently conceptualised. 

This website re-examines feedback through the lens of comparison. It starts from the premise that students learn by comparing their work against both their inner prior knowledge and against different sources of external information and that they generate inner feedback out of those comparisons. Based on this, it investigates what students learn when they compare their work against information, other than teacher comments, such as information in journal articles, in diagrams, in the work of peers, in rubrics, exemplars, videos and other resources online and offline. They might also compare against information derived from observations of others or events. I use the word comparison deliberately rather than evaluative judgement as all judgements at core involve a comparison, and research on evalutive judgement tends to assume the the comparators are either, rubrics or sssessment criteria and/or exemplars. I argue that anything that would improve students ability to think critically might serve as a comparator. The word comparison also begs the question 'what is being evaluated against what?' whereas evaluative judgement does not always surface this consideration. Furthermore when people analyse, problem-solve, make decisions, self-assess or reflect a comparator is always implicated. Internal emotional reactions might also serve as comparison information influencing what students produce or what feedback they generate.

Initial evidence across a range of disciplines shows that when students make comparisons of their work against different kinds of information, they generate more varied feedback than from comments alone, as each comparator takes students to a different place. They see their work through different lenses. Also inner feedback builds and becomes more elaborate from one comparison to the next, and multiple comparisons lead to feedback that surpasses what students generate from teacher comments. 

This idea is easy to implement as it builds on a natural process - the making of feedback comparisons - something that we are all doing all the time anyway, even without receiving comments. These ideas have been researched at the University of Glasgow, University of Manchester, University of Padova, Italy and have been piloted across a range of disciplines at Trinity College Dublin and in Medicine at the University of Edinburgh and University of Utrecht. This new lens on feedback has not to date been the subject of systematic research so there is a lot to learn.

What is clear however from current research is that the key to unlocking the power of inner feedback is to have students mindfully make comparisons of their work against different kinds of information and to make their learning from these comparisons explicit through writing self-feedback, through discussion with peers or through actions to improve their work.  Writing their own self-feedback comments is especially powerful as it makes visible to students their own feedback capability which is empowering. This approach:

  1. Acknowledges that students are the architects of their own feedback while recognising that learning from feedback can be improved by approrpriate learning designs that scaffold students' feedback productions.
  2. Helps build students capacity to regulate their own learning, as inner feedback is the catalyst for the self-regulation of all learning.
  3. Improves students' attainment and the range of feedback students generate without placing excessive burden on the educator as comparison opportunities can be accessed in any claaroom situation and comparison resources are reusalbe
  4. Gives teachers better information about what feedback students are generating and hence about what comments they really need.
This lens of comparison changes everything. It calls for a fundamental change in feedback practice and research. The intial reference for this website is the article "The Power of Internal Feedback: Exploiting Natural Comparison Processes" Nicol (2020). This sets out the theoretical framing and provides a model and practice ideas. However, more recent articles take things forward and some of them and presentations of newer ideas can be accessed below. 
 

RESOURCES

All new resources (articles, videos and tools) as they become available can be accessed below for those returning to this site for new developments:
 
Two minute Video - part of submission for Reimagine Education Award 2020 - for those who are time starved - is
 
A 15 VIDEO with of an INTERVIEW with DAVID NICOL produced for the Re-imagine Education International Conference where this work won the Siliver Award for Innovation in the Science of Learning can be found .
 
And a SHORT 900 WORD ARTICLE SUMMARISING THE MAIN IDEAS but with a specific focus about how to implement them can be found here.
 
June 2021 Two new articles added on inner feedback see Nicol and McCallum (2021) here and Nicol and Selvaretnam (2021) here. 

And conference presentation by Geetha Selvaretnam on appliations of inner feedback model in two stage exam setting is here.

Nicol, D., N. Quinn,, L. Kushaw, and H. Mullen. 2021."Helping Learners Activate Productive Feedback; Using Resource and Dialogic Comparisons, Presentation  at the Chartered Association of Business Schools (CABS): Teaching, Learning & Student Experience Conference, 29-30th June. Online. Paper HERE. Uploaded November 2021.

David Nicol and Suzanne McCallum. 2021 “The Power of inner Feedback: Comparison changes everything. Presentation at Student Experience Expert Group Meeting 4 November for JISC UK. Slides and link to JISC event page 

November 18th 2021: Keynote by David Nicol with input from colleagues Suzanne McCallum and Nick Quinn for SLO: National Institute for Curriculum Development in the Netherlands. Title of keynote "Improving Learning by building on pupils' natural capacity to generate inner feedback". Description can be found here and full video presentation here. 

2022

26 January 2022: Presentation ‘Generating inner feedback from self and peer review’ by Dr Maxine Swingler and Dr Lorna Morrow, School of Psychology and Neurosciences, & Professor David Nicol, Adam Smith Business School, University of Glasgow to TILE (Teaching Innovation and Learning Enhancement) network.  Link to abstract and recording HERE

29 March 2022: Presentation on ‘Active Feedback’ by Willie McGuire (School of Education), David Nicol (Adam Smith Business School) and Gemma Haywood (School of Education), at the Learning and Teaching Conference, University of Glasgow.  Link to recording HERE

1 June 2022: Podcast produced by Sarah Knight and Mark Lennon of Jisc UK as part of their updated Guidance and New Principles of Assessment and Feedback in March 2022. David Nicol starts by taking through the thinking behind the research undertaken into feedback in the Adam Smith Business School. Suzanne McCallum (Accounting and Finance), Dr Lovleen Kushwah (Economics) and Dr Nick Quinn (Entrepreneurship) then discuss the methods they are using to bring this new feedback approach to life. They disucss how this approch benefits students’ learning and attainment and they also discuss how students respond to these methods. Podcast can be accessed HERE

28 June 2022: Centre for Teaching and Learning in Economics: University of London  Conference: TeachECONference2022 Day 2 Session 3 “New Ideas in Assessment and Feedback”. Presentation ‘Thesis Supervision: Improving Student feedback by Harnessing Resource Comparisons". An economics example’ by Dr Lovleen Kushwah and David Nicol, Adam Smith Business School.  Recording HERE [Note: the presentation starts after approx. 21 mins and questions occur from 33 mins to 41 mins approx.] 

***NOTICE PRACTICAL RESOURCE FOR INSTRUCTORS 2 JUNE 2022***

Below is the First Practical Guide for Instructors (Teachers, Lecturers, Faculty) who want to implement comparison-based feedback. This Guide is unique. It differs from any other guidance about feedback published anywhere in the world. It is also short with 10 examples of implementation.  

Nicol, D. 2022. “Turning Active Learning into Active Feedback”, Introductory Guide from Active Feedback Toolkit, Adam Smith Business School, University of Glasgow AVAILABLE   HERE   

2023

Rose, J. 2023. “ChatGPT as a teaching tool, not a cheating tool” Times Higher Education

Quinn, N. and Gibb, A. 2023. “The power of active feedback to prepare students for professional experience” Times Higher Education.

Times Higher Education Podcast: "How to use generative AI in your teaching and research". Jenni Rose and David Nicol discuss generatvie AI in teaching and learning (first 40 mins) and Brook Szucs duscusses the research aspect

Jenni Rose 2023 provides an excellent short 4 minute explanation of the inner/active feedback approach with a helpful infographic. She also explains how she used this approach with over 600 business school students. 

Nicol, D. and McCallum, S. Keynote presentation at the Feedback Fruits inspirED conference in Amsterdam in June 2023.  "Using active feedback to develop students’ critical thinking". This recording is 1 hr 20 mins but it was an interactive session with Q&A throughout. Useful if you want latest thinking.  

PUBLICATIONS 

David Nicol & Lovleen Kushwah 2023. "Shifting feedback agency to students by having them write their own feedback comments" Assessment & Evaluation in Higher Education, The article is open access. It challenges current thinking with regards to how to develop students' feedback literacy. 

Nicol, D. 2014.  An older article on peer review that is relevant but not widely known about.  Guiding Principles for Peer Review: Unlocking learners' evaluative skills".  Proposes eight principles for the design of peer review and some practical suggestiions about how they might be implemented.  It can be found HERE  

 

FURTHER DETAIL (click section headers to expand)

This website presents a rethinking of the theory and practice of feedback in education. It identifies comparison as the fundamental cognitive process underpinning feedback processes and it identifies students as feedback generators. It does not however propose a narrow cogntive perspective on feedback processes . Rather, the proposal is that feedback thinking should take into account how the socio-material context shapes feedback processes and the learning that results. From that perspective the position in this website aligns with recent thinking by researchers (e.g. Gravett, 2020; Allal, 2020; Chong, 2020; van der Leeuw et al, date; Fawns and O'Shea, 2019) who  propose that we adopt an ecological and contextual framing of feedback.  What I add however to these largely theoretical framings is a practice perspective, an implementation methodology, that can be applied across all disciplines. Indeed, a key goal of this website is to bring together ideas for practice, both those that can be implemented in a single lecture session to those that might span the timeline of a course or programme of study.  The basic argument at the core of this new feedback thinking is simple.
All feedback rests on comparison processes
When teachers give feedback comments, students must compare the information in those comments against their own work and generate internal feedback out of that comparison. If they do not compare, there is no feedback. But student, like all of us, are making feedback comparisons all the time. "Musicians learn to become better musicians by comparing their compositions and performances against those of other musicians and by generating inner feedback out of those comparisons. Scientists become better scientists by comparing their thoeories methods and findings against those of other scientists.
Students produce valuable feedback
In academic settings, students naturally make comparisons of their work against different information sources and generate their own inner feedback out of those comparisons. For example when completing a homework assignment they might compare their unfolding work against the task instrucitons, the assessment criteria, what is said in a lecture, some information in an online resource, a youtube video, the work of their peers or information derived from discussions with others, parents, teachers or peers. Each of these sources provide information for comparison and for the generation of inner feedback. Sometimes, usually when their work is complete, students receive information as comments from a teacher which they will also use for comparison. But that information is only a drop in the ocean of the information that students use for comparison and to generate inner feedback (see Yan, 2020, for a supporting argument).
Making the comparison process deliberate and its outputs explicit
The key to unlocking the power of internal feedback is for teachers to have their students make mindful comparisons of their work against information in different kinds of resources, singly and in combination, and to make explicit the learning that results from those comparisons. For example, students might write their own self-feedback comments, or discuss the feedback they generate with peers or update their work or apply the feedback in the next related task. One design strategy is to have students make resource comparisons individually first and generate their own feedback, then have them discuss this with peers with the ouput being feedback questions for the teacher.  Teachers then see what feedback students can generate by themselves and are able to provide only the comments that are needed.  In this framing, teachers should always endload their comments after other comparisons. This methodology is simple to implement as it builds on natural processes that we are all engaging in anyway.