A Comparative Analysis of AI Grading Tools: Efficiency, Pedagogy, and the Human-in-the-Loop <br/><br/>Abstract <br/>The integration of Artificial Intelligence (AI) into educational assessment promises to alleviate teacher workload and accelerate feedback cycles. This study conducts a comparative analysis of three prominent AI grading tools—Gradescope, CoGrader, and Pregrade—evaluating their technical approaches, pedagogical alignment, and impact on teacher agency. Through a framework examining automation level, feedback quality, and integration design, the analysis reveals distinct models: Gradescope’s AI-assisted answer grouping for scalable efficiency, CoGrader’s rubric-based essay analysis for formative feedback, and Pregrade’s automated scoring. Findings from literature and tool documentation indicate that while all tools offer significant time savings, their effectiveness and acceptance are contingent on preserving meaningful human oversight. The discussion argues that the most sustainable implementation follows a “human-in-the-loop” model, where AI handles initial processing and pattern recognition, empowering teachers to provide the nuanced, contextual feedback essential for deep learning. The paper concludes with implications for ethical tool selection and professional development to ensure these technologies augment, rather than replace, pedagogical expertise. <br/><br/>Keywords: automated grading, AI in education, formative feedback, teacher workload, comparative analysis <br/><br/>1. Introduction <br/><br/>The administrative burden of grading represents a significant and persistent challenge in education, consuming time that educators could otherwise devote to instruction, curriculum development, and direct student support (Tian et al., 2025). The integration of Artificial Intelligence (AI) into educational technology offers a potential solution, with a new generation of tools promising to streamline assessment workflows. These platforms claim not only to reduce grading time, sometimes by up to 80%, but also to provide more consistent and immediate feedback, a factor critically linked to student learning outcomes (Hattie &amp; Timperley, 2007). <br/><br/>However, the adoption of AI grading is not a simple matter of efficiency. It raises fundamental pedagogical questions about the nature of assessment, the role of feedback, and the preservation of teacher professionalism. Research indicates that while teachers value AI’s capacity for rapid feedback, they often distrust fully automated scoring and emphasize the necessity of human oversight (Selvam &amp; Vallejo, 2025). Furthermore, the effectiveness of these tools varies greatly depending on their design philosophy, from those that automate simple scoring to those that aim to support complex formative assessment. <br/><br/>This paper addresses this complex landscape by presenting a structured comparative analysis of three AI grading tools: Gradescope, CoGrader, and Pregrade. The objective is to move beyond marketing claims and critically examine how these tools’ functionalities align with pedagogical goals. The analysis is guided by a framework evaluating each tool’s (1) approach to automation and human oversight, (2) quality and pedagogical design of feedback, and (3) integration into existing teaching workflows. This work contributes to the ongoing conference dialogue by providing educators, administrators, and developers with evidence-based insights for selecting and implementing AI grading tools that truly enhance, rather than undermine, effective teaching and learning. <br/><br/>2. Methodology <br/><br/>This study employs a comparative case study methodology, analyzing three AI grading tools as distinct cases within the broader phenomenon of automated assessment. The primary units of analysis are the platforms’ designed functionalities, features, and stated pedagogical alignments as presented in their official documentation, published reviews, and related academic literature. <br/><br/>The three tools were selected to represent a spectrum of approaches within the AI grading landscape: <br/><br/>Gradescope: Selected for its widespread institutional adoption and unique AI-assisted answer grouping model. <br/><br/>CoGrader: Chosen for its explicit focus on rubric-based essay grading and formative feedback generation. <br/><br/>Pregrade: Included as a representative of automated scoring tools, providing a contrast in the level of proposed automation. <br/><br/>Data was gathered from official product websites and help guides as well as independent educator reviews and tool comparisons. This triangulation of sources helps mitigate vendor bias and grounds the analysis in both practical application and theoretical concern. <br/><br/>The analysis is structured using a consistent framework applied to each tool, focusing on three core dimensions derived from key themes in the literature: <br/><br/>Automation Model &amp; Teacher Agency: How does the tool integrate AI? Does it position the teacher as a final reviewer or delegate scoring authority? <br/><br/>Feedback Philosophy &amp; Pedagogical Alignment: What type of feedback does the tool generate (e.g., numeric score, grouped comments, personalized narrative)? Is it designed for efficiency, formative growth, or both? <br/><br/>Workflow Integration &amp; Data Use: How does the tool connect with existing Learning Management Systems (LMS)? What claims are made about data privacy and the use of student work for AI training? <br/><br/>This qualitative analysis aims to synthesize patterns, contrasts, and implications, providing a nuanced understanding of how different technological designs embody different assumptions about teaching and learning. <br/><br/>3. Results and Discussion <br/><br/>The comparative analysis reveals three distinct paradigms for AI integration in grading, each with specific strengths, limitations, and pedagogical implications. <br/><br/>3.1 Tool Comparison: Paradigms of AI Assistance <br/><br/>Table 1: Comparative Analysis of AI Grading Tools <br/><br/>Feature / Tool <br/><br/>Gradescope <br/><br/>CoGrader <br/><br/>Pregrade <br/><br/>Core AI Model <br/><br/>AI-assisted answer grouping for batch grading. <br/><br/>Rubric-based analysis for automated essay scoring &amp; feedback. <br/><br/>Automated scoring and feedback generation. <br/><br/>Primary Use Case <br/><br/>Scaling grading for large classes; diverse formats (bubble sheets, handwritten math, code). <br/><br/>Providing detailed, formative feedback on written assignments and essays. <br/><br/>Automated assessment of student responses. <br/><br/>Teacher's Role <br/><br/>Manager &amp; Reviewer: Manages AI-grouped answers, grades by group, provides overarching group feedback. <br/><br/>Editor &amp; Final Authority: Reviews, adjusts, and approves all AI-generated scores and comments before release. <br/><br/>Overseer: Relies on automated output, with presumably limited intervention. <br/><br/>Feedback Type <br/><br/>Efficient, consistent application of rubric points; group-level comments. <br/><br/>Detailed, criterion-based "glows and grows" narrative feedback tailored to a rubric. <br/><br/>Automated scores and comments. <br/><br/>Key Strength <br/><br/>Unmatched efficiency for objective or semi-objective questions in high-volume settings. <br/><br/>High-quality, timely formative feedback that teachers can personalize, supporting writing development. <br/><br/>High degree of automation for standardized responses. <br/><br/>Key Limitation <br/><br/>Less effective for highly unique, open-ended responses; feedback is less personalized. <br/><br/>Primarily focused on text-based essays and written responses. <br/><br/>Potential misalignment with nuanced pedagogical goals; risks bypassing teacher judgment. <br/><br/>3.2 Discussion: Balancing Efficiency, Pedagogy, and Trust <br/><br/>The findings highlight a central tension in AI grading: the trade-off between scalable efficiency and pedagogically meaningful, personalized feedback. Gradescope excels at the former, using AI to restructure the grading workflow itself. Its answer-grouping model is a powerful tool for consistency and speed, fundamentally changing the task from grading student-by-student to grading answer-by-answer. However, this comes at the potential cost of individualized attention, as feedback is applied to groups. <br/><br/>CoGrader, in contrast, is designed around the feedback loop. By generating specific narrative feedback against a customizable or standards-aligned rubric, it aims to provide the timely, detailed commentary that research shows is essential for learning but often logistically difficult for teachers to provide at scale. Its design philosophy explicitly keeps the teacher "in the loop" as the final authority, a feature that directly addresses educator concerns about losing agency and oversight. <br/><br/>Pregrade, representing a more fully automated model, theoretically offers the highest efficiency gain. However, academic literature suggests this model is the most problematic. Studies find that teachers deeply distrust automated scoring and that students are skeptical of feedback from AI alone. Furthermore, pedagogical research strongly advocates for separating detailed formative feedback from evaluative grades to promote a growth mindset. A tool that primarily outputs a score may inadvertently undermine this principle. <br/><br/>Therefore, the most significant differentiator among tools is not their technical sophistication, but how they architect the relationship between the teacher and the algorithm. Tools like CoGrader that adopt a "human-in-the-loop" model, where AI acts as a first-pass assistant whose work is always reviewed, align more closely with both teacher preferences and sound pedagogical practice. This model leverages AI to eliminate the drudgery of initial scoring and drafting feed...

by Eunice Tang and Venus Kam

Differentiated instruction starts with instructors who ‘mark/identify in both students and possible teaching strategies and make adjustments according to what will benefit students most and best facilitate learning in the classroom’ (Blaz, 2016).  

Before this term even emerged in our head, we were discussing our students in different course sections with contrastive learning styles and trying to put our heads together about what we can do more for quiet sections. Were they introverts? Did they prefer listening to speaking? And we even wondered if they had a lack of interest in learning? Should we give them more time to think? Should we let students contribute to the class in different media like polling and writing – not just speaking? Should we add some warm-up questions before the first questions in the lesson materials? 

This was our initial stage trying to identify the needs and possible teaching strategies for our students in different sections. As can be seen in the questions above, initially we considered students’: 

• Possibility of having different personalities; 

• Different learning preferences or forms of presentation; 

• Motivation for learning; 

• Readiness to contribute their ideas; 

• Reactions to interactive and digital technology; 

• Assistance needed for understanding the initial questions in the materials – and so on.   

The process does not stop here, and neither does Blaz (2016)’s definition: ‘they then develop and implement, bit by bit, the characteristics of a differentiated classroom’. This is followed by a stage of ‘development’ - ‘assessment, evaluation and reflection are the keys to finding what works and what doesn’t work, and trying to fix the latter’. 

Venus and I will share with you in the next article more about our application of these scholarship-informed principles in our classrooms.  

Reference:  

Blaz, D. (2016) Differentiated instruction: a guide for world language teachers. New York: Routledge   

Short Descriptions

The main focus of this study is investigating introvert and extrovert university students’ perception of the use of interactive digital tools (such as Padlet and Mentimeter) in a face-to-face English for Specific Purposes (ESP) class after all classes in the university had been switched to online mode for three semesters. The pandemic has given educators various opportunities to use interactive digital tools in class, especially in an online environment. It is interesting for educators to explore the potential of such tools when classes are back face-to-face. This research thus offers the students’ perspective to using interactive digital tools in a face-to-face classroom. While a lot have been said about introvert students responding positively to digital learning online, the students’ perception of their own personality collected in the survey and the impact digital tools have on their contribution to class may shed some light about the potential of interactive digital tools in a post-pandemic era.

Possible Benefits

Psychology for learning and teaching is one of the areas that has been less talked about at the CLE but is an area of interest I discovered earlier in this semester. This study will be presented in a conference that is one of the less common conferences dedicated to the psychology of language learning and teaching. While this study is based on a reflection on the use of interactive digital tools in my own classrooms, it is interesting to hear the students’ voice in relation to the psychological aspects. In a so-called ‘post-pandemic’ era, the discussion of whether we should keep the practice of using interactive digital tools in class and how it affects student with different personalities to learn is definitely worth discussions in the CLE.

Deliverables

Presentation at the International Conference on Psychology of Language and Language Learning in July 2022

Study Focus

This study presented at the Psychology of Language and Language Learning on July 28, 2022 in London was to investigate introvert and extrovert university students’ perception of the use of interactive digital tools (such as Padlet and Mentimeter) in a face-to-face English for Specific Purposes (ESP) class after all classes in the university had been switched to online mode for three semesters.

Subjects and Methods

The subjects of the study were business students in LABU2040. The basic tool for data collection was an anonymous online survey, which included 3 required multiple-choice questions and 3 open questions (2 required; 1 optional) about the effects of interactive digital tools on their amount of contribution to the class discussions, their perception of the role of interactive digital tools to the sharing of ideas and whether the students considered themselves introvert or extrovert. The survey results were then analyzed qualitatively, particularly on the effect the use of interactive digital tools had on the amount of contribution to the class among introvert and extrovert students, their perception of a language class with and without digital tools and most importantly, the implication to educators about how interactive digital tools can be used (or not) to cater for the needs of the introvert and extrovert students.

Result highlights

•The use of interactive, digital tools resulted in an increase in the amount of contribution from students and the number of students who contributed to the class activities. They allowed anonymous responses to be given, making some students more comfortable sharing their thoughts.

•Introvert students tended to feel less pressured with the use of the digital tools and participated more in class without having to volunteer. They have expressed that this is an alternative that let them become more confident and ready.

•Extrovert students also said that the tool let everyone participate in class, even for shy people or when they are tired. They pointed out that the digital tools enable the ideas to be visualized and retrievable after class.

The pandemic has given educators various opportunities to use interactive digital tools in class, especially in an online environment. It is interesting for educators to explore the potential of such tools when classes are back face-to-face. This research thus offers the students’ perspective on using interactive digital tools in a face-to-face classroom. While a lot has been said about introverted students responding positively to digital learning online, the student's perception of their own personality collected in the survey and the digital impact tools have on their contribution to class may shed some light on the potential of interactive digital tools in a post-pandemic era.