Every week a new AI marking tool lands in a school leader's inbox. Each one promises huge time savings.
Done well, it delivers. Done poorly, it creates more work, because teachers spend longer checking and correcting the AI than they would have spent marking by hand. Worse, students receive marks nobody in the school can explain.
These five questions will help guide you to the right product for your school.
1. How has accuracy been measured and against what?
“Highly accurate” means nothing until you know how the company reached that claim. Ask how they define accuracy, how they achieve it and what framework they use to measure it. Then ask how they moderate their marking and keep it reliable over time and across different subjects and levels.
What a good answer sounds like: published results against moderated examiner marks and a clear tolerance.
2. What happens when the AI is unsure?
Every marker gets some answers wrong, human or AI. What matters is whether the system knows when it might be wrong. Ask how it flags uncertain answers, what share get flagged and who reviews them.
What a good answer sounds like: a clear rule for when work is flagged, a typical flag rate and how many errors remain after review.
3. Where does the teacher stay in control?
AI marking should support professional judgement, not replace it. Teachers should be able to see why each mark was given and change any mark or any piece of feedback before students see it. The final decision always sits with the teacher.
What a good answer sounds like: visible reasoning for every mark and full teacher control over marks and feedback.
4. What happens to student data?
Scanned scripts contain names, handwriting and sometimes personal reflections. Ask where the data is stored, how it is processed and how the supplier stays compliant with UK GDPR. Ask what happens to the data when your contract ends and whether they will support your data protection impact assessment.
The Department for Education’s Generative AI: product safety standards give leaders a useful benchmark to hold any tool against.
What a good answer sounds like: a clear account of where data is stored, how it is processed and what happens to it when your contract ends.
5. Has anyone independent looked at the evidence?
A supplier’s own testing is a starting point. Ask whether anyone independent, such as a university, research organisation or sector body, has reviewed the evidence. If they have, ask to read what they found.
Independent review matters because it tests a product’s claims against an agreed standard rather than the supplier’s own.
Then try it yourself. A short trial with one department can help you see how the tool performs with your students.
What a good answer sounds like: an independent review you can read, published data you can check and a trial with one of your departments.
Putting these questions to Running Paper
None of these questions needs technical expertise, just a leader willing to ask them. Any supplier worth choosing should welcome them, so it is only fair that we answer them ourselves.
Running Paper is the AI marking platform for schools. We mark formative and summative pen and paper assessments from Key Stage 3 to Key Stage 5, from end of unit tests to mocks, across more than 15 subjects including Maths, English and Science.
We benchmark accuracy against scripts marked independently by former exam board examiners and experienced subject leads, with any disagreements moderated to an agreed mark. Every answer is then marked by two independent AI models, a primary marker and a validator that never sees the first mark, its reasoning or its confidence. When they disagree by more than the tolerance set for that question, the answer is flagged for a teacher. Our accuracy page publishes our benchmark and simulation results, including the errors expected to remain after flagged questions are reviewed.
Nothing reaches a student until a teacher has checked it and sent it. AI handles the mechanical part of marking, while the judgement stays with the teacher. We are registered with the Information Commissioner’s Office and hold Cyber Essentials and Cyber Essentials Plus certification.
Running Paper was independently evaluated as part of the EdTech Evidence Board pilot, delivered by the Chartered College of Teaching and funded by the Department for Education. You can read the full report here.
If you are at the Schools & Academies Show Birmingham, bring these questions to Stand K22 and put them to us. You can also find out more at runningpaper.co.uk.