AI Grading: score open-ended questions on submission, without the grading queue (finally!)

Open-ended questions measure whether a learner can apply what they know. AI Grading scores them against your instructions the moment a learner submits, so you can ask the harder questions without adding a grading queue.

Key takeaways

• AI Grading scores open-ended short-text and long-text questions in standard Intellum assessments against a rubric-like set of grading instructions the course author writes, at the moment of submission.
• It returns a numeric score and a written comment on every response, and finalizes assessments automatically, including ones that mix open-ended and auto-scored questions.
• It reaches the application, analysis, and explanation that multiple choice measures poorly, so certification and technical-skill programs no longer wait on a grader.
• Accuracy tracks the scoring criteria: precise, prescriptive grading instructions produce precise grades.
• Turn it on in Labs and test it by adding one open-ended question to an assessment you already run.

If your learning objectives ask a learner to apply, analyze, or explain what they know, multiple choice questions (MCQs) alone are insufficient. MCQs measure recognition. They don't tell you whether or not someone can use what they learned.

Open-ended questions, on the other hand, provide much deeper insight into a learner's understanding, which is why so many instructional designers want to use them. However, grading written responses takes people, time, and money. And forget about doing it at scale. Even with your best efforts, responses sit in a queue and learners wait days for feedback.

That's why we're excited to announce AI Grading, available now to customers in Intellum Labs. It scores open-ended responses the moment a learner submits, against the rubric-style grading instructions you write, so higher-order questions are no longer the ones you cut due to lack of resources. It's one piece of a platform Intellum has rebuilt to be AI-native, not bolted on after the fact.

What is AI Grading?

AI Grading scores open-ended question types, both short-text and long-text, in any standard Intellum assessment. The course author writes grading instructions for each question, effectively creating a rubric. At submission, AI Grading compares the response to those instructions, assigns a score, and writes feedback for the learner. When every open-ended question has been graded, the assessment finalizes and the learner is notified automatically.

Intellum customers on the LMS platform can turn it on in Labs today, in the Manager section.

What can you assess with open-ended questions now?

You can assess whether a learner can use what they know, not just recognize it. Bloom's taxonomy draws the line plainly: recall sits at the bottom, and application, analysis, and explanation sit well above it. Open-ended questions are one way to reach those upper levels, and AI Grading scores them against your instructions at scale. Intellum measures applied skill in other ways too, including hands-on virtual labs with Skillable and AI roleplays and simulations with Yoodli.

Knowing something isn't the same as being able to use it in practice.

Administrators and instructional designers can now build more demanding assessments, the kind that ask a learner to explain a decision or work through a problem. You no longer need to design a program around what a grader has time to read. Every learner gets a written comment, not just a number grade, and your grading instructions run against every response the same way each time. It matters most for customer education and certification programs, where the point is proving a learner can apply what they learned, not just recognize it.

See AI Grading work: grading an open-ended question

Start with an assessment you already run, say an Emerging Tech Trends assessment whose multiple choice questions already measure recall. Adding one open-ended question is what raises the bar.

The open-ended question asks the learner to explain MCP tools, skills, and agents. The grading instructions spell out what a strong answer contains: award points for naming each element, award more for a worked example, and subtract points for the conditions the author sets. Here, the author subtracts points when an answer conflates concepts that should stay distinct. The instructions are that specific on purpose: the AI grades against exactly what you write, so the criteria you set are what make the score accurate and consistent.

A learner answers the multiple choice questions, then writes an open-ended response that falls short of the grading instructions. On submission, the response goes to grading and the learner sees an awaiting-grading state. Within about thirty seconds, the learner has a score and a comment.

The learner does not pass, and the feedback says why, based on the grading instructions. If you also enable the Learner AI Assessment Review, a separate Labs feature, that explanation becomes a path forward: in a guided space, the learner works back through the graded assessment with the AI, sees where the answer missed, and brings it up to the standard, question by question. AI Grading doesn't require it; Assessment Review just makes the follow-up richer.

How do you write rubric-style grading instructions AI can follow?

The structured grading instructions are the whole job. In Intellum, you enter per question grading instructions, and AI Grading applies exactly the criteria you write, so vague instructions produce vague grades, and precise ones produce precise grades.

Be prescriptive about what a good answer contains. Award points for each part, and take points off for the mistakes you name.

Name the elements a full-credit answer must include, and assign points to each one. Design instructions that give credit to learners who can properly scaffold the key topics. Decide what earns partial credit and what earns none, and name the conditions where points come off: a wrong claim, an off-topic answer, conflating ideas that should stay distinct, or whatever else you decide should cost the learner. Keep the total aligned with the question's point value.

Which questions can AI Grading score?

AI Grading works on short-text and long-text questions inside standard assessments. An assessment can mix those with auto-scored questions like multiple choice and still finalize automatically, the same as it does today. AI Grading detects the language a learner answers in and returns feedback in that same language, so a learner can answer in a different language than the assessment and still be scored correctly.

CapabilityWhat ships nowQuestion typesShort-text and long-text (Text Short and Text Long)Assessment typeStandard assessmentsMixed assessmentsAI-scored and auto-scored questions grade and finalize together, automaticallyFeedbackA numeric score plus a written comment on every responseLanguagesResponses scored correctly across languages; AI Grading detects the answer's language and returns feedback in that same languageSpeedScored at submission, usually within about thirty seconds; larger banks of open-ended questions take a little longerLearner reviewWhere the Learner AI Assessment Review is enabled (a separate Labs feature), AI-graded results feed it so learners can review and improve them. Not required for AI Grading

How does AI Grading extend Intellum's grading pipeline?

AI Grading runs through the grading pipeline you already use. A text question inside an auto-graded assessment used to be skipped and counted for nothing. Now AI Grading compares that response to the author's instructions and scores it, so an assessment that stopped short of open-ended questions carries them through to a final score. There is no new assessment type and no separate workflow.

The grading instructions are what activates it, and it applies to newly completed assessments, so it starts working on the next submission that comes in.

How do you set up AI Grading?

Setup takes five steps:

1. Turn on AI Grading in Labs (Manager section).
2. Add a Text Short or Text Long question and set the point value above 1 (recommended).
3. Write the question's grading instructions (your rubric), and save.
4. Set Grade Type to Pass/Fail and set a passing grade.
5. Publish the assessment.

AI Grading scores and releases feedback the moment a learner submits; no one reviews or approves each grade first. Your control is up front, in the grading instructions: if you refine them later, update them, republish, and re-run the grading. You keep control of the standard.

Is AI grading accurate?

AI grading is as accurate as the instructions you write. The best public evidence comes from Coursera, which grades text submissions in its courses against instructor-written rubrics, the same rubric-based method. Coursera reported grading roughly 300,000 submissions this way in beta, with about 90% of the learners who responded satisfied with the AI feedback.

The scoring came out stricter, not softer. First-attempt pass rates ran lower under AI than under human peer review, 72% against 88%, because the AI awarded fewer perfect scores and gave zero when a response met none of the rubric criteria. Learners then took more attempts to pass. Accuracy tracks your instructions, so that is where your time goes.

What does the learner see?

The learner sees a grade and a written comment attributed to "Intellum AI" in place of a named instructor. The comment explains the score against the grading instructions, so a fail is not a dead end; it shows exactly where the answer fell short.

If your program has the Learner AI Assessment Review enabled, a separate Labs feature, the learner can open it once the assessment is graded and work through the results with the Intellum AI, question by question, strengthening any answer that fell short. Tell your learners upfront that AI grades these responses, and where Assessment Review is on, point them to it. That turns a grade into something they can learn from.

Start with one question on one assessment

The easiest way to see it is to add one open-ended question to an assessment you already run. Write your grading instructions, turn on AI Grading in Labs, and test it on that single question. From there, you're measuring what a learner can do, not just what they recognize.