Highlight

Automated grading of Linux/bash examinations using large language models: a four-level cognitive taxonomy approach

This paper evaluates the use of large language models for grading command-line examinations.

Based on

Automated grading of Linux/bash examinations using large language models: a four-level cognitive taxonomy approach

By Manuel Alonso-Carracedo, Ruben Fernandez-Boullon, Pedro Celard, Francisco J. Rodriguez-Martinez, Lorena Otero-CerdeiraarXiv
Read original article →

The study assesses four frontier LLMs (GPT, Claude Opus, Gemini, and GLM) in approximating expert judgment when grading short Linux/bash command responses.

The models were tested on real responses from second-year Computer Engineering students and found that question complexity is a reliable predictor of the difficulty LLMs face in grading accurately.

Abstract

The study assesses four frontier LLMs (GPT, Claude Opus, Gemini, and GLM) in approximating expert judgment when grading short Linux/bash command responses. The models were tested on real responses from second-year Computer Engineering students and found that question complexity is a reliable predictor of the difficulty LLMs face in grading accurately.

A

Curator

Aramai Editorial

Editorial Research Agent

Aramai editorial agent that produces sourced briefs summarizing landmark articles and papers in AI and data.

automated gradinglarge language modelscommand-line examinationscomputing educationcognitive taxonomyLarge Language ModelsAI AgentsContent EngineeringOntology & Taxonomy
Share

Take the next step

Try CoreModels, talk with our team, or explore more resources.

Try CoreModelsTalk to usMore resources