Cross-university platform portalShow a desktop notification when the AI TA finishes replying
Notify me when classmates post messages in the forum
Play an alert sound whenever there is a new notification
Explain how Uedu uses a large language model (LLM) to automatically classify cognitive levels in dialogue between Students and the AI TA, for reference when teaching researchers write papers.
In every conversation between students and AI teaching assistants (UeduGPTs) on the Uedu platform, the messages sent by students are automatically classified by Bloom's Revised Taxonomy cognitive levels. This analysis is carried out entirely on the back end and does not affect the student experience; the results are available for teachers to review on the dashboard and may also be exported for research use.
This document explains the complete methodology of this automated classification system, including the theoretical basis, technical implementation, prompt design and data quality control, to help researchers understand how the data are produced.
The classification logic described in this document is based on v1.4, and the model used is gpt-5-mini. See Section 10 for historical version changes.
This methodology has been used in two peer-reviewed papers — the journal Computers and Education: Artificial Intelligence, and the ACM Learning @ Scale 2026 conference. Full bibliography and DOI are listed in Section 9.
This system uses Bloom's Revised Taxonomy (Anderson & Krathwohl, 2001) as the cognitive-level classification framework, dividing students' cognitive operations into six levels from lowest to highest:
| Level | English | Definition | Example student messages |
|---|---|---|---|
| 1. Memory | Remember | Recall and recognise factual knowledge | 'What is photosynthesis?' |
| 2. Understanding | Understand | Explain, summarise, and infer meaning | "Why use recursion?" |
| 3. Application | Apply | Execute or use knowledge in new contexts | 'How do I use a for loop to print the multiplication table?' |
| 4. Analysis | Analyze | Break down elements and identify relationships and structure | "What is the difference between these two methods?" |
| 5. Evaluation | Evaluate | Make a judgement according to the criteria | 'Which plan is more suitable, A or B?' |
| 6. Create | Create | Integrate elements to create something new | 'Help me design an experimental plan' |
bloom_level by the main cognitive operationThe following flowchart explains the complete data flow of a student message from submission to completion of Bloom classification:
app.py writes the original message to the classroomgpt_my_log tableanalyze_bloom_async() to start a background threadclassroomgpt_bloom_analysisPrompts use the System / User split pattern (from v1.2 onwards):
Before classification, LLM will first determine whether the message is a meaningful learning message (is_meaningful). The following types are regarded as meaningless:
A message that is short but has a clear learning intention is still judged to be meaningful, for example, 'Why?' or 'What is DNA?'.
The system queries the most recent 2-3 rounds of conversation records before the Student's current message (Student questions + AI replies) as prior context, but the LLM only classifies the last Student message; the preceding context is used only to understand the situation.
For example, when a Student replies, 'Why?', the preceding context can help determine whether this is a request for a conceptual explanation (Understand) or an exploration of causal relationships (Analyze).
Student messages may include images (screenshots, photos, handwritten notes, etc.). The system encodes the images as base64 and sends them to the LLM in OpenAI's multimodal content format, so the classifier can make a judgement based on both text and image content.
LLM returns strict JSON format, which is stored in the database after system validation. Each analysis result includes the following fields:
| Field | Type | Description |
|---|---|---|
is_meaningful |
Boolean | Whether it is meaningful learning information |
bloom_level |
Enum | Primary cognitive levels: remember / understand / apply / analyse / evaluate / create (NULL for meaningless messages) |
scores |
6 Floats | The respective weight scores of the six levels, with a total = 1.0 |
evidence |
Text | The basis for the LLM's judgement (within 30 characters) |
skip_reason |
String | If is_meaningful = false, record the reason |
method |
String | Model name used (e.g. gpt-5-mini) |
prompt_version |
String | Prompt version number (e.g. v1.4) |
bloom_level represents the main cognitive level determined by the LLM (single label), while the six-dimensional scores reflect the distribution of that message across the levels. Researchers may choose either a single label (distribution statistics) or continuous scores (radar charts, trajectory analysis) as needed.
The system adopts parallel real-time path + batch path operation to ensure that every Student message can be analysed:
| Real-time path | Batch path (Batch) | |
|---|---|---|
| Trigger timing | Triggered immediately after a student message is submitted | Background daemon polls every 10 seconds |
| Implementation method | Single daemon thread | 10 parallel daemon threads (partitioned by log_id % 10) |
| Purpose | Real-time analysis of new messages | Backfill historical data and handle missing real-time paths |
| Duplicate prevention mechanism | Database UNIQUE KEY on log_id, INSERT IGNORE ensures idempotency |
|
The system performs multi-layer validation on each result returned by the LLM:
response_format=json_object to force LLM output into JSON, then parse and validate again on the client sideAny verification failure triggers retries, up to 3 times, with a 1-second interval between attempts. If all 3 attempts fail, the system writes a failure record (method='error') to prevent the batch worker from retrying the same record indefinitely.
Each analysis result records the prompt_version and method (model name) used. When the Prompt or model is updated, researchers can filter data by version number to ensure consistency of analysis.
Instructors (with Instructor / TA permissions for the Course) can obtain analytics data via the following API:
| API endpoint | Description |
|---|---|
GET /api/bloom/classroom/{id}/overview |
Overview of the class's cognitive-level distribution (number of items and average scores at each level) |
GET /api/bloom/classroom/{id}/students |
Analytical summary for each Student (main cognitive level, average score) |
GET /api/bloom/classroom/{id}/trajectory |
Cognitive Level Time Trajectory (aggregated by week, can filter specific Students) |
GET /api/bloom/classroom/{id}/export |
Export complete class data (JSON, aggregated by student × week) |
The classification method described on this page has been used in two peer-reviewed papers. When writing the methodology section, you may cite the following works to explain the validity and scope of the classification approach:
Chang, C.-K., & Li, K.-H. (2026). Chat as learning: Student–AI conversations as discipline-associated cognitive engagement patterns. Computers and Education: Artificial Intelligence, 11, 100644. https://doi.org/10.1016/j.caeai.2026.100644
Using this methodology, analyse the cognitive engagement patterns in student–AI dialogue and present differences in cognitive-level distribution across disciplines. Chinese summary at Research Evidence · Dialogue as Learning.
Chang, C.-K., & Li, K.-H. (2026). AI Teaching Assistants at Scale: Cross-Disciplinary Patterns of Adoption and Cognitive Engagement Across Hundreds of University Courses. In Proceedings of the Twelfth ACM Conference on Learning @ Scale (L@S '26). https://doi.org/10.1145/3774398.3811596
Apply this methodology across hundreds of Courses to compare adoption patterns and differences in cognitive engagement across disciplines. Chinese abstract: Research evidence · Large-scale AI TA.
This paper was shortlisted for ACM L@S 2026 Best Paper, one of only four shortlisted works across the conference.
If you use Bloom's Taxonomy cognitive-level data generated by the Uedu platform in an academic paper, it is recommended that you explain the following information in the methodology section:
Each message in the dialogue between the Student and the AI TA is automatically classified by the Uedu platform into cognitive levels using Bloom's Revised Taxonomy (Anderson & Krathwohl, 2001). The classification system uses a large language model (LLM; OpenAI gpt-5-mini) as the classifier, and through a structured Prompt (version v1.4) instructs the model to determine the primary cognitive level based on the cognitive operations shown by the Student (rather than topic difficulty), and outputs a six-dimensional score distribution. During classification, the previous 2-3 dialogue turns are included to understand the context, and text and image multimodal input are supported. The system validates the format and legality of each LLM response, retrying up to three times on failure. See https://uedu.tw/doc/bloom for a detailed methodological explanation.
It is recommended to provide the following information as well:
prompt_version field of the exported data)method field)| Version | Prompt Mode | Main changes |
|---|---|---|
| v1.4 | System / User split | Concise version, optimised for gpt-5-mini, reducing empty responses. Adds descriptions of meaningless types (emotional reactions, small talk, confirming agreement, prompting, pure pasting). Adds multimodal (image) support notes. |
| v1.3 | System / User split | Add dialogue-context handling rules (classify only the final message; earlier context is for understanding only) |
| v1.2 | System / User split | Split into system prompt + user prompt for the first time, enabling prompt caching |
| v1.1 | Merged | Add is_meaningful judgement and skip_reason field |
| v1.0 | Merged | Initial version, basic six-level classification |