Cross-university platform portalShow a desktop notification when the AI TA finishes replying
Notify me when classmates post messages in the forum
Play an alert sound whenever there is a new notification
Explain the design principles of the pre- and post-test Likert scales for Simulated Debate, including the three-dimensional measurement framework (stance, confidence, openness), reverse-item handling, attitude-change calculation method, and research applications.
Before and after each simulated debate (mode=4), the Simulated Debate Attitude Survey asks students to complete a five-point Likert scale to measure attitude changes before and after the debate across the three dimensions of position, confidence, and openness.
Different from the pre- and post-test Quiz in Socratic Dialogue, the debate survey measures attitudes rather than knowledge; there are no "correct answers", but rather an observation of the extent to which student viewpoints change.
| Orientation | Socratic Quiz | Debate attitude survey |
|---|---|---|
| Question Type | Single-choice question (A/B/C/D) | Likert five-point scale (1-5) |
| Measurement objective | Knowledge Comprehension | Attitudes and viewpoints |
| Correct Answer | Yes | N/A |
| Scoring method | Number of correct answers + learning gain | Average score + dimension score + attitude change |
| Reverse-coded item | Not supported | Supports (effective score = 6 - raw score) |
| Presentation | Learning Gain Report | Attitude change report |
The survey questions cover three psychological measurement dimensions:
| Dimension | English | Measurement content | Question example |
|---|---|---|---|
| Position | Stance | Students' level of agreement/disagreement with the debate motion | "I believe AI should be allowed to make medical decisions" |
| Confidence | Confidence | The degree to which students are convinced of their own viewpoint | "I am confident that I can put forward strong arguments for my position" |
| Openness | Openness | Students' understanding and acceptance of opposing viewpoints | "I can understand the arguments of those who take the opposite view" |
Effective debate practice should improve students’ confidence (being more willing to express their views) and openness (being better able to understand the other side), while changes in position reflect whether the debate has changed students’ views.
A five-point scale is used for each question:
| Score | Tags |
|---|---|
| 1 | Strongly disagree |
| 2 | Disagree |
| 3 | Neutral |
| 4 | Agree |
| 5 | Strongly agree |
Some questions are reverse-scored (for example, 'I cannot understand the other person's point of view'); when calculating scores, the effective score = 6 - raw score. For example, if a student selects 2 (disagree), the effective score is 4, which actually indicates a tendency to agree with 'I can understand the other person's point of view'.
Each survey must include at least 1 reverse-coded item, used to check response consistency.
The system uses the GPT-4o-mini model to automatically generate Likert-scale items based on the title and background of the debate topic:
| Field | Type | Description |
|---|---|---|
item_text | String | Question text (e.g. "I think...") |
dimension | String | stance / confidence / openness |
is_reverse | Boolean | Whether it is a reverse-coded item |
source | Enum | ai or manual |
The valid scores for all items (reverse-coded items converted) are averaged, on a 1.0-5.0 scale.
Group by stance / confidence / openness, and calculate the average score for each.
After the post-test is completed, the system generates an attitude change report, including:
Attitude questionnaires are not included in Normalised Gain, because Likert scales do not have a concept of a "full score". Researchers should use a paired t-test or the Wilcoxon signed-rank test to examine the statistical significance of pre- and post-test differences, and report effect size (Cohen's d).
| Field | Type | Description |
|---|---|---|
topic_id | INT | Associated debate topic ID |
item_order | TINYINT | Question order |
item_text | TEXT | Question text |
dimension | VARCHAR | stance / confidence / openness |
is_reverse | TINYINT | Whether it is a reverse-coded item |
| Field | Type | Description |
|---|---|---|
topic_id | INT | Debate topic ID |
chat_id | INT | Associated dialogue session ID |
user_id | INT | Student ID |
phase | ENUM | pre / post |
answers | JSON | Score for each item, such as {"1":4,"2":2,"3":5} |
mean_score | DECIMAL | Overall average score |
dimension_scores | JSON | Average score across dimensions |
time_spent | INT | Response time (seconds) |
Attitude questionnaire data can be exported via Uedu Lab for the following research:
Attitude questionnaires involve students' personal views and positions, and are therefore sensitive data. Please confirm that IRB approval has been obtained before use. The system has already indicated on the questionnaire page: "This questionnaire is not counted towards grades and is for teaching research use only".