AAT Prototype Features and Mark Scheme
You may use any language, libraries/modules or tools to build an Automated Assessment Tool (AAT) prototype, so long as these are agreed upon within your team. Your updated team agreement should include any team-agreed coding-conventions, standards and justification for your choice of language, tools and libraries/modules chosen. Your code should be maintained on GitLab.
Any AAT prototype MUST feature the following 5-core functionalities to meet the needs of identified end-users (students and teaching-staff):
1. ADD (EDIT and DELETE) QUESTION-TYPE ONE
Teaching-staff must be able to Add questions. Teaching-staff should be able to Edit and Delete questions.
A question must consist of at least a question, answer, (check for correctness) and feedback.
Distinction
Teaching-staff can add, edit and delete type-one questions.
Interesting and relevant extra features may have been included e.g. question tags, difficulty levels, points.
Students can undertake type-one questions, no errors are generated, and feedback received is good, including provision for feed-forward.
Teaching-staff can add type-one questions – no reliability or usability issues are present, but the implementation does not extend beyond adding a question.
Students can undertake type-one questions no errors are generated, feedback received is good.
Teaching-staff can add type-one questions to the AAT, some errors may be present in the code but more often than not works as expected. Some user inputs have not been anticipated and elicit no feedback.
Students can undertake type-one questions but functionality is unreliable sometimes. Feedback received is adequate
Teaching-staff cannot add questions of type-one, or this functionality is hindered by so many errors as to be unusable. Teaching staff cannot edit or delete questions.
Students cannot undertake type-one questions or errors when doing so are generated frequently. The feedback that students receive is poor
2. ADD (EDIT and DELETE) QUESTION-TYPE TWO
Teaching-staff must be able to Add questions. Teaching-staff should be able to Edit and Delete questions.
A question must consist of at least a question, answer, (check for correctness) and feedback.
Distinction
Teaching-staff can add, edit and delete type-two questions.
Interesting and relevant extra features may have been included e.g. question tags, difficulty levels, points.
Students can undertake type-two questions, no errors are generated, and feedback received is good, including provision for feed-forward.
Teaching-staff can add type-two questions – no reliability or usability issues are present, but the implementation does not extend beyond adding a question.
Students can undertake type-two questions no errors are generated, feedback received is good.
Teaching-staff can add type-two questions to the AAT, some errors may be present in the code but more often than not works as expected. Some user inputs have not been anticipated and elicit no feedback.
Students can undertake type-two questions but functionality is unreliable sometimes. Feedback received is adequate
Teaching-staff cannot add questions of type-two, or this functionality is hindered by so many errors as to be unusable. Teaching staff cannot edit or delete questions.
Students cannot undertake type-two questions or errors when doing so are generated frequently. The feedback that students receive is poor
3. ADD (EDIT and DELETE) FORMATIVE ASSESSMENT
An assessment is comprised of a teaching-staff defined sequence of questions made up of all type-one or type-two questions or a mix of both
Student users must be able to undertake assessments of formative type (an assessment for learning), receiving immediate feedback from each question
Distinction
Teaching staff can add, edit and delete assessments of formative type.
Students can undertake formative assessments.
No errors are evident when using the system.
The system may contain relevant extra features for assessments e.g. difficulty levels, option for students to retake questions, randomisation of questions, assessments tagged by topic, overall assessment feedback
Functional and reliable implementation which allows teaching-staff to add formative assessments to the AAT and student users to undertake a formative assessment. No errors are present.
Teaching staff can add formative assessments to the AAT.
Students can undertake formative assessments.
Functionality for either end-user may be unreliable at times.
Teaching staff cannot add formative assessments (defined sequences of questions) to the AAT. Or this feature has errors which make it unusable/unreliable most of the time.
Students cannot undertake a newly added assessment – or errors are generated more-often-than-not when undertaking an assessment.
Provision for providing immediate feedback is not present.
4. ADD (EDIT and DELETE) SUMMATIVE ASSESSMENT
An assessment is comprised of a teaching-staff defined sequence of questions made up of all type-one or type-two questions or a mix of both
Student users must be able to undertake assessments of summative type (assessment of learning), receiving automatically generated feedback at a point defined by the teaching-staff from each question.
Distinction
Teaching staff can add, edit and delete assessments of summative type.
Students can undertake summative assessments.
No errors are evident when using the system.
The system may contain relevant extra features for assessments e.g., overall assessment feedback
Functional and reliable implementation which allows teaching-staff to add summative assessments to the AAT and student users to undertake a summative assessment. No errors are present.
Teaching staff can add summative assessments to the AAT.
Students can undertake summative assessments.
Functionality for either end-user may be unreliable at times.
Teaching staff cannot add summative assessments (defined sequences of questions) to the AAT. Or this feature has errors which make it unusable/unreliable most of the time.
Students cannot undertake a newly added assessment – or errors are generated more-often-than-not when undertaking an assessment.
Provision for providing automatically-generated feedback is not present.
5. TEACHING-STAFF REVIEW STATISTICS
Teaching-staff must be able to review relevant student-user attainment, attempt and engagement statistics. (students – individual students, cohorts – annual intakes of students)
Distinction
Teaching-staff can review statistics for students and cohorts of students. The data presented is accurate and relevant. Relevant extra features may be included in this functionality e.g. data can be exported or can be refined by other parameters than students/cohorts
Teaching-staff can review statistics for students or cohorts of students. The data presented is accurate and relevant.
Teaching-staff can review statistics for students or cohorts of students. The data presented is accurate but uninteresting or irrelevant.
Teaching-staff cannot review individual or cohort student statistics, or errors cause feature to be unusable or present erroneous data
For each member of your team over #5 your AAT should feature (any) +1 of the following optional functionalities to meet the needs of end-users, assessed with the table below: NOTE – Login will not be considered an additional features.
Distinction
The functionality meets the need of an end user.
Some relevant extra functionality has been included to exceed basic requirement of this feature, which is sufficiently functional as a proof of concept within a prototype system
Functionality meets the need of an identified end user and includes no errors.
Good user feedback is evident for anticipated inputs.
The functionality meets the need of an identified end user but includes errors which make it unreliable or cases where no feedback to the end user is given, which make impacts usability
The functionality is not featured or does not work – features so many errors as to make it unusable
6. STUDENT REVIEW STATISTICS
Students must be able to review relevant attempt, attainment and engagement statistics.
7. TEACHING-STAFF REVIEW STUDENT SATISFACTION STATISTICS
Students must be able to indicate satisfaction with the AAT, questions, assessments and other relevant factors.
Teaching-staff must be able to review relevant student satisfaction statistics informing teaching and assessment in-or-out of the AAT.
8. COMMENT SECTION
Both teaching-staff and students must be able to leave and review comments on questions set as part of a formative assessment.
9. ADVANCED/UPRATED/LEGENDARY GAMIFICATION
Must include relevant gamification features (badges/medals, profiles, profile-upgrades, leader-boards) for achievements attained by students and viewable by other student and teaching-staff users.
10. TEACHING STAFF AMMEND SUMMATIVE ASSESSMENT FEEDBACK
Teaching-staff must be able to review a completed summative assessment from a student user any the automatically generated feedback to which additional comments can be added and the assessment mark adjusted.
11. AAT SUPPORTED PEER-REVIEW OF ASSESSMENT SUBMISSION
Student users must be able to flag their completed assessments (formative and summative) for review by other student users and have their assessment commented by their peers (after completion – for formative assessment / deadline – for summative assessment).
12. WILDCARD FEATURE
Pick some interesting feature identified in your teams use-cases which must be incorporated alongside the core functionalities and any other optional functionalities chosen. (NOTE – Discuss these with the client before moving forward)