Measurement and Evaluation
v 1.0
The ReadyAI measurement system was first introduced in the 2018 World Artificial Intelligence Competition (WAICY) on July 28th, 2018, as the judging rubric for the competition. The rubric resembles our belief in engaging students of all interests by emphasizing on the “non-technical” components of the projects. 50% of the possible points are based on the creative/artistic aspects (visual design, audio design, etc.), while the other 50% is based on the technical/programming aspects. As we continue to create opportunities for AI learning, we would like to maintain the 50/50 format in our student assessments. In a way, we are sending a message, that the creative/artistic aspects of the projects are just as important as the technical/programming aspects.
We would like to put forward a measurement and evaluation system that helps us monitor and modify our programs. This system aligns with our beliefs. And, it will evolve to align with any future standards in the subject of AI.
Student Assessment
| Categories | Students should be able to understand/do: | Outcome |
| AI Concepts | Visual Recognition;
Facial Recognition; Landmark-based Navigation; Object Manipulation; Speech Recognition; Speech Generation |
Scores on quizzes
Scores on projects (based on the rubrics) |
| Command of Programming Tools and Programming Logic | Basic commands of Calypso;
Prediction of the outcomes of programs |
Scores on quizzes
Scores on projects (based on the rubrics) |
| Project Skills | Create a project with clear step-by-step outcomes | Scores on projects (based on the rubrics) |
| Multimedia/Creative Skills | Create and enhance the projects in aspects of visual/audio design, story design, presentation design | Scores on projects (based on the rubrics) |
| Thinking in the context of AI | Come up and present an idea about how AI can change the world for the better | Demonstrated in the projects |
Pre and Post Program Surveys
| Pre-program | Post-program |
| The familiarity of AI concepts | The familiarity of AI concepts |
| Attitude towards AI | Attitude towards AI |
| Time spent outside of the program learning about AI | Time spent outside of the program learning about AI |
| Interest in pursuing a career in related fields | Interest in pursuing a career in related fields |
The post-program survey will also include student opinions on
| The difficulty level of the toolkit |
| The relevancy of the topics |
| The quality of instruction |
| The effectiveness of class format and style |
| (Other case-specific aspects) |