I’ve made it to the final week of my LDT506 - Evaluation of Learning Systems course! It was rigorous and challenging, but well worth the time and effort. As the class comes to a close, let’s dive into how my evaluation skills have changed over the course of 7.5 weeks.
In March, I rated my overall evaluation skills as a solid 2 out of 6 based on the AEA Competencies. While I possessed transferable interpersonal skills, my technical and methodological skills were lacking. After completing this self-assessment, I was apprehensive about taking this course. The amount I did not know was daunting, and I was unsure of what to expect!
Today, I rate myself as a 4 out of 6 based on the AEA Competencies. This improvement of two points is based on the combination of weekly readings and assignments I completed each week. They filled my evaluation toolbox with knowledge on evaluation design, data collection methods, data analysis, and reporting. Previously, I had no prior knowledge of these topics. Each assignment was built on the others, which allowed me to clearly see the natural progression of the evaluation process. Also, the course’s final products, including my team’s Final Evaluation Report and Final Evaluation Presentation, were the culmination of our newly learned skills. Being provided with various opportunities to apply my knowledge has made me a more confident evaluator. I still have room for improvement, but I am proud of the growth I have made.
After completing the self-assessment, I see my advancement within the methodology domain. More specifically, I strengthened my skills in Competency 2.11: “Analyzes data using credible, feasible, and culturally appropriate procedures.” Although I did not reference this domain in my initial blog post back in March, it was one of my lowest-rated areas as a 1 out of 6. Previously, the extent of my statistical knowledge was from a statistics course I completed in my junior year of high school. As of today, I would rate myself a 4 out of 6. During this course, I learned to use the statistical software program, JASP. Through pre-recorded tutorials and assistance from my classmates, I was able to import data sets and critically analyze data. I learned how to calculate measures of central tendency (mean, median, mode), standard deviation, and range. I created frequency tables and distribution plots too! I understand these concepts are only the bare bones of what the tools offer, but I am planning to continue learning how to utilize JASP and its more advanced capabilities.
Next, I saw growth within Competency 2.14: “Uses evidence and interpretations to draw conclusions, making judgments and recommendations when appropriate.” Initially, I rated myself a 1 out of 6 in this competency. Now, I rate myself a 4 out of 6. After strengthening my data analysis skills by completing this assignment, we did not merely analyze the data, but we also made conclusions based on the data. I interpreted the data sets I was provided (frequency tables and distribution plots) and made reasonable conclusions and judgments. My team and I crafted actionable recommendations using these conclusions in our Team Evaluation Report assignment.
My rating for one of my initial areas of strength, Competency 4.7 “Teams with others when appropriate,” increased as well! Teamwork was the focus of this course, and I collaborated with six of my peers in Arizona, Uganda, and Rwanda. For most of the course, I followed our Project Manager, Alex. However, for our Module 6 Assignment: Team Evaluation Report, I led a smaller subset of our team. I coordinated a scheduled meeting with two team members to ensure successful completion of the Introduction and Evaluation Methods and Design section of the assignment. I checked periodically on our sub-team’s progress, and I answered questions when needed. For our Module 7 Assignment: Team Evaluation Presentation, I took on the leadership role for the entire team. With input from our team members, I designated our project timeline and divided up the responsibilities. Although these two assignments only spanned a total of 2 weeks, I felt my ability to collaborate effectively within a team improved.
Some competencies that surprised me as being necessary include 1.8: “Identifies how evaluation practice can promote social justice and public good” and 1.9: “Advocates for the field of evaluation and its value.” These are two areas I could benefit from expanding upon. As mentioned in our course textbook, Evaluation In Organizations: A Systematic Approach To Enhancing Learning, Performance, And Change, evaluation is inherently political and our evaluations have real-world repercussions. At the beginning of this course, I did not understand the gravity of this responsibility, but it’s clear that evaluators have an ethical responsibility to ensure evaluation benefits its stakeholders.
Beyond LDT506, I plan to build upon competency 1.7, “Pursues ongoing professional development to deepen reflective practice, stay current, and build connections.” I would like to seek out additional reading material and online courses to continue learning about evaluative practices. Our instructor, Dr. Salik mentioned how the advancement of AI will impact the field of learning design. Responsibilities usually delegated to instructional designers such as content writing and assessment creation will be completed by large language models such as ChatGPT, Google Gemini, Claude AI. As AI streamlines these tasks, our roles may begin to shift into program evaluation. To ensure I am prepared for the learning design job market upon completion of my program in May 2027, I plan to continue building my evaluation skills and stay informed on upcoming trends within the field.
References
American Evaluation Association. (2018). AEA evaluator competencies. https://www.eval.org
