- The program has well developed program goals, and goals are in use and appear on all syllabi.
- Long-range assessment planning has taken place.
- Data collection and synthesis is performed in a substantial and systematic manner.
- Analysis and synthesis of assessment data appears or are used in annual reports and program reviews, as well as in other program development and improvement efforts.
- The entire program faculty discusses assessment data, planning and findings in a regular or systematic manner or interval.
More Guides to Assessment Best Practices
The UM-Dearborn emphasizes direct assessment measures, but programs can use a combination of direct and indirect measures.
looking at student performance by examining samples of student work already being done in the course. This is also referred to as “course-embedded” assessment.
gathering information about student learning by looking at indicators other than student work output.
- student exit surveys
- alumni surveys
- institutional data
- questions administered to student focus groups
- essay prompt: “what have you learned about [program goal] in this course?”
Indirect assessment is especially appropriate for investigating issues such as program retention and goals such as “promoting lifelong learning.”
- Pre- and Post-tests
- Exams and quizzes
- Writing assignments and other exercises, such as labs
- Portfolios of student work in a course or in a major
- Capstone projects
In rare cases: if your courses do not have an existing assignment/exam that requires students to demonstrate a program learning goal, then you should create a new assignment or exam question that clearly addresses the goal.
*If your program cannot identify existing assignments/exams in key courses that address your program’s learning goals, then you have likely discovered a curriculum problem. That means there is a mismatch between what you say students should be learning in your program and what you are actually teaching as well as the tools you are using for students to demonstrate their learning. Assessment is intended to reveal these mismatches and encourage faculty to find ways to solve curricular problems.
How to Conduct Direct Program Assessment
- Identify assessment instruments (e.g. exam questions, assignments) that are aligned with the program learning goal(s) you want to assess.
- Use the goal-course matrix to determine which courses are best suited to assess which program goals.
Develop a long-term plan for assessment that makes sense for your program:
1) Rotate through goals, selecting a different goal to assess each semester.
2) Select one or two problematic goals to assess for several semesters, making continual adjustments to increase students’ performance.
- Ensure that the program goals are being addressed throughout the program’s curriculum. Thus, don’t just focus the assessment on the same one or two courses every time.
- Determine benchmarks for student success (proficiency level) on each program goal.
- Assess the learning of all students in a class.
- Report how many students are exceeding, meeting and not meeting the benchmark for the program learning goal(s) being assessed.
- Finally, the faculty should discuss the assessment findings and reflect on their meaning. The assessment report should be circulated among all program faculty. Program faculty should discuss how to improve student learning in the program and develop an “action plan” to implement the changes. Even if your program is satisfied with your students’ performance, you should seek ways to continuously improve. Aim to set aside time at one or two program meetings a year for assessment discussion.
It is recommended that programs involve Lecturers in the assessment process.
Program Goals should:
- Encompass a discipline-specific body of knowledge for the program.
- Be reasonable in number (3 to 5 goals, unless mandated by an external accrediting agency) so as to encompass the mission while still being manageable.
- Be measurable.
- Employ active verbs, usually from the higher levels of Bloom's taxonomy.
- Be aligned with college and university goals and with profession organizations where applicable.
- Be created by program faculty and periodically revisited by program faculty.
A course-goal matrix (or map)
- Identifies which courses in a program's curriculum are aligned with which program learning goals.
- Identifies courses' different degrees of alignment with program goals, from minimal to robust, or from introductory to mastery.
- Allows faculty to visualize how their program goals are being addressed across the curriculum.
1) In most programs, the curriculum should be designed so that the upper-level classes, especially the senior-level courses, show the strongest alignment with most or all of the program goals. This curriculum design will be displayed in the matrix.
2) The matrix will reveal gaps in the curriculum if some goals are minimally addressed.
- Aids faculty in selecting particular courses to assess for each program goal, helping to develop an assessment plan.
“The alignment of learning goals and curricula is critical. If learning goals are adopted but are not addressed in the curricula, the outcomes assessment process will be worthless” (AACSB, 2007, p. 8).
Sample Matrices from UM-Dearborn programs
- Develop a program-goal rubric for existing assignments, evaluating your students’ work using those measures, alongside the grading rubric/method you normally use. (example)
- For exams and quizzes, identify specific questions that are aligned with a particular learning goal. For assessment purposes, evaluate responses on the specific questions separately from the rest of the exam or quiz.
- Use common questions/assignments across multiple sections that are mapped to specific program goals. These could be in the form of pre-post tests to achieve even better measurement of learning resulting from a course.
The Relationship between Grades and Assessment
Grades do not normally suffice as assessment measures. The grades you assign to a test/assignment/course are usually based on many factors other than the program goal. Strive to develop a measure just for the program goal(s) you are assessing. For each program goal you assess, you should be able to measure each student on the “did not meet” to “mastery” scale, honing in on the student’s performance only for that goal.
- Examples of Direct and Indirect Assessment Methods
- More Examples
- The University of Missouri-Kansas City Assessment Handbook, pp. 7-10
- Developing a rubric aligned to a program goal
- Rubric basics and samples from DePaul University
- Tutorial on Rubric Creation
- VALUE Rubrics (note: these rubrics can be found in Canvas Outcomes, accessible in all courses)
- ABET Assessment Resources