Outcomes
Assessment
Reports
a newsletter and Web log
by Frank Vozzo
 
Director of Student Learning Outcomes Assessment
 

 
December 12, 2005

In this report:

The Fall 2005 Institutional Trend Report was just posted a few days ago at CampusCruiser/My Cruiser/Dashboard/My Committees/TSC Faculty Staff and Administrators/Shared Files/Institutional Research and Planning.  There is a section labeled "Outcomes Assessment" which contains information about library usage, the number of computers available to students on campus, and career outcomes.  Retention information appears under the separate heading "Enrollment".  

While this is very useful, there are always useful things that could be added such as 

What would you like to see added?

Here are some ideas for "documenting assessment" within a course by using available data from the grading process: 

  1. When tests contain objective questions (such as true/false, multiple choice), use SCANTRON forms*. Set the scanner to perform an item analysis (mark the boxes "key" and "2" in the line before the answers section on the answer key form). After the forms have been scored, pass the item analysis form through the scanner. The number of wrong responses to each question will be tallied, along with the number of forms processed and the class average. 
  2. When providing feedback to written work in a small class, or to a small number of groups of students doing a project together, put your comments into a word processor instead of writing them directly on the paper, then print an attachment page. This has a double advantage: you can save time by copying and pasting comments from one feedback file to another when the same issue appears over and over, and you can look at the comments you made to students in past semesters when judging the effectiveness of your own teaching, and when evaluating the results of changes you might have made between semesters.

Every one of my tests has at least one question that's a "ringer." By that I mean a question that demonstrates some deep understanding of course material in its application to a slightly new situation.  I expect that many of the students in the class will miss such questions. Rather than "teach to the test" with these questions (giving students prior warning), I deliberately keep the questions a surprise (they are on test pages that I re-collect after going over the test). I track the responses to these questions with item analysis, and design my final exams in the same semester as well as homework assignments in future course offerings based on the outcomes. One of my favorites involves the physics of static friction on an incline.

*Sage has SCANTRON optical readers on both campuses, and forms to go with them are stocked in departments that regularly use them.

Research has shown that poorly-performing students are more likely to change their study habits if 1) they can respond to probing questions about those habits in an anonymous survey, and 2) if they can see how others in the class are dealing with the same difficulties that they are experiencing. There are two techniques I'll mention here; the first appeared in assessment literature in 1993, and the second in 1995:

  1. The Minute Paper: an instructor stops teaching, passes out index cards, and asks students to write very short responses to three questions (at most). Typically the questions are a. What do you think was the main point of this class? b. What do you think was the muddiest point of this class? c. What question do you want answered at this point? The questions can be about anything, including such probing items as "how much time did you spend preparing for this class" and "have you asked questions in class at least once during this course". The instructor reads the cards immediately after class, summarizes the responses at the start of the next class, and tells the students how the feedback will be used. Students need to be confident that their anonymity will be preserved (the cards should be thrown out so that handwriting doesn't get recognized over time).
  2. Electronic real-time feedback (so-called clickers, which are familiar to people who have seen the "ask the audience" lifeline used on "Who Wants to Be a Millionaire"). Students are given a multiple-choice question. For example: how much time did you spend studying chapter 5 and doing the homework assignment: a) more than 4 hours, b) 3-4 hours, c) 2-3 hours, d) less than 2 hours. The results are displayed in front of the class immediately on a histogram, and the instructor and the students can be collectively pleased or horrified by them. The technology is not widely available at Sage (ask around; you'll find a few instructors experimenting with it). 

While not a real-time method, you can use the assessment tools in Blackboard to accomplish the same thing. Here's how: 

  1. Open the Control Panel for your course site.
  2. Choose the Assessment area.
  3. Choose Survey Manager.
  4. Follow the instructions for setting up your questions. First give the survey a title (you don't have to fill in the information or instructions blanks). After submitting that, you can choose multiple choice under "canvas" (ignore any "point value" information), and type in your questions and available answers. When done creating the survey, you should see it listed by title under Survey Manager in the Assessment area of the control panel. Next, go to one of your content areas in the control panel (such as "Assignments"), and choose "Survey" in the "Select" drop-down menu in the top right corner (click "go" to continue). Highlight your survey in the add survey box, and choose "submit". Finally, go to the content area and make the item available (choose "modify" and "modify the survey options", then activate the "link" option).
  5. After the time limit you established for the survey is up, use a few minutes of class time to show the results to the students on the table obtained via Control Panel, in the Gradebook (be careful to hide the display until the results are found and ready for viewing). Click Gradebook, and then click on the title at the top of the column for the survey item (notice that checkmarks identify which students have already taken the survey). In the next menu, choose "Assessment Attempt Details". The results are displayed as a table with the percentage response for each choice.
 

In next week's OARs: grading rubrics.

Home...