Outcomes
Assessment
Reports
a newsletter and Web log
by Frank Vozzo
 
Director of Student Learning Outcomes Assessment
 

 
December 19, 2005

In this report:

The whole point of assessment is to make informed decisions that improve student learning.  If assessment information will sit in the filing cabinet, why collect it in the first place?

We collect the information because we know it is the right thing to do, and because we have every intention of using it.  The faculty governance system conducts the Student Opinion Survey each semester.  Some individual faculty members and departments conduct student surveys at the start or at the end of a course, at the time of graduation, or at other critical moments.  Some of these surveys are class assignments, while others are outside of regular class work.  Some relate to general education, and others to the majors.  Unfortunately, surveys take time to process and the information they reveal takes time to digest.  And some of what these surveys tell us can be painful for us to read-- we have to spend time thinking about what we might be doing wrong.  

It might be less painful if you just think of the work as scientific research.  Make this New Year's Resolution: read the survey responses in your files going back five years, and figure out the trend over time for the response to just one item that seems significant to you.  For example, you could take one course and look at the trend over several semesters in responses to the SOS item "A significant amount of student effort was required to successfully complete this course."  If the responses are consistently different from your expectations, then what will you do next time to make your expectations for student effort clearer?  

At Sage, we define undergraduate academic support services as the combination of advisement, career planning, Disability Services, HEOP/Learning Support Services, the Academic Support Center (tutoring), and the Writing Studio (Troy); all under the direction of the Dean of Campus Life and the Associate Dean of Academic Services (see CampusCruiser/My Cruiser/My Committees/TSC Faculty Staff and Administrators/Shared Files/1Institutional Folder/The Sage Colleges Organizational Chart/H-Visio-DeanCL-names.pdf).  All of these offices should generate reports and share information with various constituencies.  The reports should be widely circulated, and faculty members need to become better consumers of this information.

The National Association of Developmental Educators defines itself as "a nationally recognized association of professionals with expertise to help students academically succeed throughout the entire educational experience from high school through college and graduate/professional school."  Its professionals provide many services at colleges and universities including academic advisement, student learning assistance, and even faculty development workshops.  NADE has embraced outcomes assessment, and encourages its members "to reinvent themselves as resources for the entire campus -- students and faculty alike -- in partnering with the new enriched learning environment."

Having just cooperated in the process-mapping activities, faculty members and administrators at Sage are beginning to explore new partnerships that will improve student learning and retention, beginning with a common set of goals and better information sharing.

Barbara E. Walvoord of the University of Notre Dame is famous for having co-invented the grading technique called "Primary Trait Analysis".  In this 2004 article in The Advocate (page 1 of 5; follow link next article to read the rest), she writes "when institutions want to assess learning, their most common question is: 'How can we build on the grading process?'"  The answer is: write a grading rubric.

The technique is fairly simple.  If an assignment can be divided into components that address learning objectives, assign each component a numerical or letter score based on the degree to which the objectives have been met.  This is typically done by writing descriptions of each "degree" (the student did this but not that, included this but left this out, and so on-- see the article for examples).  I use this method when assigning grades for WLD-120 poster presentations at the WORLD conference (this is one of the four criteria used for judgment):

3. Organization and layout.

A. The poster is visually appealing: it is large enough, with appropriate use of color and adequate font size, and with appropriate balance of illustrations and text. The information is organized in a logical fashion, with the main points standing out from the details. Each member of the group (if applicable) is fully knowledgeable of the poster’s content and adequately participating in its preparation and presentation.

B. Some deficiency exists in two of the three subcategories: There is a flaw in the poster’s visual appearance which distracts the observer (eg. sloppiness, small font, imbalance in layout, too much white space). The information has logical organization but it may not be apparent, or some information may be hard to find on the poster. There is inadequate participation by one or more group members (if applicable).

C. Some deficiency exists in all three subcategories, OR serious deficiency exists in one or two of the subcategories: The poster is visually unappealing-- either doing nothing to attract passers-by, or containing visual flaws that ruin its ability to convey the information. The information is randomly presented, without hierarchy of importance or without logical sequencing. The presenter(s) seem(s) to be unfamiliar with some of the poster’s content and unable to answer some questions.

D. Serious deficiency exists in all three subcategories.

F. Obviously sloppy work or careless design, or complete lack of organization, or presenters unprepared.

I give my grading rubric to the students weeks before their poster presentations.  I like doing this in order to better communicate my expectations to students, and it has the added benefit of eliminating the need to argue with them later about what my grade means.

How does this relate to outcomes assessment?  The grades carry with them a wealth of useful information.  Instead of being defined with the relative terms "excellent, above average, average, poor, failing", the students' grades are based on their meeting my written learning objectives.  I can look at the student scores in one area as measurements of learning outcomes and make informed decisions about course improvement.

 

In two weeks: some best practices at Sage.

Home...