Summarizing and Analyzing Assessment Results-Suskie Ch 16

This chapter is a little difficult to summarize in a blog, as it is comprehensive and all encompassing regarding our assessment results. However, here are some points to consider:
- Most assessments results can be summarized with simple tallies and percentages (the more direct and concise, the better).
-Error margins, which are easy to estimate, provide important information on the precision of assessment results.
-Good multiple choice items discriminate well between students who do well and poorly on the overall test.
-Keep a sample of good, bad and mediocre student work on file to provide firsthand evidence of your standards and rigor.

Analysis of results must occur before we can start sharing them and discussing them. There are several questions that must be answered first:
1. What kinds of standards are you using?
2. What kind of results do you have (qualitative, categorical, ordered, scaled, or dichotomous)?
3. Will you use technology to summarize and analyze your results? This is a highly recommended component of the anlysis. It beats doing it all by hand.

There are several ways to summarize results:
1. Tallies
2. Percentages
3. Aggregates
4. Averages
5. Qualitative summaries

We must evaluate the quality of our assessment strategies. Suskie recommends examining the following questions:
1. How well does your sample reflect all your students?
2. How precise are our sample results?
3. How difficult is each test item?
4. How well does each test item discriminate between hogh and low scorers?
5. Do other assessments corroborate your findings?
6. Do results fall in appropriate patterns?

Another point to ponder is whether we should analyze our results in varying ways. Finally, the chapter ends with a discussion of documentation and storage of the data collected.

July 20th Meeting Summary

This meeting we spent some time making sure that the Summer II chairs understood what our assessment related goals are for Fall 10:
1. Close the Loop on Program Assessment done Spring 10 and address "opportunity to learn"
2. Accomplish Gen Ed assessment of Communication Skills, Teamwork, and Social Responsibility Fall 10.

We reviewed the 3 purposes of our Fall 10 assessment:
1. Gen Ed Assessment for SACs Accountability (are our students close to graduation achieving our gen ed competencies)
2. Core Assessment for THECB Accountability (are our core students close to graduation achieving the core competencies)
3. Our desire to assess to improve student learning (how well are our students close to graduation achieving the competencies and how can we work together to improve student learning of the competencies)

We examined a data summary of the students who enrolled Fall 09 and who had taken and passed Engl 1302 and Govt 2305 or Govt 2306 by numbers of majors. These data may help us identify courses that will most likely have graduates in them Fall 10 so that we can give the faculty teaching these courses a heads up about the need for an assignment that may also be used for assessment purposes.

Before the July 27th meeting everyone will edit the generic rubrics. Several members of the team will do some background work on identifying population, sampling, and timeline feasibility for assessing graduating students and will report to the group July 27th.

Dr. Anderson Workshop Thoughts

Thanks to spending two days with Dr. Ginny Anderson, I have a lot of new ideas about our assessment work at PAC. In fact, I am hoping to capture what everyone else learned about our work during her workshop on a page in our Wiki that I dedicated for this purpose. http://pacassesscop.pbworks.com/Anderson-Workshop-Ideas
The categories that I set up on the wiki page are:
Sampling Ideas
Assessment Plan Ideas
Closing the Loop Ideas
Rubric Feedback on our three rubrics
General Ideas
Feel free to add to what I already put on the page -- Reflection really does enhance learning!

July 7th Meeting Summary

Today we reviewed the Communication Rubric and the Social Responsibily rubric and added minor edits to them up -- thanks to Sharon's help we are doing a better job writing those outcomes statements!
We also reviewed the Teamwork Guideline and talked a little bit about what the guidelines might include (the guidelines can't be completed until we get the logistics ironed out).
We talked about Faculty Development Day and recommended discipline specific groups for discussing ideas for assignments to assess using the rubrics Fall 10.
We thanked our Summer I group for all the hard work that they pulled off (we reviewed the Gen Ed Specs and the Worktime files in the Wiki -- these will be updated to reflect all the accomplishments up to this point).
The PAC Assessment Team Rocks!

July 6th Meeting Summary

Today we collectively worked on the Communication Rubric's first two outcome's rubric descriptions:
1. Content and Purpose
2. Organization
Our goal was to develop descriptions that would work across the curriculum whether the communication is oral, visual, or written.
Next Sharon Carson shared with us a handout on writing measurable student learning outcomes.
Finally we broke up into our competency teams to polish our rubrics. Some teams moved on to generate ideas for faculty guidelines.
Tomorrow's meeting we will "show and share" our rubrics and guidelines thoughts and celebrate our progress!