Academic Support and Administrative Department
Annual Report Guidelines
Academic year 2018-19 marks the fifth year of the annual report and outcomes assessment process. This process is a critical component of the college’s efforts to develop and maintain an organized and sustainable program of assessment that improves the quality of the student experience.
The main goals of assessment are to support continuous improvement, to assist in the effective use of resources (time, budget, human, etc.), and to inform departments and other decision makers about the contributions of the department to the college mission. The purpose of the annual report is to facilitate departments in this endeavor and provide evidence of its effectiveness.
The annual report has two parts:
Part 1: This section focuses on the assessment work completed in 2018-19. It includes a discussion of findings from the assessment and any decisions made on how to use the findings to improve programs or services. See the section on Closing the Loop in this document for more detail and guidance. The summary of assessment activities and results may be in paragraph form, bullets or a combination of both.
Part 2: This section focuses on department plans for 2019-20. If there are no changes to the department’s goals, simply copy and paste the goals from the previous report. Please note, goals should link to your divisional goals as well as the Ursinus 150 Plan Strategic Objectives. Outcomes and objectives may stay the same or change depending upon the department’s needs. Either restate these or list the revised and/or new objectives and outcomes. The information on goals, objectives/outcomes, and assessment methods can be in a chart or list form, depending on what is best for your department.
Assessment is only useful if the information learned is used to make change for the betterment of the students’ educational environment. “Closing the loop” is how assessment becomes meaningful. This is the most important step in the assessment process. Closing the loop refers to the use of assessment results to improve student learning, programs, and services through shared discussions informed by the results of assessment. It is part of the continuous cycle of collecting assessment results, evaluating them, using the evaluations to identify actions that will improve student learning/programs/services, implementing those actions, and then cycling back to collecting assessment results, etc.
Section A: New or Ongoing Changes or Initiatives
Section A focuses on how the findings from previous assessment activities led your department to make changes to a program or service. This is the core of the “closing the loop” cycle. It is where departments will find if the assessment activities and resulting changes make a difference in how the program or service helps the department meet their goals.
Section B: Summary of 2018-19 Assessment Activities, Results Analyses and Next Steps
The purpose of this section is to step departments through an example of articulating the assessment undertaken and meaningful findings. Remember, the summary of assessment activities and results may be in paragraph form, bullets or a combination of both. The key is to make it meaningful and useful for the department. Please attach any documentation or summaries that would serve to inform this section. Also attach any rubrics or tool that was used to measure the outcome.
- Which outcome or objective did you assess?
- How was it assessed?
- What was the target or benchmark level for achievement of objective?
- Summary of Results
- Short summary of results focusing on the data/information relevant to the objective or outcome assessed.
- Interpretation and application: What successes were revealed? What challenges or issues were revealed?
- How are results being shared and discussed with relevant staff and faculty (if applicable)
- What changes, if any, are being made as a result of the assessment?
Section C: Summary of Other Departmental Assessments, Initiatives, or Changes
Section C captures any other program and service improvements or changes that may have occurred not previously planned or noted in the annual report from last year the previous year.
- Goals are broad statements that describe the overarching long-range intended outcomes of your department.
- These goals are primarily used for general planning and are usually not measurable, but they are used as the starting point to develop outcomes and objectives.
- Goals generally are consistent from year to year.
- You may have goals for different services/programs within your department.
- Make sure your goals align with the Mission and Ursinus 150 Plan Strategic Objectives of the college as well as division goals. Another source for linking student learning goals is the college Core Curriculum learning goals.
- To maintain collection supportive of student, faculty, and staff’s needs for teaching and learning
- Develop and promote activities that help students discover their career-related purposes
- Facilitate competence in students’ career skills and decisions.
For purposes of this section, the term “Outcomes” refers to student learning outcomes. Student learning outcomes are statements that specify what students will know, be able to do or be able to demonstrate when they have completed or participated in a program or utilized a service.
The term “Objectives” refers to the more operational and administrative outcomes of a department’s programs and services. Objectives are statements that describe the desired quality of key functions and services within the academic support and/or administrative area. Outcomes and objectives are specific and clear statements of measurable actions and achievement related to reaching the goals. Unlike goals, outcomes and objectives may change from year to year. They are milestones toward the achievement of related goal and may take more than one year to complete. It is recommended having no more than three to five measurable outcomes and/or objectives for each goal.
Examples of outcomes and objectives:
- Student learning outcomes examples:
- Students will learn how to use library resources
- Students will have the ability to write a resume
- Resident assistants will develop conflict resolution skills
- Students will understand the importance of civic responsibility
- Students will take an active role in their health and well-being
- Operational objectives examples:
- The number of electronic library holdings will increase by X%
- The attendance at workshops provided by the Career & Professional Development Center will increase
- X% of students will be satisfied/very satisfied with the student billing process
- X% of external vendors will report prompt payment of billings
- The number of staff using NEW U programs will increase by X%
This section addresses how your department will track and/or measure progress of the outcome or objective, which in turn measures progress toward goals. Having an assessment plan avoids the unnecessary collection of random data and information that serves no purpose. Only collect information that relates to your goals and objectives/outcomes, and that will yield useful, relevant, and actionable information. Before starting your plan, be sure to take inventory of what data may already be available either in the department or with the Office of Institutional Research and Effectiveness. Remember to be realistic in your assessment plans and targets of success.
Questions to ask in order to prepare a plan for assessing the outcome/objective include:
- What methods will be used to measure progress toward outcomes?
- What information/data will be used to assess outcomes?
- What tools will be used?
- What is the nature of the target/benchmark?
- What is the minimally acceptable result? What is the desired result?
- When will the assessment be completed?
- How often will the assessment be performed?
- Prior to doing the assessment, what is your intent for sharing, discussing and using results that you find?
- How will the results be discussed and shared, and used for program/service improvement?
The annual report process will ask each department to articulate the following:
- Outcome or Objective being assessed
- Method of assessment being used*
- Target/ Benchmark of Success**
- Completion/ Timeline
- Plan to discuss and use the results
A Sample of Common Assessment Methods
- Measures of volume of activity (e.g., number of clients (students/faculty) served, circulation data, purchase orders processed, reports produced etc.)
- Measures of efficiency (e.g., average turn-around time for completing requests, timely service/prompt response etc.)
- Needs Assessment
- Measures of service quality (need assessment, error rates, accuracy of the information provided etc.)
- Client satisfaction/ utilization surveys (student satisfaction survey, alumni survey, employer survey, customer survey etc.)
- Standard /guidelines provided by professional associations (CAS, AIR, NACUBO, SCUP etc.)
- Compliance with federal, state, county, city regulations
- Responses to external evaluators/auditors
- Measures of performance (Pre and Post Tests etc.)
- Benchmark/comparisons with peer institutions
*Methods of Assessment - Direct and Indirect
Best practices for assessment of outcomes and objectives include at least one direct method of assessment in addition to any indirect methods of assessment. See the FAQ online for more information.
- Direct measures of collecting information require students to display learning. Examples include evaluation of collections of student work, pre-post tests, performance on a case study.
- Indirect measures ask students or others to reflect on student learning. Examples include questions asking self-perceptions of learning, job placement statistics, the percentage of students who graduate.
** Target/ Benchmark of Success
Targets and benchmarks (sometimes also called criteria or standards) indicate the desired level of performance to be reached. A target is usually the criterion (long-term) level of desired performance. Targets can be quantitative or qualitative. Benchmarks provide an overall standard of performance comparison (benchmarking is the process used to identify the level of performance). Having benchmarks helps in understanding the results from assessments in context of a norm or other comparison. Desired targets should be determined prior to the assessment as part of the planning process.