This is a posting from Jerry Somerville of Napa Valley College.  Jerry 
presented his findings at the San Jose student success conference in 
October.

Bob Gabriner, Moderator
_________________________________________________________________

Critical Factors Affecting the Assessment of Student Learning Outcomes: A 
Delphi Study of the Opinions of Community College Personnel


The two-fold purpose of this study was to identify critically important 
factors that affect the meaningful assessment of student learning outcomes 
and study why they were critically important. A three-round Delphi process 
was used in both a pilot project and in a full study to solicit the 
opinions of individuals who were actively involved in student learning 
outcomes at community colleges. Panelists reviewed and rated, on a five 
point importance scale, a list of statements describing facilitating and 
thwarting conditions associated with the assessment of student learning 
outcomes. The 22 panelists who participated in the full study came from 12 
community colleges from throughout the continental United States and 
represented six different campus groups: faculty members, campus 
researchers, administrators, consultants, administrative support 
personnel, and assessment coordinators. 

>From a thematic analysis of the results, six factors emerged as critically 
important followed by four other factors that were classified as extremely 
important. The six critically important factors were: (a) 
knowledge/experience of campus leaders, (b) trust, (c) opportunities for 
dialogue/ collaboration, (d) leadership, (e) faculty engagement, and (f) 
use of assessment results. The four factors that were classified as 
extremely important were: (a) building campus knowledge, (b) having an 
assessment plan, (c) having communication strategies in place, and (d) 
having administrators engaged in assessment.

Knowledge/experience of campus leaders. It is critically important that 
there are knowledgeable campus leaders who have experience in a variety of 
assessment methods, particularly knowledgeable faculty leaders. A lack of 
such leadership represents a critical barrier to a meaningful assessment 
process. It is important to have leaders who have walked the talk. Their 
knowledge and experience may foster trust in the process.

Trust. There are two facets to trust. First, it is critically important 
that leaders of this process are respected and accepted by all campus 
personnel groups. Trust in the leaders fosters comfort with the change 
process, helps build a collaborative campus climate, and promotes buy-in 
for the process. Second, one of the greatest fears is that assessment 
evidence will be used in punitive ways against faculty. It must be obvious 
and transparent to faculty that the results will be used in positive ways 
for institutional and program improvement and not in punitive ways.  Trust 
is facilitated when this intent is put in writing on all assessment 
documents.

Dialogue/collaboration. The presence of venues for dialogue, 
collaboration, and sharing among campus groups was judged to be critically 
important. Such venues help to overcome a significant barrier, that being 
a lack of understanding and consensus among campus personnel of what needs 
to be done. Dialogue and sharing promote a collaborative climate and serve 
as the glue that builds a strong and lasting process. One aspect of 
collaboration is the presence of an assessment team. By working together, 
team members develop a shared message, trust, and momentum.

Leadership. A culture of assessment is facilitated when there is 
knowledgeable and respected leadership. It is critically important that 
faculty members are among the campus leaders in the assessment process. 
Other faculty members are more likely to participate when lead by 
knowledgeable, respected, and enthusiastic colleagues. Leadership from 
administrators was classified in the extremely important range and 
characterized as having a support role in assessment. Administrators give 
assessment credibility and make allocation of financial and personnel 
recourses more likely. Without their leadership, assessment programs may 
be weak and wither away from lack of resources. The presence of a core 
team broadly representative of the college and the continuity of dedicated 
staff having lead responsibilities for assessment were judged extremely 
important aspects of leadership. Without these conditions, it may be 
difficult to implement assessment at the institutional and program levels.

Faculty engagement. Two personnel groups are particularly important to a 
meaningful assessment process: faculty and administrators.  Faculty 
engagement emerged as critically important while administrator engagement 
was judged to be extremely important. Engagement of faculty, including 
adjunct faculty, is a driving force behind meaningful assessment. This 
engagement is characterized by a willingness to learn about assessment, 
analyze data, use results, and share what is learned. 

Using assessment results. Among the critically important themes of a 
meaningful assessment process is the use of results. The purpose of 
assessment is to use the results to make more informed decisions to 
improve education.  If results are not incorporated into campus decision-
making, then student learning assessment work will be viewed as having no 
value and being just a waste of time. One facet of using results is 
institutional support for assessment research and data analysis.  
Assessment needs to be properly researched and data correctly analyzed. 
Most faculty members do not have the time or expertise to do this.
Building campus knowledge. Developing knowledge among college personnel 
was judged to be extremely important. It is accomplished through faculty 
teaching other faculty and through ongoing and consistent offerings of 
high-quality, motivating educational opportunities for campus personnel.  
An assessment committee could serve as a vehicle for acquiring background 
knowledge, communicating it to the rest of the campus, and overseeing the 
implementation of staff development activities.

Assessment plan. Having an institutional plan that is linked to the 
college mission, is integrated into institutional policies and practices, 
is manageable, and is periodically evaluated for its effectiveness was 
rated extremely important. The plan provides a framework for the process, 
a document for those seeking understanding about assessment, and a 
rationale for budgeting. Also, writing an assessment plan can serve as a 
collaborative activity.

Communication strategies. Keeping campus personnel informed about 
assessment activities was judged as extremely important. It is very 
important that these communications are timely and that they come from 
respected faculty, vice presidents of instruction, and college presidents. 
Having both a written assessment plan and an institutional philosophy were 
important forms of communication, because they give structure and 
direction to the assessment process.

Administrator engagement.  As indicated above two campus groups, faculty 
and administrators, emerged as most important to a meaningful learning 
assessment process. Administrator engagement was judged to be extremely 
important because, administrators position assessment as an institutional 
priority and provide the necessary personnel and financial resources to 
manage the work of assessment. However, as important as administrator 
engagement appears to be, the results of this study indicate that faculty 
engagement may be more important.

To subscribe to this Listserv:
Send an e-mail message to Listserv@Listserv.cccnext.net
Do not put any words in the subject area.
In the body of the email message, Mary Smith would write the following: subscribe LearningAssessment Mary Smith
When sending the message, substitute your first and last names in place of "Mary Smith"

To access the LearningAssessment Listserv achive:
Go to http://listserv.cccnext.net/scripts/wa.exe?REPORT&Z=3
Click on Server Archives (on left column of page)
Click on LearningAssessment
You will have a list of all the postings at this point to choose from