| 
  • If you are citizen of an European Union member nation, you may not use this service unless you are at least 16 years old.

  • Social distancing? Try a better way to work remotely on your online files. Dokkio, a new product from PBworks, can help your team find, organize, and collaborate on your Drive, Gmail, Dropbox, Box, and Slack files. Sign up for free.

View
 

critical factors in assessment

Page history last edited by PBworks 12 years, 7 months ago

 

This is a posting from Jerry Somerville of Napa Valley College.  Jerry 
presented his findings at the San Jose student success conference in
October.

Bob Gabriner, Moderator
_________________________________________________________________

Critical Factors Affecting the Assessment of Student Learning Outcomes: A
Delphi Study of the Opinions of Community College Personnel


The two-fold purpose of this study was to identify critically important
factors that affect the meaningful assessment of student learning outcomes
and study why they were critically important. A three-round Delphi process
was used in both a pilot project and in a full study to solicit the
opinions of individuals who were actively involved in student learning
outcomes at community colleges. Panelists reviewed and rated, on a five
point importance scale, a list of statements describing facilitating and
thwarting conditions associated with the assessment of student learning
outcomes. The 22 panelists who participated in the full study came from 12
community colleges from throughout the continental United States and
represented six different campus groups: faculty members, campus
researchers, administrators, consultants, administrative support
personnel, and assessment coordinators.

>From a thematic analysis of the results, six factors emerged as critically
important followed by four other factors that were classified as extremely
important. The six critically important factors were: (a)
knowledge/experience of campus leaders, (b) trust, (c) opportunities for
dialogue/ collaboration, (d) leadership, (e) faculty engagement, and (f)
use of assessment results. The four factors that were classified as
extremely important were: (a) building campus knowledge, (b) having an
assessment plan, (c) having communication strategies in place, and (d)
having administrators engaged in assessment.

Knowledge/experience of campus leaders. It is critically important that
there are knowledgeable campus leaders who have experience in a variety of
assessment methods, particularly knowledgeable faculty leaders. A lack of
such leadership represents a critical barrier to a meaningful assessment
process. It is important to have leaders who have walked the talk. Their
knowledge and experience may foster trust in the process.

Trust. There are two facets to trust. First, it is critically important
that leaders of this process are respected and accepted by all campus
personnel groups. Trust in the leaders fosters comfort with the change
process, helps build a collaborative campus climate, and promotes buy-in
for the process. Second, one of the greatest fears is that assessment
evidence will be used in punitive ways against faculty. It must be obvious
and transparent to faculty that the results will be used in positive ways
for institutional and program improvement and not in punitive ways. Trust
is facilitated when this intent is put in writing on all assessment
documents.

Dialogue/collaboration. The presence of venues for dialogue,
collaboration, and sharing among campus groups was judged to be critically
important. Such venues help to overcome a significant barrier, that being
a lack of understanding and consensus among campus personnel of what needs
to be done. Dialogue and sharing promote a collaborative climate and serve
as the glue that builds a strong and lasting process. One aspect of
collaboration is the presence of an assessment team. By working together,
team members develop a shared message, trust, and momentum.

Leadership. A culture of assessment is facilitated when there is
knowledgeable and respected leadership. It is critically important that
faculty members are among the campus leaders in the assessment process.
Other faculty members are more likely to participate when lead by
knowledgeable, respected, and enthusiastic colleagues. Leadership from
administrators was classified in the extremely important range and
characterized as having a support role in assessment. Administrators give
assessment credibility and make allocation of financial and personnel
recourses more likely. Without their leadership, assessment programs may
be weak and wither away from lack of resources. The presence of a core
team broadly representative of the college and the continuity of dedicated
staff having lead responsibilities for assessment were judged extremely
important aspects of leadership. Without these conditions, it may be
difficult to implement assessment at the institutional and program levels.

Faculty engagement. Two personnel groups are particularly important to a
meaningful assessment process: faculty and administrators. Faculty
engagement emerged as critically important while administrator engagement
was judged to be extremely important. Engagement of faculty, including
adjunct faculty, is a driving force behind meaningful assessment. This
engagement is characterized by a willingness to learn about assessment,
analyze data, use results, and share what is learned.

Using assessment results. Among the critically important themes of a
meaningful assessment process is the use of results. The purpose of
assessment is to use the results to make more informed decisions to
improve education. If results are not incorporated into campus decision-
making, then student learning assessment work will be viewed as having no
value and being just a waste of time. One facet of using results is
institutional support for assessment research and data analysis.
Assessment needs to be properly researched and data correctly analyzed.
Most faculty members do not have the time or expertise to do this.
Building campus knowledge. Developing knowledge among college personnel
was judged to be extremely important. It is accomplished through faculty
teaching other faculty and through ongoing and consistent offerings of
high-quality, motivating educational opportunities for campus personnel.
An assessment committee could serve as a vehicle for acquiring background
knowledge, communicating it to the rest of the campus, and overseeing the
implementation of staff development activities.

Assessment plan. Having an institutional plan that is linked to the
college mission, is integrated into institutional policies and practices,
is manageable, and is periodically evaluated for its effectiveness was
rated extremely important. The plan provides a framework for the process,
a document for those seeking understanding about assessment, and a
rationale for budgeting. Also, writing an assessment plan can serve as a
collaborative activity.

Communication strategies. Keeping campus personnel informed about
assessment activities was judged as extremely important. It is very
important that these communications are timely and that they come from
respected faculty, vice presidents of instruction, and college presidents.
Having both a written assessment plan and an institutional philosophy were
important forms of communication, because they give structure and
direction to the assessment process.

Administrator engagement. As indicated above two campus groups, faculty
and administrators, emerged as most important to a meaningful learning
assessment process. Administrator engagement was judged to be extremely
important because, administrators position assessment as an institutional
priority and provide the necessary personnel and financial resources to
manage the work of assessment. However, as important as administrator
engagement appears to be, the results of this study indicate that faculty
engagement may be more important.

To subscribe to this Listserv:
Send an e-mail message to Listserv@Listserv.cccnext.net
Do not put any words in the subject area.
In the body of the email message, Mary Smith would write the following: subscribe LearningAssessment Mary Smith
When sending the message, substitute your first and last names in place of "Mary Smith"

To access the LearningAssessment Listserv achive:
Go to http://listserv.cccnext.net/scripts/wa.exe?REPORT&Z=3
Click on Server Archives (on left column of page)
Click on LearningAssessment
You will have a list of all the postings at this point to choose from

 

Comments (0)

You don't have permission to comment on this page.