This document was prepared by Patricia M. Shields (MPA Director, Southwest Texas State University) for use as a handout at her October 2002 Accreditation Institute session at the NASPAA annual conference. The paper is intended to provide insight and suggestions for conducting employer assessments as part of the self study process. It is based on her personal experience and is not issued or endorsed by COPRA. COPRA’s “Self Study Instructions” ( http://www.naspaa.org/accreditation/seeking/reference/instructions.asp) provide the specific instructions and directions for including employer (and other external constituency) assessments in the self study. (click here to download this document in .pdf format)

The Roles of External Constituencies in Mission Based Accreditation 

1. Mission development and revision (Standard 2.1)
2. Program change decisions (Standard 2.3)
3. Program assessment (Standard 2.2)
4. Program activities (Standard 5.3)

External constituencies include:

• Students 
• Alumni 
• Employers 
• Professionals in the field
• University members outside the program 
• Other

Advisory Council (have systematic discussions)

1. Mission development & revision 
2. Program change decisions

Mission Development and Revision : Using an Advisory Council 

Example of MISSION PROCESS
In January of 1998, the SWT MPA Program held a day long Mission Retreat. At that time a Mission Statement was developed. During the morning session, 20 external constituencies were engaged in the process. Many of the participants were from the advisory board. Others were from the larger community. We tried to ensure diversity by including an elected official and practitioners from federal, state and local government. We also included a mix of students, alumni, and non-alumni practitioners. African Americans, Hispanics, Anglos and a person with a disability were represented. The morning discussion consisted of brainstorming. We talked about the program’s place in the community, what its graduates should be able to accomplish and contribute to the public service. The competencies, knowledge and skills we should expect of a SWT MPA were also discussed. A trained facilitator led the event and the faculty mostly listened. Jeffrey Kauffmann, one of the participants and Director of Strategic Planning for the Texas Rehabilitation Commission, suggested that we develop a short easy-to-remember one sentence mission punch line to begin the mission statement. This is the rationale behind, "The MPA s Program's mission is to prepare students for careers as managers and leaders in the public service," which is the first sentence of our mission statement.

In the afternoon session the faculty processed the morning discussion and with the help of a facilitator developed an initial mission statement. The mission statement was further refined at a subsequent faculty meeting and later brought before the MPA Advisory Council for review and approval. 

The SWT MPA Program revised its mission statement during the 1999/2000 academic year. The process began as informal faculty discussions of the existing mission statement. A consensus emerged that the current mission statement did not reflect certain aspects of the program. In addition, further refinement was necessary to take into account trends in Public Administration. Namely, the mission statement did not include any reference to diversity, leadership, technology, or ethics.

The formal mission statement revision process began as a mini-retreat (2 hours) with the MPA Advisory Council during its regularly scheduled 1999, October 3rd meeting. There were approximately 25 stakeholders at the meeting including Public Administration professionals from state and local government, private consultants, faculty, and students. The group was a rich mixture including Hispanics, African Americans, Anglos, and men and women in their 50s, 40s, 30s, and 20s. We drew upon a trained facilitator (Joy Boone) from The Department of Health Services Research. During the meeting, members of the Advisory Council discussed the importance of diversity, leadership, technology, and ethics in public administration. Key ideas from the discussion were incorporated into a revised mission statement that was formally revised during faculty meetings.

Example of Program Change: Using an Advisory Council

MPA Advisory Council: The MPA Advisory Council has been active for 12 years. The Council is composed of 19 practitioners and five students. Most of the professional members are alumni. Faculty also attend the three annual Saturday morning meetings. The Advisory Council provides both formal and informal feedback to the faculty. The Council helped us to revise the mission statement, consider admission policy, and adopt curriculum change.

The Advisory Council also has formal subcommittees that provide assessment and sometimes even implement program improvement. The three committees are Program Trends, Mentoring and Student. Currently, the Program Trends subcommittee is working on further developing the internship program. It is expected that the internship program will grow as the first undergraduate Bachelor in Public Administration students complete their degree. Preliminary evidence suggests that many are interested in an MPA at SWT. The Mentoring subcommittee plans mentoring events and suggests improvements for the next event. The Student subcommittee is responsible for student orientation (including assessment, recommendations, and implementation of the suggested improvements). For example, attendance at previous student orientation session was low. A new format was proposed and implemented. The student orientation time was moved to immediately after the Graduate College Orientation (Tuesday night before the fall classes began). Attendance tripled at the orientation. As with other orientations, faculty and several Advisory Council members were present at the event. (See Appendix B.)

Examples of Program change supported by Advisory Council 

• In response to faculty curriculum review and Advisory Council feedback, Political Science 5351 Seminar in American Political Institutions was eliminated as a background requirement.
  
• In response to faculty curriculum review and Advisory Council feedback, Political Science 5321 Problems in Federal Intergovernmental Relations was renamed Introduction to Public Policy and Administration and the course description was also changed.
  
• In response to alumni survey results, and discussions in the Advisory Council and faculty meetings, the faculty decided to be more explicit in applying technology across the curriculum.
  
• In response to Advisory Council assessment, the student orientation format was changed.
  
• In response to Advisory Council assessment, a "mentor directory" was provided at the January 2000 mentoring mixer.

Program Assessment in Mission Driven Accreditation:
Using feedback from external constituencies

Example


Southwest Texas State University
Master of Public Administration Program

PROGRAM MISSION 
The MPA Program's mission is to prepare students for careers as managers and leaders in the public service.

The program primarily serves the diverse community that is Central Texas. Such service is provided through course work, professional development opportunities, internships and applied research projects.

The Program responds to the changing public service environment through: an innovative curriculum, faculty scholarship and service, links to professional organizations, interaction with governmental agencies, and a commitment to state-of-the-art technology.

The Program is distinguished by emphasizing the central role of ethics in public service; reinforcing the use of technology in management; providing professional and educational opportunities to a diverse student body; delivering classes at convenient times and locations; offering a variety of career support areas; enabling rich and frequent contacts between students and faculty; providing students and alumni with professional networking opportunities; focusing on continuing professional development; emphasizing management in political institutions and processes; and integrating theoretical and applied approaches to public management.

Deconstructing the Mission Statement
Linking Mission to Assessment

*The element was linked to a strategic planning objective/strategy (See Appendix A for examples of the explicit link between mission elements and strategic planning objectives.) Both Objectives and Strategies are explicit in the Strategic Plan that is presented in the appendix of the Self-Study.
** CenTex is the local chapter of the American Society for Public Administration (ASPA).
*** See Appendix C. Beck (1993); Almaguel (1997); and Gute (1999)
**** See Appendix D.

Assessment procedures: Examples
Sources of Assessment Information Using External Constituencies 

Alumni Survey: In February of 2000 Cindy Cavazos (as part of her ARP) distributed an alumni survey to all alumni over the past 8 years. She used the mission elements, NASPAA standards and a previous Alumni Survey to develop her survey instrument. Formal consideration of survey findings occurred in May during the Oral Examination. The Graduate Dean, the Chair of Political Science and five faculty attended the examination. During the summer, results were shared and discussed at the Advisory Council meeting. The results indicate that the Program is doing a good job meeting its mission. (See Appendix E for selected Alumni Survey results.)

Interview at Oral Exam: At the conclusion of the oral examination students are asked to give their honest impression of the program. At present, the committee asks the student to identify the program’s strengths and ways that it could be improved. 

MPA Program Exit Survey: After the oral exam, the students are asked to complete an exit survey. The survey included specific questions about the curriculum, how the MPA Program contributed to students' knowledge and skills. In addition several questions are specifically tied to the mission elements. (See Appendix I.)

Employer Interviews: Employers were interviewed during the spring of 2002 to determine whether the MPA Program curriculum and graduates met employer expectations. MPA student Bruce Hermes, as part of his capstone project, conducted the survey. (See Appendix F for an abstract, conclusions and recommendations.)

Graduate Program Self-Assessment Exit Survey The Graduate College and the Office of Academic Assessment use a respected nationally designed instrument to collect assessment data. Students complete the survey as part of the process of filing for graduation. The questionnaire is comprehensive. The most recent published results are for 1997-1998. (See Appendix G.)

MPA Advisory Council: See above 

Part of the MPA Program's mission refers to “professional development “and “networking” between “students” and “alumni”. Every Advisory Council activity supports these Mission Elements. During the July 2002 meeting of the Advisory Council, a draft report of this Self-Study was discussed. The Council made suggestions for change and enthusiastically endorsed the draft.

Centex Activities: Note that part of the SWT MPA Program mission focuses on “professional development,” “networking,” and “alumni involvement.” The SWT MPA Program uses the Central Texas (CenTex) Chapter of the American Society for Public Administration to achieve many of these mission objectives. Approximately 80% of CenTex members are SWT alumni, faculty, or students. Approximately two thirds of the CenTex board members are faculty, students, or alumni of SWT. (See http://www.main.org/aspa/officers.htm for a list of CenTex officers.)

CenTex events create opportunities for networking and professional development. Three events merit note: (1) Public Service Recognition Week Banquets; (2) The Texas ASPA Conference; and (3) the mentoring mixers. These events are assessed at board meetings. Generally, recommendations for improvement are implemented.

CenTex has won several awards from the American Society for Public Administration. The Newsletter, edited by a SWT alumnus, has won three national awards. The Texas ASPA Conference also won an award, as did the Strategic Planning activities of the chapter. Student and former CenTex president, Bill Billingsley, won the ASPA Mode Scholarship. Professor Howard Balanoff won the prestigious, Stone Award for service to ASPA. These awards are evidence of "leadership in public service." 

CenTex/Advisory Council Collaboration: Many events that promote professional growth and networking are co-sponsored by CenTex and the SWT MPA Advisory Council. The mentoring mixer (offered at the beginning of each long semester) has drawn as many as 100 students, alumni and other professionals. The mentoring subcommittee of the Advisory Council takes the lead on organizing and assessing the mixer. CenTex pays for expenses like the hors d'oeuvres and promotes the event through its newsletter and web page. The process of collaboration works because there are three to five people that serve on both the MPA Advisory Council and the CenTex Board.

Capstone Projects: The Applied Research Project (ARP) is the SWT MPA Program's capstone project. The ARP is a cumulating experience and demonstrates in a concrete and summative way the analytic, policy, organization and writing skills of the student. The Applied Research Project process is a multidimensional assessment mechanism. First, there is a transparency to the process that allows for broad assessment and program feed back. ARPs are defended before a three-member committee. The SWT library catalogues all ARPs (each has a unique call number). Titles of ARPs (since 1992) are available on the MPA web site as are more recent abstracts. http://www.polisci.swt.edu/public_administration/applied_research_projects
/index/ARP_index.html Index of Applied Research Projects since 1993.

http://www.polisci.swt.edu/public_administration/applied_research_projects/
abstracts/arpabstracts.html Index of selected Applied Research Project Abstracts since 1999.

Secondly, students often perform program assessment activities as part of their Applied Research Projects. For example, there is an employer assessment of the MPA Program, an alumni survey, and several studies assessing the quality of ARPs. When student projects are used to assess the program we ask SWT administrators to sit on the oral committee. The result is a multidimensional assessment. Student performance (both written and oral) is assessed. The committee members are also able to provide a comparative frame of reference (How does the MPA Program compare to other SWT Graduate Programs?). In addition, the program is able to collect and share assessment data with a key external constituency –university administration. In this capacity the Deans have found the projects useful as a form of institutional assessment. (See Appendix C for a list of the capstone projects used to assess the program.)

Third, the oral examination committee structure serves as an assessment mechanism. Committee members often come from outside the MPA Program. This opens our Capstone Projects to broad outside scrutiny. Committee members include faculty from other departments as well as administrators such as deans, chairs and vice presidents. Lately, we have opened the oral examination process to practitioners. We have had the Chief of Staff for the Lieutenant Governor, the Chief of Police of the Austin Community College, the City Manager of San Marcos, a San Marcos City Council Member, librarian, Air Force Colonel and many other practitioners serve on committees. The oral exams are also now often held at San Marcos City Hall and in a Senate Conference room in the State Capitol. This practice ties to other mission components including “interaction with different agencies” and “networking opportunities.” 

Fourth, we submit the Applied Research Projects for awards. Awards provide a comparative frame of reference.

Appendix J
Summary: Linking Mission and Assessment Data

Table J.1
Linking Mission and Assessment Data

Appendix I
Exit survey: given to students after their oral exam

These questionnaire items were drawn directly from the Mission Statement and the numbers correspond to the mission element numbers that are in Mission Statement Deconstruction Table.

Click here to download this document in .pdf format, 16 pages.