Core Requirement - 2.5 The institution engages in ongoing, integrated, and institution-wide research-based planning and evaluation processes that incorporate a systematic review of programs and services that (a) results in continuing improvement and (b) demonstrates that the institution is effectively accomplishing its mission.
(Institutional Effectiveness)

Judgment of Compliance:  Compliance

Narrative/Justification for Judgment of Compliance:

At Southwest Tennessee Community College, the principles of institutional effectiveness continuously guide institutional management, operations, and services. Operationally, “institutional effectiveness” means at Southwest that every unit and program must determine and follow appropriate goals and standards, assess and analyze the quality of its results, and use those results to improve services, future goals, and even the organization itself.

Southwest’s approach to institutional effectiveness is described below in four parts: (1) institution-wide planning, and evaluation processes, (2) integrated planning and evaluation cycle, (3) continuous improvement, and (4) achievement of mission.

1. Institution-Wide Planning and Evaluation Processes
   1.1 Strategic Plan
   1.2 Performance Funding Program
   1.3 Institutional Effectiveness Plan
   1.4 On-Line Planning System
   1.5 Linking Unit Activities to Institutional Plans
    

2. Integrated Planning and Evaluation Cycle
    

3. Continuous Improvement
   3.1 Evaluation Process Perspective
   3.2 Planning Perspective
   3.3 Resources Perspective
   3.4 College Culture Perspective
    

4. Achievement of Mission
   4.1 Effective Academic Programs
   4.2 Effective Student Support Services
   4.3 Effective Administrative Services
   4.4 Quality Personnel
   4.5 Current Technology
   4.6 Continuous Planning, Evaluation, and Improvement

1. Institution-Wide Planning and Evaluation Processes

In practice, Southwest applies four college-wide programs to support and guide institutional effectiveness at all organizational levels. Each program supports planning, budgeting, evaluation, and improvement processes.

1. Strategic Plan. Five-year goals updated annually to guide college activities toward desired institutional development (focused upon four institution-wide goals and several area-specific goals)
    

2. Performance Funding Program. Five-year state-wide and college performance goals assessed annually to determine and improve academic quality (coordinated with strategic plan)
    

3. Institutional Effectiveness Plan. Annual plans and evaluations to guide unit operations and improvements (coordinated with strategic, budget, and performance funding plans)
    

4. Online Planning System. Internet-accessible database to record all annual plans, institutional effectiveness reviews, and budgets for all college units at all levels.

These institution-wide programs strongly influence the direction and operation of the college through planning, budgeting, assessment, and documentation processes. Understanding the substance of each is necessary to understanding the full scope of institutional effectiveness at Southwest.

>>back to top

1.1 Strategic Plan.

The college follows the Tennessee Board of Regents five-year strategic planning cycle. Prior to each cycle, the college and the governing board review the college mission, local and state needs, and long-range goals. College recommendations on the institutional mission statement and strategic institutional development goals are reviewed and authorized by the governing board. The current strategic plan was approved in 1999 for the 2000-01 through 2004-05 cycle. While the college may submit updates to TBR annually, the next five-year planning activity will take place during 2004 05.

The current college strategic plan emphasizes four institution-wide long-range goals. These broad goals drive all planning across the college:

Table 1. Institutional-Level Strategic Goals

Additionally, the strategic plan contains strategic goals distinctive to each of the major administrative areas of the college. Each area-specific strategic goal is derived and linked to each institution-wide goal. [1] Each unit develops annual objectives to support the goals in its respective administrative area.

Table 2. Strategic Goals for Area of Academic and Student Affairs

Table 3. Strategic Goals for Area of Business, Finance, and Information Systems

Table 4. Strategic Goals for Area of Institutional Advancement

Table 5. Strategic Goals for Area of Workforce Development and Continuing Education

Extensive reviews and evaluations are conducted under each Performance Funding standard. Most evaluations are conducted by the college. The employer survey was monitored by an external THEC committee as part of a statewide pilot project. The report on "Assessment Implementation" was reviewed and scored by a THEC-selected committee concerned about institutional use of Performance Funding findings. Key measures of annual Performance Funding analyses include the following:

Standard 1. Academic Testing and Program Review

1A-1B: General Education. Annual assessments of general education outcomes by graduates revealed persistent lower-than-desired student performance (See Table 7). In the past two years, Southwest’s scores have dropped significantly lower than the Tennessee reference group of comparable two-year colleges.

Table 7. General Education Outcome Scores From College Base Test

Meetings were held among academic departments teaching general education courses, General Education Committee members, and testing personnel to discuss these concerns in 2002. An ad hoc committee was charged in July 2003 to examine the College BASE general education exam and make recommendations to improve student scores. Armed with data from several semesters of pilot testing alternative exit exam options, the review of the data led to three long-term actions:

(1) Pilot study of general education exit tests to improve testing validity

(2) Departmental curriculum changes to improve instructional methods

(3) College-wide updating of all academic program outcomes and testing methodologies.

The committee has made specific recommendations to effect improvements. The College is implementing a two-subject version of C-BASE, thus reducing testing time from 180 minutes to 90. The intention is to improve student motivation, improve the confidence level of results and provide more decision-making data than from a one-test version.

The committee recommended the creation of a database of C-Base scores merged with student demographics to understand characteristics that may affect student scores. Another recommendation is to improve the testing environment to create a positive environment for student motivation and success. In addition, the College should communicate more effectively the importance of the C-BASE results in order to more positively engage students and to reinforce with faculty the connection between learning outcomes and assessment success.

General education departments have begun to initiate actions to improve instruction. The English faculty have added literature components to English Composition I and II, and the Natural Sciences Department is actively engaged in identifying strategies to improve student performance. Science faculty agreed, for example, to add critical thinking questions to examinations and to teach textbook chapters on interdisciplinary science skills in all courses.

1C: Program Reviews. All academic programs conduct self studies and receive external reviews on a periodic basis. A total of 28 programs are reviewed by accrediting agencies, and 19 programs are reviewed by external peers. Low producing programs (graduating fewer than ten students per year) are reviewed by the governing board and the college. New programs are not subject to review until after five years of operation. All reviews result in actions to strengthen, merge, or terminate programs (see Tables 8, 9, and 10).

Table 8. Accredited and Non-Accredited Academic Program Reviews

Table 9. Academic Programs Not Currently Reviewed by College

Table 10. Low-Producing Programs Review

1D: Major Field Assessment. Graduates of all Associate of Applied Science degrees are required to complete a major field test upon graduation. Some programs conduct tests developed by the academic department and approved by THEC (for example, most degree programs in business, technologies, criminal justice, and education). Other programs utilize professional licensure examinations (for example, nursing and the allied health sciences). Test results are reviewed and submitted to THEC on a five-year cycle.

Standard 2: Satisfaction Surveys.

Students, alumni, and employers are surveyed by the office of institutional research in rotating years to provide external insight into the quality of college programs and services. The need for this type of data is so important that the college has begun conducting all three surveys annually instead of rotating them over a three-year cycle.

The enrolled student survey and the alumni survey seek feedback on equivalent and comparable topics. The student survey obtains current perceptions, and the alumni survey obtains opinions tempered with two years of post-graduation experience. Example questions and responses are presented in Table 11 below.

Table 11. Selected Results From Alumni and Enrolled Student Surveys

The Employer Survey was conducted for the first time in 2002-03 as part of a state-wide pilot project sponsored by the Tennessee Higher Education Commission. The Career Studies department surveyed the most frequent employers of Southwest graduates through mail-out questionnaires and online forms. As a result, the department gained suggestions for improving its services, such as increasing face-to-face contact between employers and Career Services staff. Employers also stated that students should gain more hands-on experience and develop more communication/interpersonal and technical skills. Information gained from the survey was shared with all academic departments.

Standard 3: Planning & Collaboration.

The Performance Funding program places special emphasis (including external review) on strategic efforts especially related to student learning. During each five-year cycle, the college identifies key strategic initiatives that relate closely to the college mission and state priorities. Important student-related services have resulted, such as expansion of online credit and non-credit instruction, web support for students, and expanded technology for support units.

Standard 4: Student Outcomes and Implementation

4A. Output Attainment. Each year, the college and the state review retention rates for current students and job placement rates for graduating students. At Southwest, fall-to-fall retention rates have been consistent except for an unexpected one-year downturn in 2002-03 (see Table 12).

Table12. Student Retention and Job Placement Rates

Southwest is generally consistent with statewide averages. The large drop in retention for the Fall 2001 cohort was unexpected at Southwest as well as across the state. One possible explanation is financial and not academic. Fall 2002 was the second year of double-digit fee increases due to state revenue shortfalls. Some students may have delayed their enrollment and opted instead for employment.

Even after the retention rate returned to a normal level, the college is monitoring retention closely. New retention strategies have been proposed in the Enrollment Management Plan. Once approved, the plan should provide proactive interventions for at-risk students before they drop out of school. Example enrollment management activities include implementing an early warning system to identify potential dropouts, increasing student involvement in campus activities, providing students with alternatives to withdrawal, and creating a “first year” program to support new students.

Job placement rates have been consistently high. Under the Tennessee Higher Education Commission’s annual Performance Funding review, the college is expected to maintain a job placement rate of at least 75% across all career preparation programs. Southwest has always exceeded that standard. The overall job placement rate for AAS and technical certificate graduates combined exceeds even the college’s self-defined standard of 90%: 2000: 94%, 2001: 95%, 2002: 92% [5]

4B. Assessment Implementation. Reports of institutional utilization of performance funding results are reviewed annually by THEC-appointed external review committees. Following the principle of institutional effectiveness, the committees assess whether colleges actually make use of the data collected and recognize the quality of college efforts. For 2002-03, Southwest received the full ten points possible for Standard 4B along with very positive reviewer comments, such as, “Their new 4.B goals… have to do with improving general education (not merely trying to raise scores, but actually trying to produce better educated students––hooray!) and building outcomes for programs…” [6]

>>back to top

1.3 Institutional Effectiveness Plan.

Every one of the 95 organizational units in the college develops and follows an annual institutional effectiveness plan. Southwest has completed three (3) full institutional effectiveness cycles since its consolidation took place in 2000. Prior to that, the precedent colleges (Shelby State Community College and State Technical Institute at Memphis) maintained their own institutional effectiveness plans and completed six years of institutional effectiveness cycles.

Southwest’s institutional effectiveness plans are published on the web-based online Planning System (discussed below) and are used to guide annual unit activities. Beginning with 2003-04, the format of the plan is composed of five major elements which are recorded in five columns. The following example in Table 13 uses information from the complete 2002-03 institutional effectiveness report by the Medical Laboratory Technician program to illustrate the contents of each column in the new format.

Both quantitative and qualitative measures are incorporated in annual institutional effectiveness plans and the evaluation of performance. For every intended educational outcome or administrative objective, the sponsoring unit must specify the criteria for determining successful results.

Quantitative measures include sources such as exit tests for graduating students, attendance records, quarterly statistical reports, logs of training sessions, federal performance reports, inventories, graduation rates, and pre/post market surveys,

Qualitative measures include sources such as customer comment cards, advisory committee reviews, satisfaction surveys, staff evaluations, descriptive reports, peer review comments, and participant feedback.

Table 13. Southwest Institutional Effectiveness Documentation System

In evaluating its own institutional effectiveness process, the college determined a need to improve the quality of unit objectives and the means of assessing achievement and effectiveness. Consequently, two major efforts took place during 2002:

1. Southwest created a new department, the Assessment Office, and initiated a national search for a director. The Director of Assessment was hired in fall 2002 and is now providing direct support to all areas of the college. [6A]

2. In the spring of 2002, Southwest implemented a online Planning System to simplify departmental requirements and reporting. In the spring of 2003 the online planning system was revised to match the “five-column model” described above. That model has been continued for fiscal year 2004-05.

>>back to top

1.4 On-Line Planning System.

Southwest maintains an online, Internet-accessible, database system to support annual planning, institutional effectiveness, and budgeting activities. Every organizational unit of the college is represented in this system. The system is available for public review via the internet address, http://plansys.southwest.tn.edu, and contains institutional effectiveness information for every fiscal year of Southwest’s existence (beginning in 2000-01).

Every organizational unit of the college has its own account, or “home page,” on the Planning System. On that web page, units plan and document their purpose, effectiveness, and budgets.

Following user feedback and institutional needs, the format and operation of the system has been upgraded nearly every year. The current (2003-04) version records four types of organizational information (see Figure 1 below for Testing Office example).

Figure 1. Unit Home Page on the Southwest Planning System

Unit/Program Mission Statement. Each unit publishes its purpose and scope of work in a mission statement that is coordinated with other unit missions in its area. Each mission statement is linked to one or two specific sentences in the institutional mission statement to indicate its relevancy to the institutional purpose. Additionally, each unit mission is linked specifically to one of the four institutional strategic goals to indicate
where it contributes most strongly to institutional development.

Proposed Budgets for Fiscal Year 2003-04. All accounts managed by the unit are linked to the unit home page. Once the unit’s annual plan (institutional effectiveness columns 1-3) has been approved, units can propose their entire operating budget for the forthcoming fiscal year. All budgets are justified by the scope of work described in the institutional effectiveness plan.

Institutional Effectiveness for Fiscal Year 2003-04. All institutional effectiveness activity is documented through this system. The unit home page displays the institutional strategic goal most closely supported by the unit. A link connects the user directly to the detailed institutional effectiveness plan. It is through this screen that departments can enter their annual objectives, assessment designs, results, and improvement actions (use of results). See Figure 2 below for a 2003-04 example from the Testing Office.

Figure 2. 2003-04 Institutional Effectiveness Summary Screen on Planning System

>>back to top

1.5 Linking Unit Activities to Institutional Plans.

The database technology of the Planning System enables every unit of the college to link and be integrated with the institution’s mission statement and strategic plan. Unit mission statements and unit objectives and results are expressly related to institutional-level planning documents. This clarifies exactly how units support the institutional mission and plan.

Additionally, the database links allow college management to generate automatic reports that list all unit objectives and results under their respective institutional and vice president level strategic goals. In this way, annual planning and achievements are related and reported clearly in support of long-range institutional goals.

During the planning stage of the annual Effectiveness-Planning-Budgeting Cycle, each unit updates its mission statement to reflect its principal purposes and scope of work for the forthcoming fiscal year. Units also use the point-and-click approach to select one or two institutional mission statement sentences that most closely guide the role of the unit. This identifies the unit’s “home” in the institutional mission statement. Units also select which of the five institutional-level strategic goals most closely relates to the unit mission. This link allows the college to see which units provide support to each of the strategic goals. These connections are illustrated in Figure 3 below.

Figure 3. Linking Unit Mission to Institutional Mission and Goals

Also during the annual planning stage, each unit links each of its annual educational outcomes or administrative objectives to one of the strategic goals for its vice president area. That vice president’s goal, in turn, is linked directly to the president’s annual priorities and the president’s priorities are linked to the institutional-level strategic goals. The result of this string of linkages is that the college can monitor how every strategic goal is being implemented—which units support it, what objectives relate to it, and what results contribute to it. The linkages work equally well in reverse. Every unit can clearly plan how it will support each of the president’s priorities, the vice presidents goals and institutional strategic goals. These linkages are illustrated in Figure 4.

Figure 4. Linking Unit Objectives and Results to Institutional Goals

>>back to top

2. Integrated Planning and Evaluation Cycle

To implement consistent and effective planning and evaluation processes, Southwest integrates the programs described above through a carefully coordinated schedule of activities (the annual Institutional Effectiveness-Planning-Budgeting Cycle) [7] and a single system for documentation (the Planning System described earlier). The cycle follows three key principles:

• Prior fiscal year data drives planning

• Planning drives budgeting

• New fiscal year activities are accountable, evaluated, and improved

Accordingly, assessment data are compiled prior to and utilized fully by planning processes at all organizational levels. Operational goal setting and planning is conducted on the basis of factual information and is completed prior to budget planning. Budget planning follows the planning stage and serves as the allocation of resources to support the achievement of goals. The chart below summarizes how Southwest has organized and scheduled its planning and evaluation activities. Most activities are coordinated with the college’s July-June fiscal year.

Figure 5. Annual Institutional Effectiveness–Planning–Budgeting Cycle

The Assessment-Planning-Budgeting Cycle is implemented through the schedules of the key planning and evaluation programs. Typical annual schedules are shown below.

>>back to top

3. Continuous Improvement

“Continuous improvement” requires that a college has the ability and willingness to determine needs and implement change. Southwest has the advantage of many ongoing and long-range forces and resources for assessment and improvement.   

3.1 Evaluation Process Perspective. Southwest follows or is developing many large-scale evaluation
     programs:

Performance Funding Program. Southwest participates in an annual statewide program of extensive evaluation of academic programs and student performance. The college and external agencies assess general education outcomes, academic program quality, student satisfaction, goal achievement, student retention, job placement, and the use of evaluation information to improve student learning (see Table 1).

Academic Program Accreditations or Peer Reviews. All Southwest degree programs are reviewed on five to ten year cycles. All accreditable programs are accredited and follow accrediting agency schedules for self-studies and external reviews. All mature non-accreditable programs (operating for at least five years) are reviewed on a five-year cycle by external peers or the Tennessee Board of Regents (see Tables 3, 4, and 5 above).

Low Producing Program Reviews. Southwest participates in annual statewide reviews of programs producing less than ten graduates per year. The 2002-03 review by the college and the Tennessee Board of Regents resulted in major changes to the curriculum (see Table 5 above and Table 14 below).

Table 14. Low Producing Programs Reviewed by College and TBR