Skip to content

V6I3: University of Western States’ Graduate Benchmarking Consortium

Susan B. Donoff, Senior Director, Appraisal & Accreditation, University of Western States 

Rachael Pandzik, Associate Vice President for Institutional Effectiveness, University of Western States  

Dana Sims, Provost, Accreditation Liaison Officer, University of Western States 

Dana Sims, Provost, Accreditation Liaison Officer, University of Western States (left); Rachael Pandzik, Associate Vice President for Institutional Effectiveness, University of Western States

In August 2019, NWCCU finalized new requirements for disaggregated indicators of student achievement in the context of and in comparison with regional and national peer institutions (NWCCU 2020 Standards 1.B.2, 1.D.2, and 1.D.3). With a continued commitment to student achievement and a Year Seven comprehensive evaluation report that was due in Spring 2023, institutional effectiveness personnel at University of Western States (UWS) investigated various existing sources of benchmarking data, including IPEDS, the University Benchmark Project (UBP), and the Postsecondary Data Partnership (PDP). These metrics focused on undergraduate student populations (IPEDS, PDP) and associated administrative/student services (UBP) rather than the student population UWS required for comparators.  

As a graduate-level health science institution, UWS required comparative data from like institutions. The Council of Graduate Schools and gradSERU offered post-baccalaureate benchmarking focused on traditional academic master’s and PhD programs, rather than clinical programs leading to licensure. In recognition of these limitations and to meaningfully address NWCCU standards 1.B.2, 1.D.2, and 1.D.3, in late 2020 UWS invited four additional graduate-level health science institutions accredited by NWCCU to form a regional benchmarking consortium.  

Phase one, drafting and executing a data sharing agreement among five independent institutions, was no simple task. UWS institutional effectiveness and risk management personnel collaborated internally to develop a template that covered confidentiality, points of contact, and basic data collection and usage procedures. UWS then circulated this draft externally to the stakeholders and signatories of the other four NWCCU institutions for edits and comments on the data sharing agreement. Once all parties agreed to the terms, UWS utilized electronic signature software to fully execute the agreement in March 2021.  

Phase two, the pilot data collection, occurred during the summer and fall of 2021. This labor-intensive process began with the establishment of a common data dictionary to account for differences in each institution’s portfolio of programs. Unable to effectuate meaningful program-to-program comparisons due to disparate academic offerings, the newly formed Graduate Benchmarking Consortium (hereafter referred to as The Consortium) elected to benchmark by degree type. Institutional stakeholders then assigned their programs to one of four categories: Academic Doctorates, Clinical Professional Doctorates, Academic Master’s Degrees, and Professional Master’s Degrees. Academic degrees differed from clinical professional degrees in that clinical degrees provided a pathway to licensure.  

This program identification process led to further conversations surrounding confidentiality/suppression, limitations in data availability due to set configurations in student information systems, and variations in institutional reporting calendars. An overarching concern of The Consortium was student confidentiality, due to relatively small cohorts in certain disaggregated racial/ethnic groups, age ranges, and gender identities. Utilizing a ‘students of color’ designation afforded an opportunity to provide meaningful comparison through re-aggregation rather than data suppression (invoked when any individual category included fewer than five students).  

Disparate student information systems (SIS) and institutional reporting calendars proved challenging, but not insurmountable. As a key repository for matriculant data, an institution’s SIS software generally has fixed parameters to reliably collect and maintain pertinent aspects of academic records. A common example of an SIS limitation is the binary choice of male or female for gender identity. Some members of The Consortium were able to report a broader range of gender identities, while other members were limited to male or female. In this instance, no meaningful comparison could be made beyond male and female gender identities. A further complication related to academic calendars—namely, how to reconcile quarter, trimester, and non-traditional terms. To address this, The Consortium resolved to report data based on a fiscal year, in alignment with IPEDS guidelines.       

Utilizing the shared data dictionary, The Consortium entered data disaggregated by race/ethnicity, gender, and age range for each program’s 100% graduation rate, 150% graduation rate, and 1-year retention rate into a shared Excel spreadsheet maintained by UWS. Each institution benchmarked their individual data against the aggregate peer data provided by the other members of The Consortium. In compliance with NWCCU Standard 1.D.3, which directs institutions to publish benchmarked indicators, UWS created a student achievement website. This dedicated webpage provides transparent, anonymized, contextualized information for university stakeholders, including current and prospective students, senior leadership, and members of the admissions staff. A green check mark indicates areas where UWS outperforms the benchmark; a red downward arrow denotes areas where UWS falls below the benchmark. The full report is reviewed by the UWS Institutional Effectiveness and Planning Committee (IEPC). The committee makes recommendations to executive leadership regarding actions necessary to address equity gaps and improve institutional outcomes. Though this is the strategy employed by UWS, each member of The Consortium is responsible for developing its own strategy for publishing and analyzing and responding to the data.    

In 2022-2023, the universities amended the agreement to include two additional graduate-level health science institutions accredited by NWCCU, and completed a second, more refined round of regional benchmarking. UWS facilitates the regional benchmarking consortium by scheduling meetings, capturing meeting minutes, and coordinating the annual data collection process. As evidence of compliance with NWCCU standards 1.B.2, 1.D.2, and 1.D.3, UWS included copies of the data sharing agreement, meeting minutes, and the benchmarking report with notations from the IEPC review with its Year 7 Evaluation of Institutional Effectiveness report. These efforts garnered UWS a commendation in its official Commission letter for “the establishment of a data sharing agreement amongst other NWCCU-accredited peer institutions to develop common benchmarks against which disaggregated data on indicators of student achievement may be assessed to identify and close barriers to academic excellence and success.”  

Regional peer comparison exercises provide accountability and promote self-evaluation toward continuous improvement for student success. Though there are limitations in all benchmarking exercises, the insights provided by this annual data collection cycle assist both UWS and its peers in formulating realistic goals for closing institutional equity gaps. Forming a data sharing consortium comprised of peer institutions represents an innovative approach to attainment of NWCCU standards, and a commitment to analyzing disaggregated graduation and retention rates to inform university planning. 

Sign up for our Newsletter

Explore the latest issue of NWCCU’s quarterly newsletter, The Beacon, and discover educational resources, innovations, and events happening in the Northwest.