V3I3: NWCCU Quality Culture Project: From Invitation to Inclusion

By:

Alana Hoare, Quality Assurance and Accreditation Liaison Officer, Thompson Rivers University

Pamela Goad, Executive Vice President, NWCCU

In the November 2019 Beacon we wrestled with the seemingly disharmonious fundamentals of accreditation: accountability and improvement. In March 2020, we introduced two potential interventions to resolve this conflict: 1) contextualized, qualitative performance indicators; and, 2) leadership strategies that promote collaboration and trust. In June 2020, we explored early findings from the NWCCU Quality Culture Project – an initiative aimed at understanding how higher education stakeholders in the Northwest region understand quality assurance (QA) processes; what underpins their views; and, the internal and external factors that may curtail the positive intentions of assuring institutional effectiveness and student success. Since then, the project has continued to evolve, as studies aiming to understand humans often do.

North American higher education performance measurement systems privilege quantitative approaches to knowledge. Therefore, accreditors operate within a context enmeshed in the political policies and professional standards of a postpositivist paradigm. This promotes a dependence upon less informative input and output metrics. How can the NWCCU and quality assurance practitioners balance multiple worldviews and advocate for epistemological diversity and inclusion? What principles and strategies would define a theory of action model for interculturalizing performance measurement systems?

For this edition of The Beacon, we want to delve into an interesting line of inquiry that emerged from focus groups and survey data in order to answer the research questions: What do we really mean when we say “invited to participate”? How do we move from “invited to participate” to “inclusion”?

Inclusive Leadership and Institutional Effectiveness

Learning (and leading) is a relational and reciprocal process.

Inclusive leadership, a form of distributed leadership, offers a refreshing alternative to patriarchal hierarchies by advocating for a more circular structure lead from the center (as opposed to top-down). Broadly defined, inclusive leadership is the collective capacity for relational practice, which includes collaboratively expanding networks, fostering partnerships, consensus building, and engaging all (Wasserman, 2015, p.335). A key premise of inclusive leadership is open communication whereby information is shared freely (as opposed to being reserved for the privileged few). Inclusive leaders abhor restricting access to information because the risk of creating a caste system that isolates people with non-positional leadership authority is high in organizations that limit information-sharing (Helgesen, 1995). Spitzer (2007) argues that open communication is an essential component of successful performance measurement systems, because without candor, collaboration, and transparency, we cannot gain deep insight and wisdom from performance measurement data.

An inclusive leadership model of governance has been identified as one of the best practices to support equity, diversity, and inclusion in higher education institutions (Shaibah, 2020), in part, due to its multiplying effect. Research has shown that one’s “experience of inclusion often creates the desire to be more inclusive” (Cox, 2020, p.273), thereby satisfying a need for sense of belonging. Unfortunately, many higher education institutions are mired by hierarchical decision-making structures. This is particularly common with decision-making committees, in which the chair holds authority and controls the flow of meetings. Largely unidirectional and transactional communication strategies prevail, which limits committees’ capacity to engage in critical inquiry and dialogue. This can be attributed, in part, to the constraints of Robert’s Rules of Order (Robert, 2020), which was developed under the assumption that a productive committee meeting is orderly. However, while structured meetings facilitate decision-making, often overlooked is the necessity for critical inquiry, deep listening and contemplation, diverse perspectives, and asking difficult questions (Dubb, 2020), which can be a messy process.

Inclusive leadership offers an approach for including underrepresented and underserved groups. It provides a means for addressing equity gaps and supporting transformative evaluation through participatory, emergent, appreciative, and catalytic activities. As inclusive leaders, we strive to engage “educators and students as change agents as they investigate and gather data on their community, its cultures, and its problems” (Furman, 2004, p.227). This reframes how we view students as consumers (Brown, 2015; Cardoso et. al., 2013) to instead as primary researchers. It also requires that we step aside as Indigenous stakeholders take the lead in sense-making and data interpretation “to ensure the richness, subtlety, and nuance of meaning are not lost in translation” (Wehipeihana, 2019, p.372). This call to action – greater “understanding, analyzing, and attending to the many circumstances that make every evaluation unique, including culture, stakeholders, and context” (Canadian Evaluation Society, 2018, para. 10) – is something that program evaluation scholars advocate for as a means for contextualized, culturally appropriate performance measurement systems.

Figure 1: Ratio of Participants by Institution-Type {Note: In some instances, a respondent identified with two position-types (e.g., Provost and ALO). As a result, the sum total for administrators is overrepresented in this graph.}

Kirkhart (2010) argues that context is at the heart of good evaluation, as such, seeking to understand cultural location is a principle of multicultural validity. The premise of her argument is that validity requires congruence between theory and context. Kirkhart (2010) emphasizes this point by contrasting past practice in which theory was positioned by a “white, male, heterosexual, academically educated, Eurocentric majority context… that is, the invisibility of majority privilege” (p.402). Alternatively, methods that prioritize strengths-based approaches to evaluation, and situational responsiveness (e.g., Indigenous cultural ceremonies, and relational and reciprocal approaches) offer a much needed substitute for the pervasive deficit view of underserved and underrepresented learners, a perspective that attributes failures (e.g., lack of achievement: retention, persistence, graduation) to an individual or group deficiency (such as a lack of effort), rather than to failures/limitations of social systems.

The new NWCCU Standards for Accreditation focus explicitly on closing equity gaps; therefore, it requires that we investigate current understandings of “participation” and “inclusion” and the gap between our perceived expectations of participatory governance and inclusive institutions. In the next section we will outline the research methodology from which the following research questions emerged:

1. What do we mean by “invited to participate”?
2. How do we get from “invited to participate” to “inclusion”?

What follows is an investigation into North American higher education administrators, faculty, and students’ perceptions of participation in activities that support institutional effectiveness.

Research Methodology

The research followed a constructivist mixed method design that emphasized the collection of qualitative data. The results of a literature review informed the development of faculty and administrator focus group discussion questions (Phase 1: Mar., 2020). Following an inductive, emergent approach, preliminary analysis of the qualitative data lead to the development of a survey (Phase 2: Sep. – Oct., 2020). Finally, a third phase (Nov. – Dec., 2020) sought to broaden participation to include the perspectives of students through focus groups.

During Phase 1, 37 participants representative of four-year private, four-year public, two-year public institutions, and tribal colleges offered invaluable perspectives on accreditation practices. Participants in Phase 1 included: ALOs, faculty, institutional research staff, vice presidents, and provosts. In Phase 2, 198 participants representative of four-year private, four-year public, two-year public institutions and, tribal colleges contributed their voices to the project. Participants in Phase 2 included: presidents, provosts, board chairs, chief financial officers, ALOs, and faculty senate chairs. Finally, Phase 3 included 14 students, representative of four-year private, four-year public, and two-year public institutions. Approximately 250 individual participants provided data to the NWCCU Quality Culture Project (Figure 1). What follows is an analysis of one theme that emerged prominently from the data: Participation.

Results and Discussion

Figure 2: Response to “At my institution, students, faculty and staff are invited to participate in accreditation processes.”

The first thing that jumped out at us when we looked at the quantitative results from the Fall survey (Phase 2) was the overwhelming agreement with the statement: “At my institution, students, faculty, and staff are invited to participate in accreditation processes”. Aggregated results across institution- and position-type showed that, out of all 20 Likert questions, this one reflected the highest level of agreement (mean = 1.68) and the lowest standard deviation (0.78), as shown in Figure 2.

However, when asked “Whose interests are considered in goal-setting at your institution?”, it was not always clear that those invited to participate are also those whose values and interests are given priority. This is what some administrators had to say:

The Board of Trustees, President, and Executive Cabinet members interests are primary and the expectation is that evidence of performance or quality must prove value on their terms. (ALO, 4-year public)

I believe administration places student interests first. However, I believe, especially senior faculty, tend to subconsciously embed their own interests in their claims about student interests. I believe those faculty would say the same about administrators. Our junior faculty give me hope. (ALO, 4-year public)

Greater emphasis is placed on consumer (political is part of consumer). Academic considerations are done in light of consumer priorities. Executive leadership takes ownership in goal setting either through the goal setting process or the ongoing level of support provided to achieve goals. (ALO, 2-year public)

The interests of all college stakeholders are taken into account when setting institutional goals. Academic values are given priority. However, we have not discussed the cultural valence of those values – that is, we generally consider academic values to be universal or culturally neutral. (Provost, 4-year private)

Additional interest groups that respondents identified as being given priority included:

• Students (the highest number of references at 101, which accounts for 51% of respondents. If you include students as “consumers” it jumps to 70%)

Students are at the core of all goal-setting. The college tries to look at what students need. “If you do what’s best for students, you probably won’t be wrong.” We’re referring to student success overall, we don’t really look at subgroups. (2-year public, President)

• Academic (noted by 81 respondents – 41%)

I’ve observed that our shared governance model tends to put faculty first, students second, community third. This creates a tension that may be holding us back from meeting our community’s expectations. (President, 2-year public).

• Economic and Consumer (noted by 75 respondents – 38%)

We explicitly state that our focus is on serving the interests of our rural service area, focusing on students, families, and local community and business partners. I do believe we keep that focus in mind with goal setting, but frequently goals are driven more by immediate financial crises and putting out ongoing fires. (ALO, 2-year public)

• Culture and Community (noted by 73 respondents – 37%)

Student interests are paramount; the local first nations’ values and cultures as underpinnings to our strategic goals. (ALO, Tribal College)

Curious to better understand how institutions learn about the interests and values of students, we held student focus groups (Phase 3). Among the questions, we asked students:

• Do you have opportunities to tell your story and explain the challenges and opportunities you experience?
• Have you seen your feedback used to inform improvements to academic and co-curricular programming?

While the number of student participants was small, representing just 6% of the total number of participants in the NWCCU Quality Culture Project, they astutely illuminated varying levels of participation and contribution to institutional decision-making:

Our school doesn’t have a lot of opportunities to explain our personal stories other than the interview/application process, which was very limiting. The options that we do get are through our student government, which puts on a lot of events where we can go and talk… I suppose some of us have opportunities to meet with deans once a month through student government. (Student, 4-year private)

I work for campus life at the moment… So, I would say I probably do get a little bit more active engagement than the typical student would, but part of that might also be students not seeking out opportunities to talk about their challenges and opportunities because, one of the things we struggle with is actually contacting students and getting in touch with them; especially, when everything has gone remote. So, I don’t know if it is a limit as much as maybe an ignorance of students about the opportunities that students have. (Student, 4-year public)

I don’t think that there has really been focus groups or anything like that conducted, that I know of. It is more personal. So, between me and my academic advisor, and being able to establish a relationship since my first year with them. So they know when I came to college what my career goals were, what challenges I would face, what going to college meant for me as a first-generation college student. For them to understand that and see what resources I needed in order to get through all of the barriers that I would face in college, was really helpful. (Student, 4-year public)

I’m not in Senate, I’m just an RA at the dorms here, but yeah, I don’t feel like there’s a whole lot of opportunities to share our experiences and what not. The closest thing I guess we might have would be course evaluations, but again, that doesn’t really tell the full story, that just kind of tells how you felt about the class. So, I think colleges could definitely use something more to tell your story. (Student, 2-year public)

I’m part of my college’s Senate, so I’m one of the senators. It means I get a lot of opportunities to tell my story and to influence the things that happen at [the college]. So, what if I weren’t in Senate? Would I have as many opportunities? I would say not very many. I would, but I don’t know because I’m in Senate, you know? (Student, 2-year public)

I would say, as a mostly distance learner, I have a ton of opportunities to tell my story. From the course-level to the program-level to institutional-level. Like, program-wise, we have a program-wide open forum every two weeks. For courses specifically, we have surveys throughout. So, like mid-semester: How’s it going? How are you doing? How can we make this course better? Then, like institution-wide, we have our student Senate, which has representatives from each program and feeds information forward from every student. (Student, 4-year private)

Taken together, participants’ responses across the three phases of the NWCCU Quality Culture Project demonstrate varying levels of participation that, we argue, can be placed along a continuum. Using the language that emerged from participants’ responses, we suggest that these levels of participation can be represented along a six-level continuum (Table 1).

Table 1: Levels of Participation

[1] Note. This 2-year public President went on to say: “Efforts are made to include the student voice, but it’s not particularly successful. Students are invited to attend various meetings, but may not always get a good explanation of why they are invited and what they are expected to contribute. They rarely speak up. We perhaps need to be more mindful when seeking to involve students in college decision-making.”

Note. The descriptions are written from the perspective of an institutional stakeholder, in particular, a student.

Note, it is not our intent to make a judgement about the value of any one level, but rather to draw attention to the variety of levels that exist within institutions and to recommend that institutions consider which levels they may be privileging at the expense of others. Ideally, we suggest that a balance of the six levels be present at all institutions to ensure that the interests of stakeholder groups are accurately represented.

There is an appropriate time and place for all levels. For example, one may consider viewing the levels on a timeline in which one level may be a more appropriate point during decision-making. To extend this example further, consider a student governance body who may decide to seek input later in the decision-making process to confirm an earlier hypothesis and then discover that their initial interpretation of institutional data did not reflect that of a more diverse membership at a different point in time. Ideally, the student governance body has a diverse and inclusive membership, thus providing a fulsome, meaningful interpretation of institutional data and input.

After a reviewing all of the data collected from participants across the three phases of the NWCCU Quality Culture Project, in particular responses to the question: “At your institution, are efforts made to include a wide range of perspectives and marginalized voices?”, the levels (Table 1) that appeared most frequently were Level 1: Invitation and Level 2: Input.

As inclusive leaders, we must ensure that stakeholders have full participation in the design of mission fulfillment processes, which includes determining measures of institutional performance. This differs from hierarchical approaches where leaders do not ask for advice, input, or recommendations from those responsible for enacting change (Lambert et. al., 2016). This further includes providing access to information in order to determine goals, actions, and policy. This is contrary to hierarchical approaches where the leader controls information and institutional goals are set by those with formal authority.

In addition, as inclusive leaders, we must be aware of low status individuals who may be abdicating their responsibility to contribute, either due to fear or a reliance on perceived experts (Barton & Sutcliffe, 2009). Therefore, higher education leaders must provide protected time and space for learning, a container (Isaacs, 1999) where differences of opinion can be explored (Schein & Schein, 2016). We practice this by empowering committee members to own the solutions they devise and to tinker and iterate until they see their desired results. Finally, as inclusive leaders we need “to create spaces where useful changes can emerge, and then support and amplify those changes” (Bushe & Marshke, 2016, p.410) by immersing ourselves in the process and reflexively considering what narratives we may be privileging or marginalizing.

Future Considerations

While the Levels of Participation (Table 1) are an untested diagnostic tool, institutions may consider informally reviewing their decision-making processes to determine whether or not they rely more heavily on one level of participation over the others. For example, institutions could conduct an audit of processes and at different points of decision-making to explore who has (does not have) influence and who is (who is not) interpreting the data. Future considerations should assess the utility of the Levels of Participation tool for facilitating achievement of contextualized performance measurement systems. Guba and Lincoln (1989) describe transferability as the qualitative equivalent to external validity (generalizability), which allows others to determine the applicability of the research findings in similar situations. For example, an emerging question to address is: Can the levels be applied to a variety of evaluation practices, such as: academic program review, departmental review, curriculum development, and assessment of student learning? In addition, this line of inquiry should explore the linkages between equitable evaluation practices and educational policy development (both internal and external to the organization).

Navigate the articles below, or go to the current Beacon directory.

References

Barton, M.A. & Sutcliffe, K.M. (2009). Overcoming dysfunctional momentum: Organizational safety as a social achievement. Human Relations, 62(9), 1327-1356.

Brown, W. (2015). Undoing the demos: Neoliberalism’s stealth revolution. Zone Books.

Bushe, G.R. & Marshak, R.J. (2016). The dialogic organisation development approach to transformation and change. In W.J. Rothwell, J.M. Stavros & R.L. Sullivan (Eds.), Practicing organisation development: Leading transformation and change (4^th ed.) (pp.407-417). Wiley.

Canadian Evaluation Society. (2018). Competencies for Canadian evaluators. Canadian Evaluation Society. https://evaluationcanada.ca/competencies-canadian-evaluators

Cardoso, S., Rosa, M.J. & Videira, P. (2018). Academics’ participation in quality assurance: Does it reflect ownership? Quality in Higher Education, 24(1), 66-81.

Cox, T. (2020). Creating space for transformation: Cultivating containers for inclusive leadership development. In J. Marques (Ed.), The Routledge companion to inclusive leadership (pp. 273-287). Routledge.

Furman, G.C. (2004). The ethic of community. Journal of Educational Administration, (42)2, 215-235.

Guba, E.G. & Lincoln, Y.S. (1989). Fourth generation evaluation. Sage.

Helgeson, S. (2005). The web of inclusion: A new architecture for building great organizations. Beard Books.

Isaacs, W. (1999). Dialogue: The art of thinking together. Crown Business.

Kirkhart, K.E. (2010). Eyes on the prize: Multicultural validity and evaluation theory. American Journal of Evaluation, 31(3), 400-413.

Lambert, L., Zimmerman, D.P. & Gardner, M.E. (2016). Liberating leadership capacity: Pathways to educational wisdom. Teachers College Press.

Robert, H.M. (2020). In R.S. Corbin (Ed.), Robert’s rules of order newly revised (12th ed.). Public Affairs.

Schein, E.H. & Schein, P. (2016). Organizational culture and leadership (5th ed.). Jossey-Bass.

Shaibah, A. (2020, August). How to mobilize and sustain EDI change in the academy. University Affairs. https://www.universityaffairs.ca/opinion/in-my-opinion/how-to-mobilize-and-sustain-edi-change-in-the- academy/

Spitzer, D.R. (2007). Transforming performance measurement: Rethinking the way we measure and drive organisational success. American Management Association.

Wasserman, I.C. (2015). Dialogic OD, diversity, and inclusion: Aligning mindsets, values, and practices. In D.A. Noumair & R. Shani (Eds.), Research in organisational change and development (Vol. 24, pp. 329-356). Emerald Group Publishing Limited.

Wehipeihana, N. (2019). Increasing cultural competence in support of Indigenous-led evaluation: A necessary step toward Indigenous-led evaluation. Canadian Journal of Program Evaluation, 34(2), 368 – 384.