By: Dr. Jess Stahl, Vice-President Data Science & Analytics (NWCCU)
[Part I: Institutional Profiles and Technical Resources appeared in Issue V3I4 of the Beacon]
In May 2020, NWCCU member institutions responded to a survey from the NWCCU entitled “Institutional Data Capacity Survey” (IDCS), which was designed to provide insight into the capacity (e.g., data, technology, and resources) of institutions throughout our region to improve equitable outcomes through evidence-based approaches to meet the NWCCU 2020 Standards for Accreditation. Specifically, the standards (Standard One- Student Success, and Institutional Mission and Effectiveness: 1.D.2, I.D.3, and 1.D.4) require institutions to report disaggregated indicators of student achievement and engage in regional and national benchmarking using transparent methods for data collection and analysis to mitigate gaps in achievement and equity.
Thus, the IDCS covered a wide range of domains including technical resources, data staffing, data governance, and data reporting to inform our strategic planning about how best to support our member institutions. This article is the second in a three-part series discussing the key highlights of the survey results and our relevant initiatives. This article presents a summary of data staffing, governance, and ethics at NWCCU institutions as reported in the IDCS survey.Â
Data Staffing
The most prevalent data role is “Data Analyst” by a large margin. It is notable that 23% of institutions reported having “None” of the data roles presented within the survey whereas 21% reported having three or more data roles, which suggests an institutional data capacity disparity among our institutions. Similar findings across multiple sections of the IDCS inspired our “PDP Accelerator” initiative to actively facilitate increased data capacity for NWCCU institutions through access to high-quality data dashboards to explore intersectional, disaggregated data in support of data-informed approaches to improving outcomes and eliminating equity gaps.
* Totals sum to more than 100% because many institutions reported having more than one role.
The majority (75%) of institutions reported that they would benefit from having greater FTE dedicated to working with data.
When asked what additional resources would be ideal, the most widely cited resource was additional Data Analysts. Several institutions indicated that they currently lack a single, full-time Institutional Research professional. Institutions also expressed a need for additional staff with specific expertise such as qualitative research analysts, statisticians, data scientists, SQL programmers, predictive modeling experts, application developers, data visualization experts, database administrators, data coaches, assessment specialists, big data experts, data engineers, and data literacy advocates. Responses indicated that data staffing would ideally increase by at least 1 additional FTE to meet their institution’s stated goal to be data-informed.
The majority of institutions (66%) indicated that their institution would benefit from additional expertise in working with data. The most desired expertise (by a wide margin) was predictive modeling. Other technical skills mentioned were data science, data engineering, data visualization, and SQL programming. However, institutions also mentioned non-technical skills like promoting the meaningful use of data and effective communication skills. Many noted that it is challenging to remain current with technical skills given a lack of resources and busy workload that leaves little time for professional development.
Institutional Research is the role with the most responsibility (90%) for interpreting data and results of analyses. Information Technology staff are also involved in interpreting data and results of analyses at 25% of institutions in collaboration with Institutional Research or Other staff. Among nearly 40% of institutions, a wide variety of other roles were associated with interpreting data such as Institutional Effectiveness, Provost, Human Resources, Finance, Enrollment Services, Assessment, Data Science, Student Services, Accreditation Liaison Officers, Deans, Registrar, President, Vice President, academic centers and units, and Faculty, which suggests dedication and progress towards building a culture of meaningful data use and promoting data literacy. However, it is important to note that 47% of institutions reported having 2 or more roles associated with interpreting data and results of analyses while 44% reported that Institutional Research alone bears this responsibility. Furthermore, 3% of institutions reported having no roles associated with interpreting data and results of analyses.
* Totals sum to more than 100% because many institutions reported having more than one role.
Information Technology (IT) staff control access (permissions) to databases and data warehouse(s) for most (91%) institutions. At about 30% of institutions this it the only role IT plays related to institutional research and assessment. However, more than half of IT staff also provide Excel files for reports or design custom SQL queries to generate reports. At nearly 20% of institutions, IT staff create data dashboards, analyze data, and generate reports for data stakeholders. At about 40% of institutions IT staff fulfill three or more of these roles. Many responses emphasized a close working relationship between IT staff and Institutional Research.
Data Governance and Ethics
There are data governance structures in place at 63% of our institutions. Very few (14%) have implemented data ethics structures. While 14% of institutions have both, 34% reported having neither. Only 32% of institutions reported having a data catalog, which suggests varying levels of data maturity between our institutions.
Data governance is most often directed by a committee (52%) or a department/office (6%) or working group (3%). Data governance is most often managed by IT, Institutional Research, or both (co-managed). At 2 institutions, data governance is directed by an individual.
Data ethics is most often directed by a committee or a department/office (both 3%). Other structures include university policies or data stewards (i.e., individuals that direct data ethics).
In our next issue, we will discuss IDCS results regarding investment in data analytics, data tools and methods (e.g., data visualization, predictive modeling, big data, and artificial intelligence).
Bonus Section:
In Your Own Words (What additional resources would be ideal?)…
“We would benefit from extra personnel with knowledge/skill in increasing data literacy, programming, data engineering, and data visualization and analytics.”
“More people is just part of the solution. We also need training and improvements to internal processes and structures at the University (not one particular unit).“
“Data analyst dedicated to data governance. An application developer in IT to refine applications and improve data collection.”
“At a minimum, one full-time staff member with a highly flexible skill set. Perhaps an analyst with the ability to parse data and validate theories for how best to improve our business processes; a data manager who can design queries and perform other data extraction and transformation; or an assistant who reports results to format tables, make plots, and perform basic tasks of uploading and downloading files. In addition, we would benefit from having additional FTE dedicated to creating and maintaining tools that allow college personnel to perform a number of queries that could be pre-set in order to save IR time for genuinely complicated problems. Additional FTE could also be dedicated to the overhaul, storage, and cleaning of data at all levels. As stated previously, the data submission process would benefit from software that takes [SIS] data and converts it for external reports.
The current model is that IR will provide for the analysis needs of a diverse group of clients, internal and external, and across all programs and college units. The challenge is that clients must be more informed users of data because it is not enough to rely on the knowledge of IR staff. To create a working culture of evidence and data-informed decisions, the client side must have sufficient understanding to actively participate in the research/analysis process because this allows them to learn from the data in their program or area and have more ability to engage and interpret it deeply. The client role should not be viewed one-dimensionally as a question-forming process, with the expectation that IR provide answers that conform to client expectations. While IR works within that client-centered framework to create accessible reports, that approach is insufficient in many research contexts where questions are not well-suited to automated, self-service dashboards, and often the results are more complex and require a more active analytical process. This suggests the need for more training in research for people outside of IR, if not FTE/capacity across the college.”
“Yes. IR is understaffed, needing to replace one worker who has moved to part-time in preparation for retirement, and in need of a senior analyst. Marketing and our unit responsible for international education need at least one analyst each.”
“2-3 Personnel; An IT person to build and strengthen the technology infrastructure, an additional IR person to do predictive modeling, and a data coach.”
“Another analyst would enhance the IR office’s capacity to not only provide routine data but to expand data literacy and advance data analytics and modeling.”
“Staff with background/degrees in statistical analyses, management of Big Data. Broader circle of staff with access to the data warehouse resources we currently have.”
“At least one (and preferably two) additional IR/IT staff focused on data automation and professional development/staff support on effective data use.”
“There is a desire to move from descriptive to prescriptive analyses. In addition, there is a need to address data literacy challenges at our college.”
“Ideally, a part to full-time database analyst would be useful query data and build reports and/or files for export to tools such as Tableau. This would allow the Director of Institutional Effectiveness more time for teaching data literacy, developing data governance and building data visualizations for use across campus and for the public. Alternatively, a single IR position with more technical background than the incumbent might allow for a leaner approach to fulfilling these roles.”
“One qualified person, with the sole purpose of working with departments to collect and analyze data as well as ensuring that the data is used for decision-making across the institution.”
“Data integrity auditing to ensure data is updated over time.”
“There are a variety of ways to build additional capacity: add a data manager; add someone who can focus on assessment alone; add someone who can focus on qualitative data; add a data coach to ‘market’ the use of data and present the data across the institution; add an analyst to allow the other person to be a data scientist. Or, possibly, build the staff to develop and maintain a data warehouse and/or data lake that would make data analytics more accessible.”
“A more robust team in IR; someone whose job description focuses on streamlining data, including a clear plan, and a cross-departmental process for working with data.”
“While more FTE devoted to working with data would be potentially beneficial, additional training and increased data literacy of current staff would also be beneficial.“
“”We need more data analysts looking for data quality issues; more IT staff to build and maintain tools used for reporting and analysis. Data scientist to build predictive models and incorporate artificial intelligence. Current office has two people serving the entire institution.”
In Your Own Words (What additional expertise would be ideal?)…
“Predictive analytics, programming, data engineering.”
“Use of data for planning and budgeting.”
“Learning from a partner or having a department at a similar institution would be beneficial. If there are no other [similar] colleges that have the expertise, then being matched with an institution that has experienced building data collection and analysis from the bottom up.”
“The best use of data to improve student success and efficiencies across the college.”
“Training in review, analysis, interpretation and presentation of data.”
“Data analysis capability. Report generation facilitation. Outside experts to teach new or updated methods.”
“Technical assistance for college leaders regarding interpretation of data. Data coaching for leaders who are seeking to use data to inform decision making.”
“Additional 1.0 FTE with 10+ years’ experience in Higher Education analytics at an R1 institution.”
“We have insufficient capacity for analysis and interpretation of data.”
“We need a data architect to help us design a comprehensive system overhaul and aid in the creation of more effective data architecture, including stable datasets, definitions, data modeling, and predictive analytics. Tableau requires some coding and data analytics, so an additional analyst for program review, which requires individual data analytics, a trainer to help staff to become more literate data users, and the skills mentioned in the previous questions.”
“People who understand ingestion, storage, and reporting of data within the IT department so that algorithms and other efficiencies can be built to support IR and assessment work.”
“Statistical analysis, predictive modeling, survey design, unit level or program level assessment.”
“HTML coding and web authoring, Python, Powershell.”
“Guidance in how to set up, integrate, and effectively manage a data warehouse. Also, how to set up a centralized repository for data and create effective institutional dashboards and data visualizations.”
“Information on how other small colleges manage data and data requests. Expertise on using data (we collect a lot of information but fall down when it comes to connecting it to actions/decisions).”
“More training in data analytics and visualization software. The learning curve for these is steep, which can lead to lost time in producing valuable analytics.”
“Use of a data warehouse and hands-on opportunities to experience the potential with data technology tools.”
“Dedicated professional and staff development time each month to stay current with the rapidly changing AI environment and other reporting requirements. Time to “document” processes and procedures including updates as these tasks take a lot of time given the constantly changing nature of data and reporting.”
“The concepts surrounding the use of data to inform decision-making are new to many on our campus. There has been remarkable progress in learning about these concepts over the last year, and additional training would improve our ability to create measurable goals/outcomes for all of our mission fulfillment tasks.”
“Data governance expertise.”
“High-level statistics and modeling would enhance our ability to produce insights and inform decision-making. In general, staff have uneven fluency with data and levels of expertise in analysis.”
“Artificial Intelligence and Predictive Analytics.”
“Actually well trained individuals in data issues, collection, analysis and reporting are a premium.”
Navigate the articles below, or go to the current Beacon directory.
