The Limitation of Predictive Analytics - Even in Stable Times

Oct 06, 2020

In our previous post in the Black Swan series we discussed how valuable leading indicators can be during periods of volatility. One of the biggest challenges for analytics during the era of the black swan involves decisions regarding the validity and usefulness of predictive models.

Over the last five to ten years, many institutions have made investments in home-grown predictive analytics capabilities and commercial software products. Institutional leaders are very interested in advanced analytics because of the potential they offer to help plan effectively – especially for impacts on revenue. However, even during more stable times, questions that involve student behavior are not easy to forecast: What applicants are most likely to enroll? What students are most likely to retain or complete? What alumni are most likely to contribute to a campaign? So many internal and yet seemingly prosaic changes can affect the accuracy of predictive models in determining outcomes including altering of financial aid award structures, varying of the timing of when applications are released, or transforming the campus visit experience. The impact of these internal pressures is magnified by external forces—a pandemic, economic instability, civil unrest—that dramatically influence one of the biggest and most expensive decisions people can make for themselves and their children: choosing a college.

As analytic leaders and practitioners it is up to us to have honest conversations with leadership about the effectiveness and reliability of these tools and techniques during such volatile times. In this segment of our conversation you will hear from Dr. Nick Wagner and Dr. David Kowalski as they discuss how they have been navigating the use of predictive analytics and providing guidance to their leadership at their respective institutions.

The video below is the third segment in a series from a conversation ASR's Higher Ed practice had in July 2020 with several of our client analytic leaders, about the impacts of the year's events on their use of analytics. In this segment, you will hear the voice of Moderator John Van Weeren, Principal at ASR; Dr. Nick Wagner, Executive Director of Institutional Effectiveness at Saginaw Valley State University; and Dr. David Kowalski, Associate Vice President for Institutional Effectiveness and Strategic Innovation. Dr. Nicola Richmond, Chief Strategist and Vice Chancellor of Strategy, Analytics and Research at Pima Community and Greg Flanik, Chief Information Officer at Baldwin Wallace University also participated in the conversation.