Nine months into the transformation, a vendor arrived with a demo. An AI-powered student success platform: predictive analytics, personalized intervention recommendations, institutional retention dashboards. The visualizations were striking. Sandra saw faculty-level visibility into student struggles. Claire saw operational efficiency. Even James, usually sceptical of technology promises, noted that the retention improvement figures would generate significant tuition revenue recovery. Marcus asked four questions: Which specific capability gap does this address? What happens when the prediction is wrong? What data does the model need, and do we currently have it integrated and reliable? Who would act on the predictions, through what process, with what authority? The vendor answered the first confidently. The other three revealed a 23% false positive rate, data that was not yet integrated, and an organizational process the platform had not considered. This chapter teaches you how to evaluate digital and AI proposals from the business architect's perspective -- not whether the technology works, but whether it serves the architecture.
By the end of this chapter, you'll be able to:
Nine months into the Student Experience Transformation, a vendor arrived with a demo.
The product was an AI-powered student success platform. It used predictive analytics to identify students at risk of dropping out, generated personalized intervention recommendations for advisors, and produced institutional dashboards showing retention risk across programs, demographics, and faculties. The demo was polished. The data visualizations were striking. The vendor cited implementation success at twelve institutions, with retention improvements of 4-8% within two years.
The room was impressed. Sandra Mwangi saw the potential for faculty-level visibility into student struggles. Claire Dubois saw the operational efficiency: automated risk scoring instead of manual advisor judgment. Even James Firth, usually sceptical of technology promises, noted that the retention improvement numbers, if accurate, would generate significant tuition revenue recovery.
Marcus watched the demo carefully. Then he asked four questions.
"Which specific capability gap does this platform address in our architecture?"
"What happens when the prediction is wrong and an advisor contacts a student who is not struggling?"
"What data does the model need, and do we currently have that data integrated, governed, and reliable?"
"If we deployed this tomorrow, who would act on the predictions, through what process, and with what authority?"
The vendor answered the first question confidently: student retention. The other three received less confident responses. The second question revealed that the model had a 23% false positive rate. The third revealed that the model required integrated data from the SIS, LMS, and advising platform, three systems that were still in the early stages of integration at Lakeshore. The fourth revealed that the vendor had not considered the organizational process: the platform generated predictions, but the process of acting on them (who contacts the student, through what channel, with what support options) was outside the platform's scope.
This chapter is about evaluating digital and AI-enabled solutions from the business architect's perspective. Not whether the technology works (that is the solution architect's assessment) but whether it serves the architecture. The building architect does not need to understand the engineering of smart home systems. But they need to know whether the smart system serves the client's needs or merely adds complexity to the house.
Here is the distinction that organizes everything that follows. Some digital and AI solutions genuinely enable a capability that could not exist before. Others add complexity to a capability that already works. The vendor's demo was impressive. But the business architect's job is to determine which category a given proposal falls into. The five-question framework in Section 7.3, the data governance discipline in Section 7.2, and the capability filter throughout this chapter are all practical tools for making that determination. When you can distinguish enabling from complicating, you can advocate for the right investments and challenge the wrong ones with equal confidence.
Create a free account to access Shaping What Gets Built and start learning.