Fundamentals of AI System Architecture

The recent surge of public interest in Artificial Intelligence (AI), particularly with the rise of generative AI, has ignited a wave of excitement and demand for comprehensive AI solutions. This heightened interest extends beyond tech enthusiasts and researchers to businesses, governments, and individuals seeking to harness AI’s power to solve real-world problems and enhance their capabilities. In this landscape, the architecture of AI systems, which defines their structure, components, and interactions, plays a pivotal role in shaping the development and deployment of effective AI solutions.

AI has emerged as a transformative force, revolutionizing industries and reshaping the way we interact with technology and the world around us. At its core, AI refers to computational models that mimic human cognitive functions, including learning from data, recognizing patterns, making decisions, and even interacting with their environment. This revolutionary technology spans a wide spectrum, from simple rule-based systems to sophisticated deep learning models, each with unique applications and capabilities.

A major aspect of any AI system is that the results of the inference being done need to be relevant and trusted. To ensure trust is gained and maintained, the use of strong architecture is paramount. One not only architects the technology but also how the technology is going to be used, managed, and evaluated by the span of stakeholders. The stakeholders need to be able to pinpoint issues, rapidly correct model parameters, and deploy changes in a deliberate and rapid manner. In more common parlance, the architecting and supporting processes can be described as “guard rails.” How one employs guard rails is very specific to the domain and use case that is to use the AI technology. There are classes of guard rails that can be discussed – for example, the use of canaries to judge model correctness from a known gold standard, time and data flow metrics to judge model performance, and the use of filters and robust data quality checks so that only consistent and correct data enters the system. Another class of guardrails is human system interfaces, such as alerting frameworks to classify errors and monitors, the use of troubleshooting tools, and preset protocols for handling unexpected errors. Written procedures or guidance from modeling allow for the maintenance of a system without the need to call upon the model developer to do troubleshooting.

Trust is a paramount consideration for system success, so one needs to architect a system with that in mind. In many ways, the presentation and lessons learned described in this book look to ensure trust in an AI system.

This chapter highlights, in a broad sense, the key aspects of AI architecture considerations that drive a successful AI implementation. The topics are as follows:

• Introduction and key AI concepts
• Components of an AI system
• AI technologies and microservices
• AI systems and technical considerations
• Deployment considerations