What's special about architecting for AI?
Traditional solution and technical architecture are well-established disciplines with a range of solid approaches and methodologies that can all lead to good outcomes. However, all of these methodologies are based on assumptions that are questionable, if not decidedly false, when architecting for AI solutions.
Next, I will present seven key differences from traditional architectural assumptions that you should keep in mind throughout the rest of the book and in the future when you apply the knowledge in practice.
In short, AI solutions have the following characteristics:
- Probabilistic
- Model-based
- Data-dependent
- Autonomous
- Opaque
- Evolving
- Ethically valent
While not unique to Salesforce, these considerations are essential when creating AI solutions on the Salesforce platform. Because you are given so much out of the box, it can be tempting to follow a traditional mindset in your architecture and design. This will backfire.
Probabilistic
Days before the beginning of the 2018 soccer World Cup, researchers from the German Technische University of Dortmund, the Technical University in Munich, and Belgium's Ghent University predicted the winner of that year's trophy. They had run 100,000 simulations and had concluded that Spain was going to win. They weren't alone. Researchers from UBS, Goldman Sachs, and several other universities joined in the fun. They used a variety of approaches and predicted different winners. The only thing they shared was that they all got the winner wrong. Only a single machine learning prediction – from EA Sports, makers of the FIFA 18 computer game – picked the correct winner of the tournament, France.
This story might seem disheartening to some. However, it is not something to preoccupy yourself with much, as long as you understand that machine learning systems are inherently probabilistic, not deterministic. In the aforementioned predictive model, Spain was given an overall 17.8% chance of winning. This was more than France's 11.2% but hardly a ringing endorsement. Therefore, we shouldn't be surprised at what happened but acknowledge that any prediction is most likely going to be wrong for one-off events.
Where AI solutions have real value, instead, is when we have repeated events occurring over and over again. If we had 1,000 world cups running one after the other, most likely Spain would have won more of them than France, and this would be actionable information we could use in our processes – perhaps to manufacture or promote more Spanish merchandise.
In our day-to-day processes, we have events happening millions of times and usually with much less variability than in a world cup. Our ability to predict is, therefore, much better. However, that can also lead to problems. A prediction that is too good can come to be taken as a certainty, and we can end up designing our systems so they fail when we encounter outliers. The key when developing AI-based solutions is to look hard at the data and the predictions and then to come to a reasonable compromise about what level of process use they will sustain.
Model-based
In contrast to traditional solutions, AI-based architectures use models rather than prescriptive code to solve a problem. This requires a shift in thinking on behalf of the architect. There is a famous mini short story by the Argentinian writer Jorge Luis Borges, called On Exactitude in Science. It is written in a single paragraph and can be found at the following link: https://walkerart.org/magazine/empire-art-cartography-attained.
A map covering the whole territory is useless, just as trying to capture all the complexity of your processes and data in a machine learning model is futile. In traditional solution design, we tend to be precise and specify all the rules and exceptions; in an AI system, that leads to your predictions not generalizing. You get good results on the data you already have and terrible results on future data. The most useful map size, just as the most useful model size, is big enough to let you see the amount of detail you need and no larger.
Data-dependent
This point is hopefully apparent based on the discussion we have had so far in this book. The quality of predictions in an AI-based system is proportional to the quantity and quality of the data used to build the models by which the system predicts.
Think back to the recommendation systems of the early internet, based as they were on synonyms and manual encodings of likeness. Let's for a moment assume that you are searching for thermal socks. The website might show you different pairs of socks based on that search, but you would be unlikely to be shown other pieces of thermal ware that you might want unless the retailer was very good at managing their catalog. Most likely, you would not be offered a good selection of other winter gear that might be relevant to your current pursuits, and certainly, the website would not customize it to your personal preference or the preference of other shoppers like you.
Love them or hate them, these features are all run-of-the-mill today, and that is mostly because certain internet companies have vast troves of data that they can use to generate such recommendations. The actual improvement in recommendation algorithms pales in comparison to the impact of more data.
Autonomous
Ethereum, the second-largest digital cryptocurrency, saw its price drop from more than $300 to as low as $0.10 in a matter of minutes on June 22, 2017. This crash was caused by a single massive sell order triggering more than 800 stop-loss orders, orders set to sell once the price hit a certain level automatically. There seems to have been no malice or wrongful action involved, merely the interplay of many automated agents acting in an uncoordinated but similar way.
If the interplay of relatively dumb rule-based agents can lead to this level of disruption, what will happen when we start to divulge more autonomy to AI-based systems? We don't know, but almost inevitably, when we start having bots and predictive automation on our key business systems, there will come a time when we start seeing unexpected behavior. Maybe our bots will begin to undo each other's work because they have conflicting instructions, or perhaps we will see messages being sent in a loop because there is a hidden circularity in one of our models.
For now, the consequences are likely to be minor inconveniences, but as these systems grow in responsibility and complexity, so will the problems. It is, therefore, essential to ensure that you have appropriate monitoring and humans in the loop at the right points in the process. You might be able to get away with not having it for a little while, but the long-term consequences of inaction will most likely be considerable.
Opaque
When I was starting out building machine learning models, I worked on a binary node classification problem using large graphs. I had a large set of graphs that contained different structures, and based on those structures, the nodes within the graph should be labeled either Yes or No, depending on whether the program should include the node in question in the output. I ran my initial model and was pleased with myself when I got 97% accuracy.
I then tried the model in practice, and it failed utterly. It just didn't work. I started digging around in the data and the model training, and after a (too) long time, I found the problem. Within my training data, a small number of graphs were huge (100–1,000 x the size of the other graphs) and had a structure that meant everything should be classified as No. These graphs represented unsolvable problem instances. When I had trained my model, what had happened was because of this overwhelming preponderance of No in the training set that I hadn't spotted because the unsolvable graphs had not been a part of my initial data analysis, the model had learned to say No 100% of the time. Because 97% of the cases were No, that gave 97% accuracy.
There were many failings on my part in this example:
- I didn't do proper exploratory data analysis.
- I didn't check my assumptions.
- I jumped too quickly into implementation.
- I had an inadequate evaluation framework in place.
However, what it also underscores is that AI systems can be opaque. There isn't a simple way to go into debug mode, step through the code, and work out what is happening. Therefore, the evaluation and gradual implementation of models are critical factors to consider whenever you are rolling out these kinds of systems.
Evolving
On March 22, 2016, Microsoft unveiled Tay, a Twitter chatbot, as an experiment in conversational understanding. Tay used advanced deep learning technologies to learn from conversations with real humans. According to Microsoft, the more you talked with Tay, the better he would get at conversation. Less than 24 hours after, Microsoft took Tay offline after spewing Nazi and anti-feminist rhetoric, which is too colorful to include in a serious work on technology. Effectively, after being targeted by an army of Twitter trolls, Tay had learned what they had to teach him and parroted them with alarming accuracy.
While Tay presents an extreme example, the fact of the matter is that machine learning systems learn. And they learn from the data that you feed them. Most of the models you build on the Salesforce platform will continue to learn after you deploy the initial model, and as the incoming data changes, so will the models, mostly for the better and sometimes for the worst. That may mean that your models' performance also changes over time, and you may get a question from your business users as to why. Again, monitoring the model regularly and having a plan for continuous validation is a good idea.
Ethically valent
The final factor to consider is that AI systems are ethically relevant in a way that most traditional computer systems are not. Data contains bias, and if you aren't careful, your models will reflect those biases. Google, for instance, was recently forced to apologize for their computer vision model generating racist labels. For example, a black hand holding a thermometer was assigned a label of gun, while an identical white hand holding the same thermometer was labeled monocular.
There are good frameworks and principles for addressing these problems, and Salesforce is one of the major technology vendors that has dedicated the most effort to ensuring responsible use of AI, incorporating such principles in its work.
Now, having understood how to architect for AI solutions, let's move on and meet the company whose requirements we'll be following throughout the book.
