The Internet of Things (IoT) and Decision Science have been among the hottest topics in the industry for a while now. You would have heard about IoT and wanted to learn more about it, but unfortunately you would have come across multiple names and definitions over the Internet with hazy differences between them. Also, Decision Science has grown from a nascent domain to become one of the fastest and most widespread horizontal in the industry in the recent years. With the ever-increasing volume, variety, and veracity of data, decision science has become more and more valuable for the industry. Using data to uncover latent patterns and insights to solve business problems has made it easier for businesses to take actions with better impact and accuracy.
Data is the new oil for the industry, and with the boom of IoT, we are in a world where more and more devices are getting connected to the Internet with sensors capturing more and more vital granular dimensions that had never been touched earlier. The IoT is a game changer with a plethora of devices connected to each other; the industry is eagerly attempting to untap the huge potential that it can deliver. The true value and impact of IoT is delivered with the help of Decision Science. IoT has inherently generated an ocean of data where you can swim to gather insights and take smarter decisions with the intersection of Decision Science and IoT. In this book, you will learn about IoT and Decision Science in detail by solving real-life IoT business problems using a structured approach.
In this chapter, we will begin by understanding the fundamental basics of IoT and Decision Science problem solving. You will learn the following concepts:
Understanding IoT and demystifying Machine to Machine (M2M), IoT, Internet of Everything (IoE), and Industrial IoT (IIoT)
Digging deeper into the logical stack of IoT
Studying the problem life cycle
Exploring the problem landscape
The art of problem solving
The problem solving framework
It is highly recommended that you explore this chapter in depth. It focuses on the basics and concepts required to build problems and use cases. As hands-on exercises are not added, I am sure most software engineers would be tempted to skip this and move to the later chapters. The later chapters will frequently refer to concepts and points elucidated here for more realistic context. Hence, it's very important to go through this chapter in detail before moving on.
To get started with the IoT, lets first try to understand it using the easiest constructs. Internet and Things; we have two simple words here that help us understand the entire concept. So what is the Internet? It is basically a network of computing devices. Similarly, what is a Thing? It could be any real-life entity featuring Internet connectivity. So now, what do we decipher from IoT? It is a network of connected Things that can transmit and receive data from other things once connected to the network. This is how we describe the Internet of Things in a nutshell.
Now, let's take a glance at the definition. IoT can be defined as the ever-growing network of Things (entities) that feature Internet connectivity and the communication that occurs between them and other Internet-enabled devices and systems. The Things in IoT are enabled with sensors that capture vital information from the device during its operations, and the device features Internet connectivity that helps it transfer and communicate to other devices and the network. Today, when we discuss about IoT, there are so many other similar terms that come into the picture, such as Industrial Internet, M2M, IoE, and a few more, and we find it difficult to understand the differences between them. Before we begin delineating the differences between these hazy terms and understand how IoT evolved in the industry, lets first take a simple real-life scenario to understand how exactly IoT looks like.
Let's take a simple example to understand how IoT works. Consider a scenario where you are a father in a family with a working mother and 10-year old son studying in school. You and your wife work in different offices. Your house is equipped with quite a few smart devices, say, a smart microwave, smart refrigerator, and smart TV. You are currently in office and you get notified on your smartphone that your son, Josh, has reached home from school. (He used his personal smart key to open the door.) You then use your smartphone to turn on the microwave at home to heat the sandwiches kept in it. Your son gets notified on the smart home controller that you have hot sandwiches ready for him. He quickly finishes them and starts preparing for a math test at school and you resume your work. After a while, you get notified again that your wife has also reached home (She also uses a similar smart key.) and you suddenly realize that you need to reach home to help your son with his math test. You again use your smartphone and change the air conditioner settings for three people and set the refrigerator to defrost using the app. In another 15 minutes, you are home and the air conditioning temperature is well set for three people. You then grab a can of juice from the refrigerator and discuss some math problems with your son on the couch. Intuitive, isn't it?
How did it his happen and how did you access and control everything right from your phone? Well, this is how IoT works! Devices can talk to each other and also take actions based on the signals received:
Lets take a closer look at the same scenario. You are sitting in office and you could access the air conditioner, microwave, refrigerator, and home controller through your smartphone. Yes, the devices feature Internet connectivity and once connected to the network, they can send and receive data from other devices and take actions based on signals. A simple protocol helps these devices understand and send data and signals to a plethora of heterogeneous devices connected to the network. We will get into the details of the protocol and how these devices talk to each other soon. However, before that, we will get into some details of how this technology started and why we have so many different names today for IoT.
So now that we have a general understanding about what is IoT, lets try to understand how it all started. A few questions that we will try to understand are: Is IoT very new in the market?, When did this start?, How did this start?, Whats the difference between M2M, IoT, IoE, and all those different names?, and so on. If we try to understand the fundamentals of IoT, that is, machines or devices connected to each other in a network, which isn't something really new and radically challenging, then what is this buzz all about?
The buzz about machines talking to each other started long before most of us thought of it, and back then it was called Machine to Machine Data. In early 1950, a lot of machinery deployed for aerospace and military operations required automated communication and remote access for service and maintenance. Telemetry was where it all started. It is a process in which a highly automated communication was established from which data is collected by making measurements at remote or inaccessible geographical areas and then sent to a receiver through a cellular or wired network where it was monitored for further actions. To understand this better, lets take an example of a manned space shuttle sent for space exploration. A huge number of sensors are installed in such a space shuttle to monitor the physical condition of astronauts, the environment, and also the condition of the space shuttle. The data collected through these sensors is then sent back to the substation located on Earth, where a team would use this data to analyze and take further actions. During the same time, industrial revolution peaked and a huge number of machines were deployed in various industries. Some of these industries where failures could be catastrophic also saw the rise in machine-to-machine communication and remote monitoring:
Thus, machine-to-machine data a.k.a. M2M was born and mainly through telemetry. Unfortunately, it didn't scale to the extent that it was supposed to and this was largely because of the time it was developed in. Back then, cellular connectivity was not widespread and affordable, and installing sensors and developing the infrastructure to gather data from them was a very expensive deal. Therefore, only a small chunk of business and military use cases leveraged this.
As time passed, a lot of changes happened. The Internet was born and flourished exponentially. The number of devices that got connected to the Internet was colossal. Computing power, storage capacities, and communication and technology infrastructure scaled massively. Additionally, the need to connect devices to other devices evolved, and the cost of setting up infrastructure for this became very affordable and agile. Thus came the IoT. The major difference between M2M and IoT initially was that the latter used the Internet (IPV4/6) as the medium whereas the former used cellular or wired connection for communication. However, this was mainly because of the time they evolved in. Today, heavy engineering industries have machinery deployed that communicate over the IPV4/6 network and is called Industrial IoT or sometimes M2M. The difference between the two is bare minimum and there are enough cases where both are used interchangeably. Therefore, even though M2M was actually the ancestor of IoT, today both are pretty much the same. M2M or IIoT are nowadays aggressively used to market IoT disruptions in the industrial sector.
IoE or Internet of Everything was a term that surfaced on the media and Internet very recently. The term was coined by Cisco with a very intuitive definition. It emphasizes Humans as one dimension in the ecosystem. It is a more organized way of defining IoT. The IoE has logically broken down the IoT ecosystem into smaller components and simplified the ecosystem in an innovative way that was very much essential. IoE divides its ecosystem into four logical units as follows:
Built on the foundation of IoT, IoE is defined as The networked connection of People, Data, Processes, and Things. Overall, all these different terms in the IoT fraternity have more similarities than differences and, at the core, they are the same, that is, devices connecting to each other over a network. The names are then stylized to give a more intrinsic connotation of the business they refer to, such as Industrial IoT and Machine to Machine for (B2B) heavy engineering, manufacturing and energy verticals, Consumer IoT for the B2C industries, and so on.
Now that we have a clear understanding of what is IoT and the similar terms around it, lets understand the ecosystem better. For convenience, IoE will be referred as IoT while exploring the four logical components of the stack in brief.
When we deconstruct the IoT ecosystem into logical units, we have People, Processes, Data, and Things. Lets explore each of these components in brief.
People or we interact with devices and other people on a daily basis. The communication could be either People to People, People to Device, or Device to People. Considering People as a separate dimension in the IoT ecosystem is an essential move as the complexity in understanding this is really challenging. When any form of communication occurs where People play a role on either end of the interaction, it embeds a unique pattern that is intrinsic to the People dimension. Lets understand this better with an example. Most of us use social networking sites such as Facebook, Twitter, LinkedIn, and so on, where we are connected to multiple people/friends. Here, the communication paths are mainly People to People. Considering the previous example, we had people to device and device to people communication paths (communication between the smartphone and microwave). Considering People as a dimension, everyone would differ in the way they interact with the system. I might find the new interface of Facebook very difficult to use but a friend may find it extremely easy. The real problem here is everyone is skilled, but the skillsets differ from person to person. The characteristics of the interaction identified by a person may be a representative for a very small community.
We have a population of six billion plus, and over 1/6th of them have already been connected. With such a huge population consisting of a plethora of communities representing people of different geographical areas, culture, thinking, and behavior, defining one generic set of rules or characteristics to define people interaction is very challenging. Instead, if we understand the People dimension in a more constructive way, we can tap the opportunity to capture the behavior more accurately and help them benefit from the ecosystem in the best way.
With the advent of IoT, we have sensors capturing information and characteristics at more granular levels than ever before. Here, if we can accurately define People as a complete dimension, personalized experience will be a complete game changer. The smart watch industry is investing humongous efforts to get its offering more personalized; if it succeeds, it will be a pivotal player in the coming revolution.
The most lucid definition for Processes would be everything required to deliver the right information to the right person/system at the right time. A wide variety of things fall in the Processes dimension that includes technology, protocols, business logic, communication infrastructure, and so on. Broadly, they can be classified into two components-Technology and Business Processes. Lets explore these two components in brief in order to understand the Processes dimension in more detail.
The technology required in the Processes dimension of IoT comprises of the software, protocol, and infrastructure. We will explore Technology by understanding its three broad divisions for Processes.
Software consists mainly of the operating system. Devices in IoT require a special kind of an operating device. Smart devices such as the smart refrigerator, smart microwave, and many others require an operating system running on them that can then enable it to be an active component in the network. Tasks executed can vary from sending, processing, and receiving data or executing instructions and sending signals to respective controllers within the device for action. Now, the question is, why do these devices require a special operating system? Why cant the existing rich flavors of Unix/Linux, Windows, Mac, or even Android be used? The answer is the same as the reason that we used Android for smartphones and not the existing OS back then. The devices that connect to the network in IoT are small or sometimes tiny. Ideally, these devices would be equipped with less powerful computing capabilities, lower memory, and lower battery life. It is almost impossible to run a fully-fledged operating system on them. We need a specially designed OS that can take care of the limited memory, processing power and battery life of the device and yet provide maximum functionality to tag the device as a smart device. Google recently launched an operating system for IoT devices called Brillo. Brillo is an Android-based embedded operating system specifically designed for low power and memory-constrained IoT devices. It provides the core platform services required for IoT devices along with a developer kit freely available for developers/hardware vendors to get the OS running and build additional services on their devices. Some similar examples would be Apple's Watch OS for Apple Watch, Android Wear from Google for smartwatches, and others. Soon, we can expect a vast community of devices running Brillo and a plethora of apps that can be installed additionally for even better functionality (something very similar to the Google Play store).
Once the devices are software-enabled, we need to get a protocol in place that can help them communicate with other heterogeneous devices in the network. To understand this better, recollect the first example where we could defrost the refrigerator using our smartphone. The smartphone needs to talk to the refrigerator that also needs to understand what exactly is being communicated. With a huge variety of heterogonous devices, this communication path just gets more and more complicated. Hence, we need to have a simplified protocol in place where complicated process can be abstracted and the devices can communicate with each other effectively. Google recently launched an open source protocol called Weave. Weave is basically an IoT protocol that is a communications platform for IoT devices that enables device setup, phone-to-device-to-cloud communication, and user interaction from mobile devices and the web. It has ushered productivity in the developers efforts by easing up device interoperability regardless of the brand or manufacturer.
Infrastructure can simply be defined as the integration of the operating system, communication protocol, and all other necessary components to harmonize the environment for an IoT use case. All major cloud infrastructure providers are now focusing on providing an IoT-specialized environment. Google launched IoT Cloud Solutions, Amazon launched AWS IoT, Microsoft launched Azure IoT Suite, and so on. All of these solutions integrate the disparate systems together to make the ecosystem scalable and agile. Digging deeper into these suites will be beyond the scope of this book.
The second part of the Processes dimension is Business Processes. It basically covers the set of rules and processes to govern the communication and operation of the devices connected in the IoT ecosystem. There isn't a concrete definition till now that can be used here and the discussion of the topic will be beyond the scope of this book. However, we will take a look at this closely while solving an IoT use case in Chapter 3, The What And Why - Using Exploratory Decision Science for IoT and Chapter 4, Experimenting Predictive Analytics for IoT.
Things form the crux of the IoT ecosystem. They include any form of sensors, actuators, or other type of devices that can be integrated into machines and devices to help them connect to the Internet and communicate with other devices and machines. These things will be always active during their lifetime and will sense events, capture important information, and communicate them with other devices.
A typical example would be the refrigerator, TV, or microwave that we considered in the previous use case. The sensors installed in these devices capture data and send information/signals to other devices that can then be used to take action.
Data is by all means the most value-adding dimension in the IoT ecosystem. Today, the devices that are connected to the Internet are capturing tons and tons of data that can represent the most granular-level details for the devices they are connected to. The magnitude of this data is colossal. Storing and processing such vast and varied amounts of data questions the fact whether the data is really valuable. In a true sense, most of the data is transient in nature and loses its value within minutes of generation. With ever-improving technology and computing capabilities, the amount of data processing and storage that the devices are capable of today is great, but we can leverage this power to deliver better value than just delivering raw data. Tons of algorithms can be executed and business rules can be applied where a lot of value can be extracted from the data before sending it over to the server. This requires the combination of multiple disciplines together to solve the problem and deliver value.
To understand this better, consider the example of a pedometer installed in our smart watch. Rather than just reporting the number of steps that we have walked, it can calculate the amount of calories we have burned, average time taken for the activity, metrics like deviation from the previous days activity, deviation from milestones, and other social information such as how do we compare with our friends, and so on. To capture and process all of this information locally and send the final results to the server that can be directly stored for future actions requires the combination of multiple disciplines to make the task efficient. Math, business, technology, design thinking, behavioral science, and a few others would need to be used together to solve the problem. In reality, it would be futile to send across all the raw data captured from devices to the servers assuming that it can be leveraged for future use. A variety of new algorithms have been designed to ingest this data locally and deliver only rich, condensed, and actionable insights in real time. We will explore this in more detail with fog computing in Chapter 8, Disruptions in IoT. Smart watches such as the Microsoft Band and self-driving cars such as Tesla Model S are the best examples to understand the true scenarios where we can study the challenges of processing data in real time for insights and actions. In all true sense, data is what essentially helps in delivering the last mile value in the IoT fraternity. Hence, we need talent to deal with the data as a separate dimension in the IoT stack.
You learned about IoT and explored its logical stack to understand more about People, Processes, Data, and Things. The core agenda of this book is to solve IoT business problems using Decision Science. Problem solving has been an art and has its origin ever since mankind evolved. I would like to introduce The Problem Life Cycle to learn how the problem keeps evolving. Understanding this topic is very essential to solve better problems in IoT.
Every industry has been trying to solve a problem. E-retail solved the problem of inconvenience in physical shopping for busy and working consumers, the printing press solved the problem of mass producing documents for the consumers, and so on. A few visionaries such as Apple Inc. have tried to solve a problem by first creating it. The iPod and iPad were devices that were a part of this revolution. The biggest challenge in solving a problem is that the problem evolves. If we take a deeper look at the problem life cycle, we can understand that the problem evolves from a Muddy to Fuzzy and finally to a Clear state and keeps repeating the cycle:
Lets take a simple example to understand this better. Consider the Marketing problem. Every organization wants to promote their products and services better by marketing them. Marketing has been a problem since ages. Lets assume that the inception of marketing happened with the invention of the printing press. Initially, the problem for marketing would be in the muddy phase, where a team of analysts would try to get the best strategy to market a product or service in place. Back then, newspapers and print media were the only medium, and the strategies and nature of the problem was very much limited to them. When the problem is new, it is in the muddy stage; we have no clear idea about how to solve it. We would try to understand the problem by experimenting and researching. Gradually, we gain some knowledge about the system and problem and then define a couple of best strategies and guidelines to solve the problem. This is when the problem evolves to the fuzzy stage. Here, the solution for the problem is still not clear, but we have a fair understanding of how to go about it. Finally, after a lot of research and experiments from a large pool of people sharing their results and understandings, we might finally have a concrete methodology in place that can be used as a complete guide to solve the problem. This is when the problem reaches the clear stage. It is the pinnacle of the problem solving methodology where we have a clear understanding about how to tackle the problem and solve it. However, one fine day, a big disruption happens and the problem that was finally in the clear state collapses and returns to the muddy stage. In the case of marketing, when people aced the best strategies to advertise using print media and newspapers, it collapsed with the invention of the radio. All of a sudden, the nature of the problem changed and it required a radically different approach to solve it. The experts, who had concrete approaches and strategies for the problem solving back then, had to revisit and start from the beginning as the problem went back to the muddy stage. The problem life cycle kept evolving, and this was repeated when television was introduced and again when social media was introduced. Today, with the social media booming and expanding to newer areas, we have the marketing problem currently stable at the fuzzy state. Soon, with the advent of virtual reality and augmented reality, it is expected to roll back to the muddy phase.
To get more real, lets relate the scenario with a more recent version of the problem. Consider a social media analyst trying to solve a problem: optimizing targets for sponsored ads that need to be placed in the Facebook newsfeed for a user based on his behavior. If we find the user to be a football enthusiast, we would insert an ad into his newsfeed for a sportswear brand. To keep things simple, assume that we are the first ones to do this and no one has ever attempted this in history. The problem will currently be in the muddy state. So logically, there would be no references or material available over the Internet for our help and research. Our problem solving task begins by identifying the users interest. Once he has been identified as a potential user with an interest in football, we need to place a sponsored ad in his newsfeed. How do we discover the users interest? There are a variety of metrics that can help us discover his interests, but for simplicity, lets assume that the users interests will be identified purely by the Status Updates he posts on his wall.
We can then simply try to analyze the statuses updated by the person and define his interests. If the word Football or names of any popular football players or football teams appear more than a desired threshold, we can possibly say that he would be following football and hence would be a potential target. Based on this simple rule, we create better strategies and algorithms where our accuracy of finding the potential users can be reached with the minimum amount of time and effort. Gradually, the problem moves from the muddy stage to the fuzzy stage. We now have a good amount of understanding regarding the problem. We may not have the best and most effective solution for the problem, but we definitely have a fair idea to get started and find a solution without too much research. Over a period of time, we, and many other similar folks, conduct various experiments, publish various blogs and research papers of the results, and help others learn from our methods and experiment more. Eventually, there would be a time when we will have attempted the exhaustive solution paradigms and have the knowledge for the best and most effective solution for any sort of analysis in that domain. Finally, it reaches its pinnacle point-the clear stage.
One day, all of a sudden, Facebook and other social media giants launch a new feature. Users can now share photos along with their status updates. A radical change will be seen in the way the user will now use the social network. People tend to post more photos than text updates. All the thought-leadership frameworks and research papers and blogs that proved to be highly successful earlier now seem to be ineffective. We are not sure how to analyze photos updated by the user in order to understand his interests. Unfortunately, the problem goes back to the muddy stage. These big changes keep happening again and again. After photos, it will be videos, then audios, and so on, and the cycle keeps repeating as usual. Recently, the user behavior on social networks has dramatically changed. People post more pictures than type any comment or status updates. These photos may or may not be symbolic of the message that the user wants to convey. Sarcasm or satire may be the objective. The memes that get viral over the Internet have no clear message embedded in them. It may be sarcasm or simple smileys that the user wants to comment on. Analyzing the meaning of these images (memes) to understand the actual message that the user wants to convey with algorithms and computers to find out his interests is a challenging deal.
Hence, understanding the problem life cycle helps prepare us better for the evolution of the problem and adapt the problem solving strategies better and faster.
Two questions that will have definitely surfaced in our thoughts are as follows:
Why is understanding the problem life cycle really important?
How does this add any value to the IoT problem solving?
Lets see how this will be helpful. While solving a problem, understanding the current state of the problem is essential for the analyst. Whenever we solve a problem, we would always prepare for the next state of the problem life cycle knowing that change in the problems current state is inevitable. If the problem is currently in the clear state, then the amount of time and effort we would invest as a data scientist would be considerably less than if the problem would have been in the muddy or fuzzy stage. However, the problem remains for the least amount of time in the clear stage. The jump from clear to muddy is shorter compared to any other transition in the problem life cycle. Being aware about the problem life cycle, an organization/data scientist would then prepare better for radical changes that are bound to happen in a short while. We would need to design our solution to be agile and prepare for the next change. Similarly, if the problem is in the fuzzy stage, a lot of our solutions will be designed in such a way that they can be productized for a particular use case or industry. Finally, when the solution is in the muddy state, our solutions in problem solving will be more of a service-based offering than a product. The amount of experiments and research that we would need for the problem to be solved is highest in the muddy state and least in the clear state:
So how does this relate to IoT and Decision Science and the intersection of the two? Decision Science has been a bit more widespread and prevalent in the industry than IoT. There have been tons of experiments and research conducted on data to find insights and add value that make Decision Science currently in the fuzzy stage. IoT, on the other hand, is fairly new and requires loads of research and experiments to get tangible results, which makes it in the muddy stage. However, when we talk about the intersection of the two, we are dealing with a set of interesting problems. On one side, we have a fairly mature ecosystem of Decision Science that has given tangible value to the industry through its experiments whereas IoT is still nascent. The intersection of the two is a very promising and lucrative area for business. It is in a position where it is steadily moving from the muddy to fuzzy stage. Very soon, we will see tangible results from large-scale IoT use cases in the industry that will immediately trigger the revolution for productization on Decision Science for IoT. Decision Science for IoT is rapidly being experimented and the initial results seem to be very promising. The era, where Decision Science for IoT will be in the fuzzy state, is very near.
With this in mind, we can now get to the basics of problem solving while being prepared for the use case to evolve into a fuzzy state. With the understanding of the problem life cycle concrete, lets now explore the problem landscape in detail.
What is the problem landscape? Why do we need to bother about it?
A simple answer would be, understanding the current state of the problem is just one dimension, but understanding the type of problem is a more essential part of problem solving. Lets make this simple. To understand the problem landscape, refer to the following image and try to visualize the problems on two dimensions-frequency and impact. Just like any other scatterplot, this one can also be divided into four major areas:
Low impact: Low frequency
Low impact: High frequency
High impact: Low frequency
High impact: High frequency
Apart from these four components, we can identify one large spot/circle that has a flavor of all these areas. Here, the problems can be with a high or low frequency and also a high and low impact. So we name it the Region of Uncertainty:
Lets understand what kind of problems appear in each of these boxes. Every organization will have a plethora of problems; some of them occur very frequently and some of them very rarely. Some may have a huge impact whereas some may have a small impact. Consider a large organization with hundreds to thousands of employees. There are a couple of problems where the frequency might be low and impact might also be very low. We would generally tend to avoid solving these problems as they are not worth the effort. Some problems, even though they may have a low impact, might have a huge frequency. They would mostly happen on a daily basis. These problems are solved with the typical IT solution approaches such as Support for Technology Infrastructure, CRM, attendance management, employee leave application portal, and so on. There are some problems where the impact will be extremely huge, but the frequency will be extremely low. Events such as making the company public, acquiring a new company, or changing the business model would happen probably once in a lifetime or once in a few years. These problems can be solved from a consulting approach. Then there is one class of problems that has an extremely huge impact and occurs very frequently, for example, a pricing model for Amazon, Google's page rank algorithm, Search Engine Optimization, and others. These problems again require a completely different approach to solve. Here, we would need an approach that would be a combination of heuristics as well as algorithms blended with products.
Apart from these four obvious types of problems, we will have a special set of problems that has a flavor of all these types: moderate problems. Here, we might have a moderately good impact and frequency. Solving these problems requires a special approach. These are neither completely heuristic-based nor completely algorithmic. These are sweet spots for businesses where the tangible results can be experimented and validated very early and many companies can target for conceptualizations to deal with specific areas of the problem landscape:
When we explore the sweet spot, that is, the Circle of Uncertainty, we find that the problems again are of a different nature. They could be any one of the following:
Descriptive: What happened?
Inquisitive: How and why it happened?
Predictive: When will it happen?
Prescriptive: So what/now what?
To understand the nature of the problem, we basically try to ask what question is the solution answering. It could be what, how, when, why, and so on. Lets understand this better with a simple example.
Consider a Loyalty Program launched by a retail giant such as Walmart, where customers can use a Loyalty Membership Card to earn and burn cash points on each transaction. For simplicity, lets assume that the campaign ran for around three months and the Director of the Loyalty Program would like to know the answers to a few questions.
He would first like to know what happened?
This means how many people enrolled, how many transactions were recorded, how many products were sold, how many points were earned or burned, how much profit was earned during this season, how much revenue was generated, and so on. We basically get into the details of what happened during the period.
The nature of the problem that we are trying to solve here is Descriptive. The entire solution can be captured by asking one question-What happened?
Once he has a fair understanding of what happened, he would want to know about-Why it happened in a few scenarios. For example, he will have observed that sales from one particular geographical location, say Texas, have not increased in spite of the Loyalty Program, so he would like to understand specifically, why did that happen? Here, the problem solving will focus on understanding the reasons for no increase in sales in the Texas area when other areas have performed much better. We would then try to understand the Why question by digging deeper into the problem. We may study the offers that were rolled out to Texas compared to other regions or analyze how targeting customers and marketing campaigns differ between them and so on.
Here, the nature of the problem would be Inquisitive. The entire solution can be captured by asking one question-Why did it happen?
After understanding the reasons for why the event happen, we would probably want to take precautions to avoid failure due to the reasons found. Say, we found out that due to bad services, a lot of customers churned out to other competitors. We would then try to understand more about customer churn propensity where we would like to predict when the customer might churn out, so that we can take preventive measures to keep the customers happy.
Here, the nature of the problem would be Predictive. The entire solution can be captured by asking the question-When will the event happen?
Finally, once we have a complete picture of the series of events that happened and why and how it happened, we would like to take corrective measures to mitigate the perils of the event. So we would then ask Now what/So what, where we would try to seek guidelines for corrective actions. For example, we might have observed that due to bad service, a large number of customers churned out to other competitors and we would like to run customer retention programs and campaigns that could win back the churned customers.
Here, the nature of the problem would be Prescriptive. We can understand the entire solution with the question, Now what/So what?
To understand the nature of the problem better from an IoT perspective, consider the following example of an Oil and Gas Industry. Lets say that Shell, a leading oil company, has subsea operations set up in one of its prime locations. It would then deploy tons of machinery for the operations in order to extract oil from the reserves. In the IoT ecosystem, all the machinery or assets here would form a connected network where machines are equipped with a variety of sensors that can capture information about various real-time parameters and communicate to other machines and a central server. Assume that you are now the Operations Head for the plant and you are entitled with the responsibilities of executing the operations smoothly and effectively. As the head of Operations, at the end of the day, we would like to know what happened during the day in course of the oil extraction process. This would be answering the question, What happened? We would basically explore how much amount of oil was extracted, how many hours the machinery was under operation, and how many man hours and machine hours were utilized. This would be the basic set of analyses where the nature of the problem would be Descriptive. During the analysis, we discovered that the total amount of oil extracted today was extremely low compared to the threshold benchmarks and targets. We would then want to know what exactly happened, why the production decreased, and what were the causes. We would try to dig deeper into the problem and understand whether there was any issue with the workforce, did any device/equipment have a downtime, or whether any machine was underperforming. Here, the nature of problem would be Inquisitive, where we try to answer, Why did the event happen? Similarly, when we identify that the root cause for the problem was the downtime due to the failure of the drill machine deployed on the site, we would then try to understand when the assets would fail in future so that we could prepare in advance with maintenance and logistics to reduce downtime. A statistical model can be built that predicts the failure of an asset based on the real-time dimensions captured from the sensors to implement predictive maintenance for the assets to reduce downtime. This would be a classic Predictive problem. Finally, when the failure was catastrophic, we understood that we need to get a corrective action plan in place to reduce the effects to the best extent. We would get logistics ready for periodic maintenance of assets and condition-based maintenance for the machinery deployed on the site. Here, the nature of the problem is Prescriptive.
In a nutshell, we have explored the problem landscape and studied various dimensions of the problem. We studied how problems can be in different stages of the life cycle, how problems can be defined based on their type as low and high impact and frequency, and we also synthesized the nature of the problem, which can be Descriptive, Inquisitive, Predictive, or Prescriptive. Now that we have understood how the problem can be defined, lets move on to another important topic: understanding what it takes to solve a problem.
Now that we have a concrete understanding of how we can define the problem, lets try to understand what it takes to solve a problem. There could be a problem that could possibly be in any stage of its life, say fuzzy, the impact that it could create could be high with a moderately high frequency, and the nature of the problem could be predictive. Such a problem is really complicated if we try to understand it from its initial vibes. To make the example sound more concrete, lets assume that a renewable energy (solar) provider has one of its plants set up in a completely off-grid location to supply electric energy to a large college campus for its daily operations. The problem to solve would be predicting the amount of solar energy that would be generated based on weather and historic operational parameters. As the operations are completely off-grid, the admin of the campus would be keen to know the amount of energy that would be generated in the coming days so as to take necessary precautionary measures in cases of low production and high consumption. This would be a classic case of a predictive problem with a high impact and moderately high frequency and still in the fuzzy state. We know a few things about how to go about but no clear roadmap has been identified.
How do we solve this problem? What does it take in terms of skillsets or disciplines to get started with solving the problem? Decision Science, on a high level, takes multiple disciplines together to solve the problem. It generally takes the combination of math, business, and technology to design and execute the initial version of the solution, and then design thinking, behavioral science, and other disciplines to improvise the solution. Lets understand how and why this is required.
Solving the problem of predicting solar energy generation will initially require math skills where we would apply a variety of statistical and machine learning algorithms to get the predictions more and more accurate. Similarly, we would need technology skills to program in one or more computer languages based on the infrastructure where the data would be stored. The technology skills will help us extract data from various internal and external sources and clean, transform, and massage the data to render in the format where we can perform analysis. Finally, we will require business skills where we would have an understanding on how the college operates during the day, which operations are the most energy-consuming, how does the forecasted result add value for the college operations, and how do they plan to take precautionary actions for survival. The business skills required will make more sense if we would try to imagine a classic retail industry problem where we are trying to forecast the sales at a store level. We would need to take into account a variety of features and dimensions that are crucial from the business perspective but may be statistically insignificant. For example, the customer value bucket (high / medium / low) may appear insignificant mathematically during the analysis, but it would probably be one of the most crucial variables for business that may persuade us to consider the problem rather than ignore it.
Additionally, to get more and more granular in the problem solving phase, we would need skillsets on engineering and other disciplines. In our example, where we try to predict energy that would be generated in the future, a sound background of physics and engineering that would aid us in understanding the functioning of the photovoltaic cells and solar panel architecture and its engineering will be of great value when improving the solution becomes the core objective.
Similarly, in some other use cases, we would need disciplines of behavioral science and design thinking in more depth to study user behavior in specific scenarios and its implications in the business context. Thus, to solve any problem, we would need a curious mindset where our approach would be very interdisciplinary. With the use cases in IoT, the level of granularity of data that gets captured using sensors is altogether different. This mammoth and rich dataset now brings us the opportunity to deal with use cases at more and more granular levels than before. We can talk about use cases as abstract as increasing the product/asset life for an oil and gas refinery equipment or something as granular as reducing the vibrations in the gears of a diesel engine.
Now that we have a fair understanding about what skillsets are required to solve the business problem, lets try to understand how we go about solving the problem. Generally, the initial vibes that we get from a problem is the complexity. Not every problem is complicated; the simplicity of the problem is represented when it is broken down into smaller problems and we study how these smaller problems are connected to each other. Solution design gets easier when we think about one small problem at a time than the entire big problem.
Lets say that we are trying to solve the problem of increasing sales for a retailing customer. Here, increasing sales is the bigger problem that can be broken down into smaller and more focused problems where we deal with one small problem at a time. Increasing sales for a customer can be composed of smaller problems such as improving marketing campaigns, optimizing marketing channels, improving customer experience, designing customer retention programs, optimizing the supply chain model, and so on. The bigger problem can always be broken down into smaller and more focused problems. Similarly, when we solve one problem, it is also important to understand how these problems connect with other problems in the universe. The solution of the current problem may have a direct impact on another problem or solving this problem also requires solving the other connected problem. Here, we are talking about the art of problem solving rather than solving specific problems. Every problem is a part of a universe, where it may be connected to one or many other problems and may have a direct or indirect impact with other problems. Understanding the network of the problem is crucial before finalizing the design for our solution to the problem.
When we map the smaller problems connecting with each other to create the bigger problem, we have a universe of problems where each small problem can be identified with its life stage, nature, and type. Then we can solve each of these problems using a different approach meticulously drafted for its type and nature rather than using one generic approach. An incremental step-by-step approach to problem solving is not only time-saving but also impactful. The following diagram showcases the example discussed here visually. We can see how large problems are essentially smaller problems interconnected to each other:
You have learned about how the problem evolves through its life stages and how it can be represented using its type and nature. We also studied how Decision Science and problem solving require a curious mindset with an interdisciplinary approach for the solution. We also explored how problems are actually interconnected in nature and internally composed of smaller problems that can again be of a different type, nature, and life stage. Lets now go ahead and study the problem solving framework.
The problem solving framework basically represents a blueprint for the entire solution designed for the problem. Lets say that we are building a software or house; we would basically have an entire list of items that we need to acquire as resources and steps to be executed as per the plan for the end product to look as we have envisioned. Problem solving is also a similar story. Here, the first step is to break down the problem into smaller problems (in case the problem is a big one) and then gather an exhaustive list of hypotheses. To solve a problem, we basically collect a plethora of hypotheses and then test them to get results. Finally, we combine all the results together in such a way that we create a story where we have an answer to the question that we are trying to answer in the problems context. The hypotheses can be data-driven or heuristics-driven.
Lets take an example to understand how the problem solving framework looks. Consider the case of a hydroelectric power plant, where we have a small setup of devices necessary for the hydroelectric power generation: a turbine, generator, transformer, dam, penstock with an intake control gate, and other essential ones. These devices have their regular roles, such as the dam takes care of storing water for the hydroelectric plant, and there would be a penstock that is basically a long intake pipe that carries the water from the reservoir through controlled gates into a powerhouse that hosts the turbine. The turbine is a device equipped with large blades that rotate when water falls on them, and finally the generator generates AC electric energy from the rotation of these blades in the turbine. (Lets ignore the physics behind this to an extent.) The transformer then converts the electric energy to a higher voltage energy. In the entire flow, the gates of the penstock can be controlled to change the rate of the flow of water into the powerhouse:
So, what would be the problem?
You are the site engineer who has been asked the question: Why has the generation of hydroelectric energy been low for the past one month? Assume that you have no physical access to the location currently, but still you would first like to gather the maximum amount of information before you begin working on the site when you have access. This would be a scenario where you only have time at the site to fix the problem and not go there and then find out the root cause by testing and inspecting a couple of things. This is a use case that you can solve using data, data, and data. So, as a high-level approach, you would now use every dimension of data and find out the root cause that is possibly dipping the energy generation from the power plant.
Now that the context of the problem is clear, lets take a step back and try to understand more about the problem and then enter the problem solving framework. In this problem, we are trying to find out the root cause of an event-in a nutshell, we are trying to answer the question, Why did the event happen? This indicates that it is inquisitive in nature. Secondly, the problem is not radically new and neither has it been completely solved and tested to have an exhaustive solution guideline for all problems. Therefore, the problem is in the Fuzzy stage. Finally, the problem definitely has a high impact and is not a once-in-a-lifetime or once-in-a-couple-of-years event. We can probably conclude that the problem has moderate to high impact and moderate to high frequency. With the current landscape of the problem, we should probably build a product for this problem with a permanent automated solution. For now, lets explore the problem solving framework.
The framework is a very simple deal. If we are new to the business domain, we would first gather knowledge around the business domain before starting with the problem. In this scenario, we would explore the working of the hydroelectric workstation and how each component in the plant contributes to the overall process. Then we start collecting the list of hypotheses that can be a factor to the problems solution. So here, we lay down all the factors that could possibly be a reason for the root cause of the problem we are trying to solve:
In this scenario, we can think of a few factors that can be valid hypotheses for the root cause. For example, there would contamination in the oil of the transformer or there could be an oil spill. The rotors of the turbine could have overheated or the runners could have eroded. The amount of water flowing into the penstock and the water levels set in the gate control could be different, that is, water pressure in the penstock may be lower than usual, the RPM of the turbine blades is lower, or the critical parameters for the turbine have been operating at a lower value for a longer duration. Similarly, the critical parameters for the transformer or generator could have been performing beyond the normal operating range for a longer duration. Oil levels in the gears of the devices may be below the ideal point or the devices may be operating at a temperature beyond the normal range. For these devices, we would have multiple parameters in place that define the status of operation for the devices and deviance from normal operations. A closer look at these parameters will help us define the entire picture of the power plant. All of these factors that build our initial layer of root cause analyses forms the collection of heuristic-driven hypotheses.
Once we have the heuristics-driven hypotheses defined, we have action items in place to test what went wrong. We can individually test these hypotheses, evaluate the results from them, and gather insights. Secondly, our collection of hypotheses is still not exhaustive. We might have missed out on tons of important correlated factors that are probably latent in nature and might only be visible when we explore the data in more detail. Lets keep the data-driven hypotheses aside now. (This will be more realistic as we move to Chapter 3, The What And Why - Using Exploratory Decision Science for IoT). Consider a usual problem solving approach where we have collected a couple of heuristic-driven hypotheses, performed exploratory data analysis for the data, and tested the hypotheses collected. We would find that a few of the hypotheses that we designed earlier are not accurate as the results were not intuitive. We may discard a few hypotheses and prioritize a few others. We would also find a lot of new relationships between data dimensions that we had not accounted for initially. Now, if we revisit the previous list of hypotheses, we would probably have a version with better and more accurate hypotheses and also new hypotheses that we uncovered during our data exploration exercises. The new hypotheses may not be our final version. It may go through a series of iterations before it gets finalized. The refined final list of hypotheses can be called the problem solving framework. This is where the convergence of data-driven hypotheses and heuristic-driven hypotheses takes place. This can be represented as a matrix or prioritized list of hypotheses, which needs to be validated to solve the problem:
The initial list may have a few hypotheses that may not make sense to test as there may be data limitations or they may be counter intuitive with a few latent data relationships that we explored during our analysis. Once all the hypotheses have been tested, we will have results gathered from various tests from them under a single roof. The next step is to assimilate the results so that we can make sense of the story from the results. Synthesizing the results assimilated, we may find the root cause for the event as a result of a few other events. Lets say that while gathering results from our data, we might conclude that the malfunctioning of the controlled gates in the penstock was the root cause. This might be inferred from the critical parameters of the turbine and generator operating at lower thresholds continuously for a prolonged stint. A few data tests on the water pressure and its correlation with the value from the controlled gates differing over a period of time can be an indicative value for the same.
In a nutshell, we have glanced through a very high-level problem with a structured approach using the problem solving framework. The problem solving framework is a simplified approach to design and draft the exhaustive hypotheses generated as a result of the convergence of heuristics and data exploration. With the exhaustive list of hypotheses, we conduct various data tests and assimilate results from them to synthesize and solve the problem by gathering insights and creating a story. In the coming chapters, we will solve real business problems using the problem solving framework and also understand each incremental phase in more detail.
This chapter was an introduction to Decision Science and IoT. You learned about the basics of IoT and how it evolved and understood the differences between ambiguous names such as M2M, IIoT, IoE, and others. We studied the logical architecture of the IoT ecosystem by considering IoE and learned how People, Processes, Data, and Things together form the IoT ecosystem. We also discussed decision science and understood more about defining a problem based on the current life stage as Muddy, Fuzzy, or Clear, based on its type as Impactful and Frequent, and finally based on its nature as Descriptive, Inquisitive, Predictive, or Prescriptive. We also studied that problem solving in decision science requires an interdisciplinary approach using a combination of math, business, technology, and so on. Finally, we also studied about the problem solving framework using a generic example of a hydroelectric power plant.
In the next chapter, you will learn in depth about the IoT problem universe and use a concrete example to understand the problem and design the blueprint for the problem using the problem solving framework.