Most probably, if you are reading this book, you have already heard about the serverless paradigm and the terms serverless engineering and serverless architecture. Nowadays, the way developers deploy applications has changed drastically, especially in the domain of data engineering and web development, thanks to event-based architectural designs, also called serverless architectures.

It is not uncommon to have idle resources and servers in production idle after the server workload has finished, or waiting for the next workload to come. This introduces a bit of redundancy in the infrastructure. What if there was no need for idle resources lying around when there is no workload? What if resources can be created when necessary and be destroyed once the work is done?

At the end of this chapter, you will understand how serverless architectures and functions as a service work, and how you can build them into your existing software infrastructure. You will also learn what microservices are, and decide whether microservices or serverless operations are well-suited for your architecture or not. You will also learn how to build serverless applications with Python on major cloud service providers, such as Amazon Web Services (AWS) and Microsoft's Azure.

This chapter will cover the following points:

• Understanding serverless architectures
• Understanding microservices
• Serverless architectures don't have to be real-time only
• Pros and cons of serverless architectures

The concept of serverless architectures or serverless engineering revolves entirely around understanding the concept of functions as a service. The most technical and accurate definition of serverless computing on the internet is as follows:

Now, let's go into the details of each part of that definition to understand the paradigm of serverless computing better. We shall start with the term function as a service. It means that every serverless model has a function that is executed on the cloud. These functions are nothing but blocks of code, that are executed depending on the trigger that is associated with the function. This is a complete list of triggers in the AWS Lambda environment:

Now let's understand what manages the starting and stopping of a function. Whenever a function is triggered via one of these available triggers, the cloud provider launches a container in which the function executes. Also, after the function is successfully executed the function has returned something, or if the function has run out of time, the container gets thatched away or destroyed. The thatching happens so that the container can be reused in the event of high demand and whenever there is very little time between two triggers. Now, we come to the next part of the sentence, the function's container. This means that the functions are launched and executed in containers. This is the standard definition of a container from Docker, a company that made the concept of containers very popular:

This helps in packaging the code, the runtime environment, and so on of the function into a single deployment package for seamless execution. The deployment package contains the main code file for the function, all the non-standard libraries which are required for the function to execute. The creation process of a deployment package looks very similar to that of a virtual environment in Python.

So, we can clearly make out that there are no servers running round the clock in the case of serverless infrastructures. There is a clear benefit for this, which includes not having a dedicated Ops team member for monitoring the server boxes. So the extra member, if any, can focus on better things, such as software research, and so on. Not having servers running through the entire day saves a lot of money and resources for the company and/or personally. This benefit can be very clearly seen among machine learning and data engineering teams who make use of GPU instances for their regular workload. So having on-demand serverless GPU instances running, saves a lot of money without the developers or the Ops team needing to maintain them around the clock.

Similar to the concept of serverless, the design strategy, which is the microservice-oriented strategy, has also been very popular recently. This architecture design existed a long time before the idea of serverless came into existence though. Just as we tried to understand the serverless architectures from the technical definition on the internet, we shall try to do the same for microservices. The technical definition for microservices is:

Planning and designing the architecture in the form of microservices has its fair share of positives and negatives, just like serverless architectures. It's important to know about both, in order to appreciate and understand when and when not to leverage microservices in your existing architecture. Let's look at this and understand the positives of having microservice architectures, before moving over to the negatives.

Microservices help software teams stay agile, and improve incrementally. In simpler terms, as the services are decoupled from each other, it is very easy to upgrade and improve a service without causing the other to go down. For example, in social network software, if the chat and the feed are both microservices, then the feed doesn't have to go down when the software team are trying to upgrade or do minor fixes on the chat service. However, in large monolithic systems, it is difficult to break things up so easily in the way one can do with microservices. So, any fix or upgrade on even a small component of the architecture comes with downtime with the fix taking more time than intended.

The sheer size of the code base of monolithic architectures itself acts as a hindrance progress in the case of any small failures. Microservices, on the other hand, greatly help in boosting developer productivity by keeping code bases lean, so that they can fix and improve the service with very little or no overhead and downtime. Microservices can be much better leveraged via containers, which provide effective and complete virtual operating system environments, processes with isolation, and dedicated access to underlying hardware resources.

However, microservices come with their own bunch of disadvantages and downsides, the major one being having to deal with distributed systems. Now that each service is surviving on its own, the architect needs to figure out how each of them interacts with the others in order to make a fully functional product. So, proper co-ordination between the services and the decisions regarding how services move data between them is a very difficult choice that needs to be taken by the architect. Major distributed problems such as the consensus, the CAP theorem, and maintaining the stability of consensus, and the connection, are some issues that the engineer needs to handle while architecting for microservices. Ensuring and maintaining security is also a major problem in distributed systems and microservices. You needs to decide on separate security patterns and layers for each microservice, along with the security decisions necessary for the data interaction to happen between the services.

Serverless architectures generally are leveraged as real-time systems as they work as a function as service which is triggered by a set of available triggers. However, this is a very common misconception, as serverless systems can be leveraged equally well both as real-time and batch architectures. Knowing how to leverage the concept of serverless systems as batch architectures will open up many engineering possibilities, as all engineering teams don't necessarily need or have real-time systems to operate.

Serverless systems can be batched by leveraging the following:

• The cron facility in triggers
• The concept of queues

Firstly, let's understand the concept of the cron facility in triggers. Serverless systems on the cloud have the ability to set up monitoring, which enables the trigger to get triggered every few minutes or hours and can be set as a normal cron job. This helps in leveraging the concept of serverless as a regular cron batch job. In the AWS environment, Lambda can be triggered as a cron via AWS CloudWatch, by setting the frequency of the cron by manually entering the time interval as the input and also by entering the interval in the cron format:

One can also leverage the concept of queues when building serverless batch architectures. Let's understand this by setting an example data pipeline. Let's say the system which we intend to build does the following tasks:

1. A user or a service sends some data into a database or a much simpler data store, such as AWS's S3.
2. Once there are more than 100 files in my data store, we'll want to do some task. Let's say, doing some analytics on them, for example, such as counting the pages.

This can be achieved via queues, and this is one of the simpler serverless systems we can consider as an example. So, this can be achieved as follows:

1. The user or the service uploads or sends the data to the data store which we have selected.
2. A queue is configured for the purpose of this task.
3. An event can be configured to S3 buckets or data stores so that as soon as data enters into the store, a message is sent to the queue which we have configured earlier.
4. Monitoring systems can be set to monitor the queue for the number of messages in it. It is advisable to use the monitoring system of the cloud provider you are using so that the system stays completely serverless.
5. Alarms can be set to the monitoring systems, configuring a threshold for these alarms. For example, the alarm needs to be triggered whenever the number of messages in our queue reaches or exceeds 100.
6. This alarm can act as a trigger to the Lambda function which does the analytics by first receiving messages from the queue and then querying the data store using the filename received from the message.
7. Once the analytics are completed on the files, the processed files can be pushed to another data store for storage.
8. After the entire task is completed, the container or the server where the Lambda function has run will be terminated, thus making this pipeline completely serverless.

As we now understand what serverless architectures and pipelines look like, how they may be leveraged into existing architectures, and also how microservices help keep architectures leaner and boost developer productivity, we shall look at the pros and cons of serverless systems in detail, so that software developers and architects can make decisions regarding when to leverage the serverless paradigm into their existing systems and when not to.

The positives of serverless systems are:

• Lower infrastructure costs: By deploying serverless systems, the infrastructure costs can be greatly optimized, as there would not be a need for servers to be running around the clock every day. As the servers start whenever the function is triggered, and stop whenever the function gets executed successfully, the billing would only be done for that brief time period when the function was running.
• Less maintenance needed: By virtue of the preceding reason, there is also no need for continuous monitoring and maintenance of servers. As the functions and triggers are automated, there is almost zero maintenance required for serverless systems.
• Higher developer productivity: As the developers don't need to worry about downtime and server maintenance, they can focus and work on better software challenges, such as scaling and designing functionalities.

The remaining part of the book will show you how serverless systems are changing the way software is done. So, as this chapter is intended to help architects decide whether serverless systems are a good choice for their architecture or not, we shall now look at the disadvantages of serverless systems.

The disadvantages of serverless systems are:

• Time limit of the function: The function which is whether executed, be it AWS's Lambda or GCP's cloud functions, has an upper time limit of 5 minutes. This makes execution of heavy computations impossible. However, this can be solved by executing a provisioning tool's playbook in nohup mode. This will be covered in detail, later in the chapter. However, making the playbook ready and setting up the container and anything else should be completed within the 5 minute time limit. The container gets automatically killed when the 5 minute limit is exceeded.
• No control over the container environment: The developer has no control over the environment of the container that is being created for executing the function. The operating system, the filesystem, and so on, are all decided by the cloud provider. For example, AWS's Lambda functions are executed inside containers that run the Amazon Linux operating system.
• Monitoring containers: Apart from the basic monitoring capabilities that are provided by the cloud provider via their in-house monitoring tools, there is no mechanism to do detailed monitoring of the container that is executing the serverless function. This becomes even more difficult when scaling up serverless systems to accommodate distributed systems.
• No control on security: There is no control on how the security of the data flow is ensured, as there is very little control over the container's environment. The container can be run in the VPC and subnets of the developer's choice, though, which helps work around this disadvantage.

However, serverless systems can be scaled up to distributed systems for large- scale computations where the developer need not worry about the time limit. As already mentioned, this will be discussed in detail in the upcoming chapters. However, for insight into an intuition on how one can choose serverless systems over monolithic systems for large-scale computations, let us understand some important pointers that need to be kept in mind when taking that architectural decision.

The pointers to be kept in mind when scaling serverless systems to distributed systems are:

• To scale up serverless systems into serverless distributed systems, one must understand how the concept of nohup works. It is a POSIX command that allows programs and processes to run in the background.
• Nohup processes should be properly logged, including both the output and the error logs. This is the only source of information for your processes.
• A provisioning tool, such as Ansible or Chef or a similar one, needs to be leveraged to create a master-workers architecture which has been spawned via the playbook running in nohup mode in the container where the serverless function is being executed.
• It is a good practice to ensure that all tasks that are being executed by the provisioning tool via the master server are properly monitored and logged, as there is no way one can retrieve the logs once the entire setup finishes executing.
• Proper security needs to be ensured by using a temporary credential facility available from the cloud providers.
• Proper closure should be ensured for the system. The workers and the master should self-terminate immediately after the pipeline of tasks finishes executing. This is very important and this is what makes the system serverless.
• Generally, temporary credentials come with an expiry time, which is 3,600 seconds for most environments. So, if the developer is using temporary credentials to execute a task which is supposed to take more than the expiry time, then there is a danger of the credentials getting expired.
• Debugging distributed serverless systems is an extremely difficult task for the following reasons:
  • Monitoring and debugging a nohup process is extremely difficult. Whenever you want to debug one, you have to either refer to the log file created by the process or kill the nohup process by using the process ID, and then manually run the scripts for debugging.
  • As the complete list of tasks executes sequentially in the provisioning tool, there is a danger that the instances may get terminated because the developer has forgotten to kill the nohup process before starting the debugging process.
  • As this is a distributed system, it goes without saying that the architecture should be able to self-heal in the case of any failure or a disaster. An example scenario can be when one of the workers goes down while performing some operation on a bunch of files. The entire bunch of files is now lost, and there is no means of recovery.
  • Another advanced disaster scenario can be when two worker servers go down while performing some operations on a bunch of files. In this case, the developer does not know which files have been executed successfully and which haven't.
• It is a good practice to ensure that all the worker instances receive an equal amount of the load to execute so that the load across the distributed system stays even and time and resources are well optimized.

In this chapter, we learned what serverless architecture is. Most importantly, the chapter helps architects decide if serverless is the way forward for their team and their engineering, and how to transition/migrate from their existing infrastructure to a serverless paradigm. We also looked at the paradigm of microservices and how they help make lightweight and highly agile architectures. This chapter also went into great detail about when a team should start thinking about microservices and when can they migrate or break their existing monolith(s) into microservices.

We then learned the art of building batch architectures in the serverless domain. One of the most common myths is that serverless systems are only for real-time computation purposes. However, we have learned how to leverage these systems for batch computations too, thus facilitating a whole lot of applications with the serverless paradigm. We looked at the pros and cons of going serverless so that better engineering decisions can be taken accordingly.

In the next chapter, we will cover a very detailed understanding of how AWS Lambda works, which is the core component of serverless engineering in the AWS cloud environment. We will understand how triggers work and how AWS Lambda functions work. You will learn about the concept of leveraging containers for executing serverless functions and the associated computational workload. Following that, we will also learn about configuring and testing Lambda functions, along with understanding the best practices while doing so. We will also cover versioning Lambda functions, in the same way versioning is done in code, and then create deployment packages for AWS Lambda, so that the developer can accommodate third-party libraries comfortably, along with the standard library ones.