Handling Data on the Cloud

What is cloud computing? By definition, it is the idea of getting things done with someone else’s computer. Have you noticed that little cloud-shaped icon in the “shapes and icons” collection in word editors that we use to represent the internet, in our flow charts and architecture diagrams? That is where it started. The idea of hosting any service on the internet became known as cloud computing, inspired by the icon. Cloud computing was created to accommodate the exponentially growing volumes of data, resources, and services that organizations use, which has been causing more expense and a lot of maintenance effort for several decades now. It is the kind of technology where we store, process, maintain, and visualize applications and data and other dependencies in a central, managed environment, basically with someone else’s resources (the provider who hosts and manages).

In this chapter, we’ll dive into the details of...

Types of cloud services

Cloud services are divided into three main types:

• A public cloud provides open and accessible public services on the internet. It is a cloud computing service that provides computing resources, such as virtual machines, storage, and networking, over the internet to multiple customers or users. It is operated and maintained by a cloud service provider and is made available to the public on a pay-as-you-go or subscription basis.
• A private cloud provides hosted services to a private group of people, mostly within an organization, with access restrictions and permissions in place in terms of infrastructure, network, and data. Unlike public clouds, which serve multiple customers, a private cloud is dedicated to a single tenant. It can be hosted on-premises within an organization’s own data center or infrastructure, or it can be hosted by a third-party provider off-site. This is preferred when organizations require greater control, security, and...

Use case categories

Use cases are divided broadly into three main categories:

• Infrastructure as a Service (IaaS) is based on a model where the infrastructure that the organization needs is rented, but virtually. In the IaaS model, cloud service providers deliver fundamental IT infrastructure components, including virtual machines, storage, and networking, to users on a pay-as-you-go basis.

  With IaaS, organizations can offload the management and maintenance of physical hardware and data centers, allowing them to focus on building and managing their applications and services. Users have the flexibility to scale resources up or down based on their specific needs, making it an ideal choice for businesses with variable workloads or those seeking cost-effective solutions.

• Platform as a Service (PaaS) is based on a model where the platform for development, designing, and deploying software applications is offered without having to worry about the server, storage, maintenance, security...

The benefits of cloud computing

There are numerous benefits to cloud computing. Let’s look at a few of the most important ones to our topic:

• Ease of use: The programs and applications (in the case of SaaS) are exclusively managed by vendors, and you do not need to worry about how to run an application.
• Customization: Even updates to existing applications have become easy, with SaaS involving a point-and-click method and no need to change code extensively.
• Pay-as-you-go: You pay only for what you use and can terminate or pause services at any time.
• Avoiding the cost of infrastructure: Build and deploy applications without worrying about the cost of compute and other resources, such as servers, operating systems, and networks.
• Pay only for what you need: You choose what you want to use and pay only for those features and services, depending on the service, and you do not have to be concerned about paying for anything else.
• Upgrades and updates: You...

Data applications on cloud

Storing information on the cloud allows you access from anywhere with internet connectivity. It allows data to scale up or down exponentially with elasticity. This is the main advantage for any organization migrating to the cloud because, this way, they can focus on ideas and innovation for business transformation and agility, rather than worrying about data management and infrastructure to support that. Let us take a look at some areas of application for data on the cloud in this section.

Storage

Cloud computing manages storage and access for organizations’ data in any format – files, audio, video, images, tables, documents, and so on. The need and application to have a storage area that can be accessed from anywhere on any device at any time (as per the agreed service-level agreements) are critical these days for all businesses across industries. Cloud providers also support this with easy-to-use interfaces that allow interactive access...

Managed, unmanaged, and database as a service

Handling data on the cloud involves storage and access, and this is enabled using databases. It is not a one-size-fits-all choice. There are several options and criteria to make the right database choice for your application and product, which we discussed in detail in Chapter 1, Data, Databases, and Design of this book. In this section, let us look at the managed and unmanaged types of database categories.

Managed databases

When your team needs to focus more on creating an application and product rather than focusing on operations and maintenance, you need a solution or a service that gives you those capabilities by shifting the infrastructure, maintenance, and latency requirements to the cloud provider. That is exactly what a managed database provides.

Managed databases provide users the ability to maintain clusters, routine updates, scalability, backup, recovery, replication, latency, availability, and administration in a very...

Cloud database considerations

There are a few considerations when we decide to move to cloud databases. The previous chapter covered the various business, technical, and design considerations you must go through before choosing the right database for your application or business. Here, we want to address a few more cloud-specific aspects to keep in mind while approaching a cloud database design:

• Security, regulations, privacy, and compliance: Security is the most important software design and delivery factor, and it sometimes can pose a challenge as well, depending upon the nature of the data and the complexity of your requirements. Teams and organizations should pay attention to cloud security standards, configurations, regulatory requirements for their product, regional (geo-specific) requirements, encryption features and stages, business policies, and other best practices and guidelines.
• Cost: Even though we have established that you only pay as you go or pay per use...

A quick follow-up

Remember, in the last chapter, when we discussed the different database categories and decided to defer discussing examples of each of those categories to the following chapter? Let’s address that here! So, now that you know what cloud computing is and the role it plays in handling data and applications, we should be able to look at cloud database examples for each type:

• Relational database:
  • Online Transaction Processing (OLTP): Spanner, Cloud SQL, and Alloy DB
  • Online Analytical Processing (OLAP): BigQuery
• NoSQL database:
  • Document database: Firestore
  • Key-value database: Memorystore
  • Wide-column database: Bigtable
  • Graph database: JanusGraph

We will discuss some of these databases in detail with real-world use cases, hands-on exercises, and design practices in the upcoming chapters of this book.

Summary

Now that we have discussed what cloud computing is, and its implications, types, ways of handling data on the cloud, use cases that determine those methods, real-world applications, and considerations to make the right choice for your data requirements, you should be in a better position to design a robust, scalable, and cost-efficient cloud database infrastructure that meets the needs of your applications and users. However, before that, you should learn about the various data formats, the different types databases to handle that data and the real-world applications, hands-on. The following chapter is a short introduction to structured data, its database design considerations, its applications, and the Google Cloud options with real world use-cases and hands-on implementations that inform your design.