Chapter 3. Networking, Load Balancers, and Ingress

In this chapter, we will be covering how the Kubernetes cluster handles networking and how it differs from other approaches. We will be describing the three requirements for Kubernetes networking solutions and exploring why these are key to ease of operations. Further, we will take a deeper dive into services and how the Kubernetes proxy works on each node. Finishing up, we will see a brief overview of some higher level isolation features for multitenancy.

This chapter will discuss the following:

• Kubernetes networking
• Advanced services concepts
• Service discovery
• DNS
• Namespace limits and quotas

Kubernetes networking

Networking is a vital concern for production-level operations. At a service level, we need a reliable way for our application components to find and communicate with each other. Introducing containers and clustering into the mix makes things more complex as we now have multiple networking namespaces to bear in mind. Communication and discovery now becomes a feat that must traverse container IP space, host networking, and sometimes even multiple data center network topologies.

Kubernetes benefits here from getting its ancestry from the clustering tools used by Google for the past decade. Networking is one area where Google has outpaced the competition with one of the largest networks on the planet. Earlier, Google built its own hardware switches and Software-defined Networking (SDN) to give them more control, redundancy, and efficiency in their day-to-day network operations (you can refer to more details about this in point 1 in the References section at the end of the...

Advanced services

Let's explore the IP strategy as it relates to services and communication between containers. If you recall, in the Services section, Chapter 2, Pods, Services, Replication Controllers, and Labels, you learned that Kubernetes is using kube-proxy to determine the proper pod IP address and port serving each request. Behind the scenes, kube-proxy is actually using virtual IPs and iptables to make all this magic work.

Kube-proxy now has two modes—userspace and iptables. As of now, 1.2 iptables is the default mode. In both modes, kube-proxy is running on every host. Its first duty is to monitor the API from the Kubernetes master. Any updates to services will trigger an update to iptables from kube-proxy. For example, when a new service is created, a virtual IP address is chosen and a rule in iptables is set, which will direct its traffic to kube-proxy via a random port. Thus, we now have a way to capture service-destined traffic on this node. Since kube-proxy is running on all...

Service discovery

As we discussed earlier, the Kubernetes master keeps track of all service definitions and updates. Discovery can occur in one of three ways. The first two methods use Linux environment variables. There is support for the Docker link style of environment variables, but Kubernetes also has its own naming convention. Here is an example of what our node-js service example might look like using K8s environment variables (note IPs will vary):

NODE_JS_PORT_80_TCP=tcp://10.0.103.215:80
NODE_JS_PORT=tcp://10.0.103.215:80
NODE_JS_PORT_80_TCP_PROTO=tcp
NODE_JS_PORT_80_TCP_PORT=80
NODE_JS_SERVICE_HOST=10.0.103.215
NODE_JS_PORT_80_TCP_ADDR=10.0.103.215
NODE_JS_SERVICE_PORT=80

Listing 3-13: Service environment variables

Another option for discovery is through DNS. While environment variables can be useful when DNS is not available, it has drawbacks. The system only creates variables at creation time, so services that come online later will not be discovered or would require some additional...

DNS

DNS solves the issues seen with environment variables by allowing us to reference the services by their name. As services restart, scale out, or appear anew, the DNS entries will be updating and ensuring that the service name always points to the latest infrastructure. DNS is set up by default in most of the supported providers.

Note

If DNS is supported by your provider, but not set up, you can configure the following variables in your default provider config when you create your Kubernetes cluster:ENABLE_CLUSTER_DNS="${KUBE_ENABLE_CLUSTER_DNS:-true}"DNS_SERVER_IP="10.0.0.10" DNS_DOMAIN="cluster.local" DNS_REPLICAS=1

With DNS active, services can be accessed in one of two forms—either the service name itself, <service-name> or a fully qualified name that includes the namespace, <service-name>.<namespace-name>.cluster.local. In our examples, it would look similar to node-js-90 or node-js-90.default.cluster.local.

Multitenancy

Kubernetes also has an additional construct for isolation at the cluster level. In most cases, you can run Kubernetes and never worry about namespaces; everything will run in the default namespace if not specified. However, in cases where you run multitenancy communities or want broad-scale segregation and isolation of the cluster resources, namespaces can be used to this end.

To start, Kubernetes has two namespaces—default and kube-system. The kube-system namespace is used for all the system-level containers we saw in Chapter 1, Introduction to Kubernetes, in the Services running on the minions section. The UI, logging, DNS, and so on are all run in kube-system. Everything else the user creates runs in the default namespace. However, our resource definition files can optionally specify a custom namespace. For the sake of experimenting, let's take a look at how to build a new namespace.

First, we'll need to create a namespace definition file like the one in this listing:

apiVersion...

A note on resource usage

As most of the examples in this book utilize GCP or AWS, it can be costly to keep everything running. It's also easy to run out of resources using the default cluster size, especially if you keep every example running. Therefore, you may want to delete older pods, replication controllers, replica sets, and services periodically. You can also destroy the cluster and recreate using Chapter 1, Introduction to Kubernetes as a way to lower your cloud provider bill.

Summary

We took a deeper look into networking and services in Kubernetes. You should now understand how networking communications are designed in K8s and feel comfortable accessing your services internally and externally. We saw how kube-proxy balances traffic both locally and across the cluster. Additionally, we explored the new Ingress resources that allow us finer control of incoming traffic. We also looked briefly at how DNS and service discovery is achieved in Kubernetes. We finished off with quick look at namespace and isolation for multitenancy.

References

1. http://www.wired.com/2015/06/google-reveals-secret-gear-connects-online-empire/