VMware provides a complete end-to-end cloud platform and solution using VMware vCloud Director, which is built on VMware technologies and solutions to deliver cloud computing. Cloud computing brought a new approach to computing that leverages efficient pooling of an on-demand, self-managed virtual infrastructure to provide resources consumable as a service.
In this chapter, we will cover the following aspects:
Installing vCloud Director
Basic vCloud configuration
Security hardening of vCloud in a nutshell
Looking at a simple high-level cloud architecture, it might contain a VMware vCloud Director server or a group comprising of multiple vCloud Director servers. Each server can run a collection of services called a vCloud Director cell.
The following figure shows the vCloud architecture and depicts the core architecture and the optional components of vCloud. Though you can have multiple vCloud Director servers in a group, all the vCloud Director servers in the group share a single vCloud Director database. To provide resources for cloud tenants, vCloud Director (vCD) connects to one or more VMware vCenter Server systems and the VMware ESXi hosts.
VMware uses one VMware vCloud Networking and Security server for each vCenter Server instance, that is, the vCloud Networking and Security manager always has a one-to-one relationship with vCenter. vCloud Networking and Security servers provide network security services and deploy VMware the vCloud Networking and Security Edge devices (virtual appliances) on demand from vCloud Director to provide static routing, VPN, NAT, DHCP, gateway, and firewall services. This not only enables vCloud Director to provide multitenancy but also a provides a foundation for Software Defined Networking (SDN), which allows network connectivity that is programmable and decoupled from the physical infrastructure. Thus it enables workloads to be placed and moved anywhere.
vCloud Director uses vSphere to provide the CPU and memory to run virtual machines. For virtual machine networking, it uses vSphere's Distributed Switches and Standard vSwitch as well. However, the vSphere Distributed Switch must be used for cross-host fencing and network pool allocation. vSphere VMFS (Virtual Machine File System) datastores provide storage for virtual machine files and other files necessary for virtual machine operations. These underlying vSphere resources are used by vCloud Director to create cloud resources. This is depicted in the following figure:
vSphere clusters should be enabled with VMware vSphere Distributed Resource Scheduler (DRS) that should set to balance the vCloud Director deployed workloads across the physically compute resources of the vSphere DRS cluster. You can define a single cluster for the cloud provider resource or use multiple vSphere resource pools to provide the cloud provider resource. Though resource pools are supported, the best way to use them is in a cluster-wise format from a scaling perspective.
Let us take a closer look at the vCloud side. A vCloud Director Server group consists of one or more vCloud Director servers, which are also called vCloud cells. These servers share a common database and are linked to the vCenter Server systems and ESXi hosts. The vCloud Networking and Security servers provide network services for vCloud Director. If you want to segregate and allocate vCloud resources to the organizations, there is a web-based portal for vCloud administrators to do this. This web-based portal can be used for each organization as well and can provide consumers with the means to create and manage their own virtual machines. However, access is controlled through a role-based model set up by the organization administrator. A vCloud administrator has the ability to set the lease time to control how long vApps can run and be stored.
Let us look at the hybrid cloud scenario:
vCloud Connector (vCC) is a key differentiator in the vCloud Suite for making hybrid cloud.
vCC helps customers realize the hybrid cloud vision by providing them with a single pane of glass to view, operate, and copy VMs/vApps/templates across vSphere/vCloud Director and vCloud Service Providers.
The following diagram gives an overview of this scenario:
vCloud administrators can also set quotas that limit the number of virtual machines that an organization can have, define an isolated or shared network, have complete control of the network flow, have preestablished pools of resources, and implement security policies. The following figure shows the vCloud components and the integration of them:
Other than the core vCloud components, you can also add other VMware components to increase the capabilities or control. One example is VMware vCenter Chargeback. vCenter Chargeback provides resource metering and reporting to facilitate resource chargeback. vCenter Chargeback comprises of the vCenter Chargeback server and vCenter Chargeback data collector. Though a Chargeback component is optional, it is a must to meet the NIST (National Institute of Standards and Technology) cloud computing definition. Another additional component is VMware vCloud Connector. vCloud Connector helps facilitate the transfer of a "powered-off" vApp in the Open Virtualization Format (OVF) format from a local cloud (this could also be vSphere) to a remote cloud or a vSphere instance. vCloud Connector is a virtual appliance that is installed in vSphere and handles all the logic of dealing with other clouds. The GUI is displayed in the VMware vSphere Web Client or the C# client through the vCloud Connector browser plugin.
vCloud management cluster is a VMware vSphere High Availability (HA) and vSphere DRS (Distributed Resources Scheduler) cluster that is created to manage a vCloud architecture. A management cluster contains the standard management components, such as ESXi hosts, vCenter Server system, vCloud Director cell servers, database server/s for vCloud Director, and vCenter. A management cluster should have its own shared storage that will store the virtual machines running inside the management cluster. The management cluster should also be separated into a single physical site. We would like to emphasize that for the cloud, it is a must to have a separate management cluster. It is a best practice to place the management components in a management cluster.
You should use vSphere HA and DRS on the management cluster to provide availability for all the management components. For vSphere HA, use the Percentage of Cluster Resources Reserved admission control policy in an n + 1 fashion instead of defining the amount of host failures a cluster can tolerate or specifying the failover hosts. This approach will help you to allow management workloads run evenly across the hosts in the cluster without the need to dedicate a host strictly for host failure situations. But this is not just limited to n + 1; for higher availability, you can add a host for an n + 2 cluster, although doing so is not a requirement of the vCloud private or public service definitions.
You may be wondering why you need a vCenter Server inside your vCloud management cluster. This management vCenter Server will carry clusters that will host cloud workloads. These resources are allocated by vCloud Director as a provider datacenters. Within a distinct vSphere cluster, a provider datacenter translates into a resource pool that is created automatically by vCenter, issued on a request from vCloud Director.
Although you can physically separate the management cluster and resource cluster, it is not a good practice to do so. You should put the management cluster and vCloud consumer resources on the same physical site. If you use a single site, it ensures a consistent level of service. Otherwise, latency issues might arise if workloads must be moved from one site to another.
Even before you start the installation of the vCloud, you should remember that this is a complex system and thus requires proper planning for the installation. If you choose the correct steps and choices, you can save a lot of time during the installation.
For installing vCloud Director, there are lots of prerequisites that have to be in place before you can proceed further. Let us look at those:
vCenter Server for the resource cluster should set HA, DRS, and Storage DRS.
vCenter Server should trust their ESXi hosts.
Use proper vSphere licenses. If you use vSphere Distributed Switch, the Enterprise Plus license is necessary. If not, you need to use the Enterprise license for DRS. For the private or public cloud, the Enterprise Plus license is a must to provide cloud-level scaling.
vCloud Networking and Security Manager needs to be installed before installing vCloud. The vCloud Networking and Security Manager can be downloaded as an OVF appliance and can be easily deployed as a VM in your management network. The vCloud Networking and Security Manager manages the vCloud Networking and Security Edge appliances and Virtual Extensible LAN (VXLAN) (software-defined Layer 2 networking) for providing redundancy and isolation of the network inside your cluster. In other components, vShield also provides the Endpoint and Data Security components for your VMs. vCloud Networking and Security Manager should be properly licensed. A basic license for the vCloud Networking and Security is included with vCloud Director 5.1, but it does not include advanced features. If you would like to know more, take a look at this article: http://kb.vmware.com/kb/2042799.
VMware strongly recommends that vCenter Server 5.1 and ESXi 5.1 be used with vCloud Director 5.1. Although earlier versions are supported, some features are not available if these earlier versions are used.
Check the supported operating system for the vCloud Director cell. vCloud Director Server requires Linux OS. Red Hat Enterprise Linux 5 (64 bit), update 4, 5, or 6 is supported. In addition, Red Hat Enterprise Linux 6 (64 bit), update 1 or 2 is supported.
The minimum hardware requirement for a vCloud Director cell requires 950 MB free on disk and 1 GB of memory (RAM). For better performance, 2 GB of RAM is recommended as with 1 GB RAM, it sometimes becomes irresponsive.
The minimum Java version required for the cell is Java Runtime Environment (JRE) 1.6.0 update 10 or later. Only the 32-bit version is supported.
vCloud Director requires Adobe Flash Player version.
The database that will be used by vCloud Director must be created before installing the first vCloud Director cell.
Before configuring vCloud Director, you must install security certificates.
You must use the JRE
keytoolcommand to create your certificate requests.
Transfer Server Storage is used as a temporary storage for uploads and downloads. It must be mounted at
On the internal networks, only a few ports should be open for vCloud Director servers. See the VMware knowledge base article 1030816 at http://kb.vmware.com/kb/1030816.
For more information, please see the VMware vCloud Director 5.1 Documentation Center at http://pubs.vmware.com/vcd-51/index.jsp.
vCloud Director uses both Microsoft SQL Server and Oracle Database. In this section, we will consider SQL Server only. VMware suggests that a database server configured with 16 GB of memory, 100 GB of storage, and four CPUs should be adequate for most vCloud Director clusters.
SQL Server databases have specific configuration requirements when you use them with vCloud Director. Install and configure a database instance, and create the vCloud Director database user account before you install vCloud Director.
The vCloud Director database performance is an important factor in the overall vCloud Director performance and scalability. vCloud Director uses the SQL Server
tempdb file when storing large result sets, sorting data, and managing data that is being concurrently read and modified. This file can grow significantly when vCloud Director experiences a heavy concurrent load. It is a good practice to create the
tempdb file on a dedicated volume that has fast read/write performance. To do so, follow the given steps:
Create the master instance.
The following script creates the database and log files, specifying the proper collation sequence:
USE [master] GO CREATE DATABASE [vcloud] ON PRIMARY (NAME = N'vcloud', FILENAME = N'C:\vcloud.mdf', SIZE = 100MB, FILEGROWTH = 10%) LOG ON (NAME = N'vcdb_log', FILENAME = N'C:\vcloud.ldf', SIZE = 1MB, FILEGROWTH = 10%) COLLATE Latin1_General_CS_AS GO
The values shown for
SIZEare suggestions. You might need to use larger values.
Set the transaction isolation level.
The following script sets the database isolation level to
USE [vcloud] GO ALTER DATABASE [vcloud] SET SINGLE_USER WITH ROLLBACK IMMEDIATE; ALTER DATABASE [vcloud] SET ALLOW_SNAPSHOT_ISOLATION ON; ALTER DATABASE [vcloud] SET READ_COMMITTED_SNAPSHOT ON WITH NO_WAIT; ALTER DATABASE [vcloud] SET MULTI_USER; GO
The following script creates the database username
vcloudwith the password
USE [vcloud] GO CREATE LOGIN [vcloud] WITH PASSWORD = 'vcloudpass', DEFAULT_DATABASE =[vcloud], DEFAULT_LANGUAGE =[us_english], CHECK_POLICY=OFF GO CREATE USER [vcloud] for LOGIN [vcloud] GO
Assign permissions to the vCloud Director database user account.
The following script assigns the
db_ownerrole to the database user created in step 3:
USE [vcloud] GO sp_addrolemember [db_owner], [vcloud] GO
The vCloud Director installer verifies that the target server meets all the platform prerequisites and installs the vCloud Director software on it. The vCloud Director software is distributed as a digitally signed Linux-executable file named
nnnnnn represents a build number. You should first upload this bin file to the vCloud Director VM. Let's get started:
Log in to the target server using SSH as the root user.
Change the folder where you have uploaded the bin file with the following command:
# cd <Path>
Enable the installation file for execution as this installation file requires permission to execute:
# chmod u + x installation-file
Run the installation file:
For the question Would you like to run the script now (y/n)?, answer
n. We will first need to create the SSL certificates for vCloud Director 5.1.
At this time, we need to create the SSL/TLSv1 certificates. Cloud computing has become one of the hottest technologies today. It is being used by service providers and enterprises alike. As more and more people have been accessing cloud services via the Internet or within their corporate environments, traffic passing through the cloud has multiplied. Along with this growth and proliferation have come heightened security risks and resulting attacks to the information being shared. Security has become a paramount concern, because authenticity, confidentiality, and integrity of the information are vital and must be guaranteed.
Network security leverages numerous techniques to aid in the protection of transmitted information. Traditionally, it relies on the principles of cryptology to provide the foundation of security. This involves the conversion of information into an incomprehensible form factor that is usable only to selected recipients capable of transforming the information back into a usable form. Transport Layer Security (TLS) and its predecessor Secure Sockets Layer (SSL) are cryptographic protocols commonly used today to aid in network security.
Complex infrastructures such as cloud computing involve multiple connections between various hosts and external communication channels. The use of TLSv1/SSL certificates is an important tool to encrypt those connections to provide data privacy.
TLSv1/SSL certificates also provide for two-way authentication. This enables a host to validate that it is connected to the intended recipient. This decreases the ability of an imposter to intercept the information transmitted.
vCloud Director requires SSL to secure communications between clients and servers. Before you install and configure a vCloud Director Server group, you must create two certificates for each member of the group and import the certificates into the host keystores. This certificate installation requires that you create a Java keystore file using the keytool utility for certificate installation. The resulting keystore file will contain two SSL certificates along with the necessary certificates.
Each vCloud Director Server that you intend to use in a vCloud Director cluster requires two SSL certificates we just mentioned, one for each of its IP addresses. Self-signed certificates can provide a convenient way to configure SSL for vCloud Director in environments where trust concerns are minimal.
Each vCloud Director Server requires two SSL certificates, one for each of its IP addresses, in a Java keystore file. The vCloud Director installer places a copy of a keytool in
The console proxy and the HTTP alias use the same hierarchy of certificates. Because this one keystore file contains both certificates, you can use this single file wherever it is needed after it has been created.
Because this file contains private keys and is protected by a single password, it is strongly recommended that you do not keep copies of this file in unsecured locations. You should maintain a copy of a keystore file only where absolutely needed.
Before beginning the procedures, the following prerequisites must be fulfilled:
Obtain the IP addresses for the vCloud Director Server and the fully qualified domain name (FQDN) for each. The configured IP addresses on the vCloud Director host can be identified through the use of the
ifconfig –acommand. The FQDN for the IP addresses can be displayed using the
nslookup_<ip address>command, where
<ip address>equates to a configured IP address.
Note the FQDN names for each IP address because this name will be used for the HTTP server and console proxy service SSL certificates. Noting the IP addresses will assist in the installation of the SSL certificate.
Access the keytool utility. This utility is installed with vCloud Director by default. It is possible to use the keytool utility on another computer that has the Java Runtime Environment (JRE) version 6 or later installed, and then import the created Java keystore file onto your vCloud Director Server.
This assumes you are using the keytool installed on your vCloud Director Server as in the following example:
Create an untrusted certificate for the HTTP service:
# /opt/vmware/vcloud-director/jre/bin/keytool -keystore certificates.ks -storetype JCEKS -storepass vmware123 -genkey -keyalg RSA -alias http
Create an untrusted certificate for the proxy service console:
# /opt/vmware/vcloud-director/jre/bin/keytool -keystore certificates.ks -storetype JCEKS -storepass vmware123 -genkey -keyalg RSA -alias consoleproxy
At this time, we can go back and configure vCloud Director. To run the configuration script, we now need to run the following script:
HTTP service IP Address: Remote Console Proxy IP Address: Java Keystore path: Java Keystore password:
2for a Microsoft SQL Server database type.
The required database information is as follows:
Database host: Database Port: Database Name: Database Instance: Database Username: Database Password:
It will connect to the database through JDBC and database script will run.
Once the scripts have been completed, you will be presented with the link to the vCloud Director cell. You will also be asked to start the vCloud Director service; answer
Y to start the service, and the vCloud Director service will be started.
Once you have completed with the vCloud Director configuration, you can use the vCloud Director Web Console to complete the initial provisioning of your cloud. However, before you use the vCloud Director Web Console, you have to go through the setup wizard. The setup wizard gathers the information that the Web Console requires before it can start. Thus, once the wizard is finished, the web console starts and displays the login screen. The vCloud Director Web Console provides a set of tools for provisioning and managing a cloud environment. It includes a quickstart feature as well that guides you through steps such as attaching vCloud Director to vCenter and creating an organization.
Follow the prompts to complete the setup:
Accept the terms of the license agreement.
Enter the license key.
Enter the administrative account username, password, full name, and e-mail address.
Specify the system name and the installation ID. A vCloud Director installation ID is used to ensure the network addressing uniqueness and network traffic separation between distinct vCloud Director instances that happen to utilize the same Layer 2 network.
At this time, you will get a login prompt. Log in to this vCloud Director using the system admin credentials just created.
Click on Attach a vCenter.
You will be presented with the following screen where you have to input the vCenter Server information:
Specify the vCenter connection information and click on Next.
Specify the vCloud Networking and Security Manager server connection information and click on Next.
On the final screen, click on Finish.
Once you add the vCenter Server, you can see it under the Manage & Monitor tab.
As a prerequisite, vCenter Server has to be registered with your vCloud Networking and Security Manager. If not, you will see an error, vShield Manager is not registered with the VC <VC Name>. Perform VC registration in vShield Manager and retry. Open the vCloud Networking and Security Manager URL in a supported browser.
Log in to the cloud as the administrator. This should have been done as part of the initial configuration.
In the main Settings and Reports section, find the vCenter Server section, and you will see there is no vCenter Server registered with the vCloud Networking and Security Manager.
Click on the Edit button.
Specify the vCenter Server information and its credentials.
Click on OK.
Click on Yes on the security warning.
VMware vCloud Director has been designed to be a really secured environment right from the bottom to the top layers. However, it is up to the vCloud Director administrators how they can use security roles, and the LDAP integration to keep VMware vCloud secure. However, this was based in vCloud Director Version 1.5.
The vCloud Director security guide is available at http://www.vmware.com/files/pdf/techpaper/VMW_10Q3_WP_vCloud_Director_Security.pdf, which covers in detail how to address the security needed for specific environments.
Locally defined in vCloud Director (not desirable from a security standpoint)
Imported users from a Lightweight Directory Access Protocol (LDAP) server into vCloud Director
Locally defined users in each organization (not desirable from a security standpoint)
Imported users from an LDAP server into a specific organization
Imported users from either the VMware vSphere identity provider (IdP) or the external identity provider (IdP)
System administrators have been defined at the system level, and they carry full system-level access.
As VMware vSphere, vCloud Director also uses roles and permissions to determine what actions a user can perform in an organization. vCloud Director comes with a number of predefined roles with specific rights. System administrators and organization administrators have the ability to assign each user or group a role. It is possible to have the same user imported into different organizations from one LDAP system. That user can then be assigned different rights in each organization if desired. System administrators can also create roles and modify existing ones. Also all the roles can be modified by the system administrator. They can also create custom roles.
The main benefit of using LDAP is that you can use it to provide a directory of users and groups to import into an organization. Otherwise, you have to create a user account for each user in the organization. However, it is limited to the system administrator only, that says, an organization admin cannot modify this. A system administrator can set the LDAP in such a way that each organization will have its own LDAP configuration. They should import users and groups into the organization and assign roles before they can be used.
Another good part here is that with the release of vCloud Director 5.1, it supports importing users from VMware vCenter Single Sign-On. A Single Sign-on, also known as SSO capability, is where a user can have a single user ID and password that works throughout the system. vCloud Director provides SSO by integrating either LDAP or vCenter SSO identity. It is a system administrator's job to import users from LDAP or vCenter SSO as vCloud Director does not import users automatically.
vCloud Director does not support hierarchical domains in LDAP. Also, vCloud Director cannot modify the information in an LDAP directory.
vCloud Director does not import users' passwords from external LDAP systems. Instead, vCloud Director will confirm that a password is correct when a user logs in by checking the supplied hashed password against the hashed password currently stored in the LDAP directory.
vCloud Director has the ability to use LDAP at both the system level and the organization level. At the system level, you can either connect to an external LDAP system or create and use users who are internal to vCloud Director. You can use an external LDAP system to bring in users, but VMware recommends that you create at least one system user, which is only internal. The existence of at least one internally defined system administrator allows you to log in to your vCloud Director console even if the LDAP system is offline.
There are two ways to log in to the LDAP server. One is simple authentication and the other one is with Kerberos authentication. Simple authentication is, well, simple. However, Kerberos is a ticket-based system of client and server authentication. In Kerberos, both parties must prove their identity to each other. Kerberos uses symmetric key cryptography and can also leverage public key cryptography. If you are using Kerberos authentication, you must add a Kerberos realm to the vCloud Director Server first.
If you use simple authentication without at least combining it with SSL, the user ID (DN) and the password are sent in clear text on the network.
In order to use SSL, you must select it. You must then determine whether you will automatically accept all the certificates, or you will insist on browsing to a specific certificate. Using all certificates is much easier to configure. If your LDAP server has a certificate, it is accepted automatically. The use of SSL also provides an encrypted password exchange with the LDAP server. But the certificate from the LDAP server must be located on your system (the one the vCloud Director console is running from) and you must know the location of your SSL keystore file and have the password.
At the organization level, vCloud Director presents the following three options:
Do not use LDAP. In this case, all the users in this organization are internally defined in the vCloud Director system.
Use the vCloud Director system LDAP service. The organization leverages the LDAP service that has been configured at the system level. In order to leverage the system-defined LDAP, all the organization users must be defined in the same Organization Unit (OU) in the LDAP database.
vCloud Director system administrators are authenticated by the vSphere identity provider when you use vCenter SSO. However, as a prerequisite, vCenter SSO must be configured in vSphere. vSphere Lookup Service must be registered in the vCloud Director Administration tab under Federation. vCloud Director should also be configured with the vSphere Lookup Service URL. vCloud Director system administrator users must be imported (either as a user or a group) from the vSphere identity provider. Only vCloud Director's system administrator users can be authenticated through vCenter SSO.
One of the most important factors for the overall system security is to record and monitor the activities of the users. The organization maintains their compliance with rules by maintaining an audit log of significant activities. Using audit logs, an organization verifies and detects any violations and initiates remediation activities.
As a vCloud system administrator, you can view the system log to monitor system-level tasks that are in progress. Also, you can find and troubleshoot failed tasks as well. You can also analyze vCloud Director logs to monitor vCloud Director cells.
As a vCloud organization administrator, you can view the log for an organization to monitor organization-level tasks that are in progress. In addition, you can find and troubleshoot failed tasks.
So essentially, we are talking about system-level and organization-level tasks.
You can find the logs for a cell at
The following table shows the log names and their purposes:
What the log shows
The console output from the vCloud Director cell
Debug-level log messages from the cell
Warnings or errors encountered by the cell
When the cell crashed, restarted, and so on
Diagnostics information (but this first needs to be enabled in the local logging configuration)
HTTP request logs in the Apache common log format
Apart from the diagnostics logs in the vCloud Director, you have audit logs mentioned in the preceding table as well. However, by default, these files are not forwarded to the centralized logging server. You have to manually configure the vCloud cell to forward these to the centralized logging server.
It is recommended that you configure this option for the following reasons:
It allows audit logs from all the cells to be viewed together at a central location at the same time.
Database logs are not retained after 90 days, but logs transmitted via Syslog can be retained as long as desired.
It protects the audit logs from loss on the local system due to failure, lack of disk space, compromise, and so on.
Supports forensics operations in the face of problems as those listed previously.
Logging to a remote system, instead of the system the cell is deployed on; provides data integrity by inhibiting tampering. Even if the cell is compromised, it does not necessarily enable access to or alteration of the audit log.
For enabling a centralized Syslog server in vCloud Director 5.1, follow this knowledge base article from VMware, http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1026815.
In this chapter, we have installed VMware vCloud Director and performed a basic configuration of it. We have successfully installed and configured vCloud Director. Also, we looked into the basic security hardening aspects. We discussed basic security aspects of vCloud Director systems. This security aspect includes the user and different security roles, integrating different types of LDAP servers, and its options at various levels, which include the system level and the organization level.
In the next chapter, we will talk about VMware the vCloud Networking and Security App, and how we can integrate it to vCloud Director. We will also discuss the installation steps of the vCloud Networking and Security App, the vCloud Networking and Security App firewall management, and flow monitoring.