VMware vSphere 5.x Datacenter Design Cookbook

By Hersey Cartwright
  • Instant online access to over 7,500+ books and videos
  • Constantly updated with 100+ new titles each month
  • Breadth and depth in over 1,000+ technologies
  1. The Virtual Datacenter

About this book

VMware vSphere 5.x provides a robust and resilient platform for virtualizing server and application workloads in datacenters. The features available in VMware’s vSphere 5.x suite of products simplify the management of resources, increase the availability of applications, and guarantee the performance of workloads deployed in the virtualized datacenters. "VMware vSphere 5.x Design Cookbook" covers the design process from creating the conceptual design, to determining the resource requirements for the logical design, and then on to creating the physical design.

"VMware vSphere 5.x Datacenter Design Cookbook" begins with the process of identifying the design factors and applying them to the logical and physical design process in order to create a successful vSphere 5.x design. The key to a successful VMware vSphere 5.x design is the design’s ability to satisfy the design factors. This Cookbook walks you through the process of identifying the design factors which include the requirements the design must satisfy and the assumptions the architect makes to form the conceptual design.

From the conceptual design, the logical design is created. The logical design process includes determining the architecture of and the resources required for the management, storage, network, and compute resources. This Cookbook includes recipes for calculating the resources required and determining the architecture and features to use while applying VMware recommended practices.

The physical design takes the logical design requirements and maps them to the physical infrastructure required to support the resources. "VMware vSphere 5.x Datacenter Design Cookbook" offers recipes to determine the physical resources and configuration required to support the logical design while satisfying the design factors. Finally, the book provides recipes for creating design documentation. Documenting the design, the implementation procedures, and the verification procedures are also covered towards the end of the book.

Publication date:
January 2014


Chapter 1. The Virtual Datacenter

In this chapter, we will cover:

  • Becoming a virtual datacenter architect

  • Using a holistic approach to datacenter design

  • Passing the VMware VCAP-DCD exam



This chapter focuses on many of the basic concepts and benefits of virtualization. It provides a quick overview of VMware virtualization, introduces the virtual datacenter architect, and lays some of the groundwork necessary for creating and implementing a successful virtual datacenter design using VMware vSphere 5.x.

We will also take a look at the VMware Certified Advanced Professional-Data Center Design (VCAP-DCD) certification, including a few tips that should help you prepare for and successfully complete the certification exam.

If you are already familiar with virtualization, this chapter will provide a review of many of the benefits and technologies of virtualization.

Since the focus of this book is on design, we will not go into great detail discussing the specifics of how to configure resources in a virtual datacenter. Most of you probably already have a good understanding of VMware's virtualization architecture, so this section will provide just a basic overview of the key VMware components that are the building blocks to the virtual datacenter.

Virtualization creates a layer of abstraction between the physical hardware and the virtual machines that run on it. Virtual hardware is presented to the virtual machine granting access to the underlying physical hardware, which is scheduled by the hypervisor's kernel. The hypervisor separates the physical hardware from the virtual machine as shown in the following diagram:

The hypervisor

At the core of any virtualization platform is the hypervisor. The VMware hypervisor is named vSphere ESXi, simply referred to as ESXi. ESXi is a Type 1 or bare-metal hypervisor. This means it runs directly on the host's hardware to present virtual hardware to the virtual machines. In turn, the hypervisor schedules access to the physical hardware of the hosts.

ESXi allows multiple virtual machines with a variety of operating systems to run simultaneously, sharing the resources of the underlying physical hardware. Access to physical resources, such as memory, CPU, storage, and network, used by the virtual machines is managed by the scheduler or Virtual Machine Monitor (VMM) provided by ESXi. The resources presented to the virtual machines can be overcommitted; this means more resources than are available can be allocated to the virtual machines on the physical hardware. Advanced memory sharing and reclamation techniques, such as Transparent Page Sharing (TPS) and ballooning, along with CPU scheduling allow for overcommitment of these resources to be possible, resulting in greater virtual to physical consolidation ratios.

ESXi is a 64-bit hypervisor that must be run on a 64-bit hardware. An ESXi installation requires less than 150 MB of space for installation. It can be installed on a hard disk locally, a USB device, a Logical Unit Number (LUN) on a Storage Area Network (SAN), or deployed stateless on hosts with no storage. The small footprint of an ESXi installation provides a reduction in the management overhead associated with patching and security hardening.

With the release of vSphere 5.0, VMware retired the ESX hypervisor. ESX had a separate Linux-based service console for the management interface of the hypervisor. Management functions were provided by agents running in the service console. The service console has since been removed from ESXi and agents now run directly on ESXi's VMkernel.

To manage a standalone host running ESXi, a Direct Console User Interface (DCUI) is provided for basic configuration and troubleshooting. A shell is available that can either be accessed locally from the console or remotely using Secure Shell (SSH). The esxcli and other commands can be used in the shell to provide advanced configuration options. An ESXi host can also be accessed directly using the vSphere Client. The ESXi DCUI is shown in the following screenshot:


The DCUI can be accessed remotely using SSH by typing the command dcui in the prompt. Press Ctrl + C to exit the remote DCUI session.

Virtual machines

A virtual machine is a software computer that runs a guest operating system. Virtual machines are comprised of a set of configuration files and datafiles stored on local or remote storage. These configuration files contain information about the virtual hardware presented to the virtual machine. This virtual hardware includes the CPU, RAM, disk controllers, removable devices, and so on, and emulates the same functionality as the physical hardware. The following screenshot depicts the virtual machine files that are stored on a shared Network File System (NFS) datastore:

The files that make up a virtual machine are typically stored in a directory set aside for the particular virtual machine they represent. These files include the configuration file, virtual disk files, NVRAM file, and virtual machine logfiles.

The following table lists the common virtual machine file extensions along with a description of each:

File extension



This is a virtual machine configuration file. It contains the configurations of the virtual hardware that is presented to the virtual machine.


This is a virtual disk descriptor file. It contains a header and other information pertaining to the virtual disk.


This is a preallocated virtual disk. It contains the content or data on the disk used by the virtual machine.


This is a file that stores the state of a virtual machine's Basic Input Output System (BIOS) or Extensible Firmware Interface (EFI) configurations.


This is a virtual machine swap file. It gets created when a virtual machine is powered on. The size of this file is equal to the amount of memory allocated minus any memory reservations.


This is a virtual machine logfile.


This is a virtual machine file used with snapshots to store data about each snapshot active on a virtual machine.


This is a virtual machine snapshot data file.

Virtual machines can be deployed using a variety of methods as follows:

  • Using the New Virtual Machine Wizard in the vSphere Client

  • By getting converted from a physical machine using the VMware Converter

  • By getting imported from an Open Virtualization Format (OVF) or Open Virtualization Alliance (OVA)

  • By getting cloned from an existing virtual machine

  • By getting deployed from a virtual machine template

When a new virtual machine is created, a guest operating system can be installed on the virtual machine. VMware vSphere 5 supports more than 80 different guest operating systems. These include many versions of the Windows server and desktop operating systems, many distributions and versions of Linux and Unix operating systems, and Apple Mac OS operating systems.

Virtual appliances are preconfigured virtual machines that can be imported to the virtual environment. A virtual appliance can be comprised of a single virtual machine or a group of virtual machines with all the components required to support an application. The virtual machines in a virtual appliance are preloaded with guest operating systems and the applications they run are normally preconfigured and optimized to run in a virtual environment.

Since virtual machines are just a collection of files on a disk, they become portable. Virtual machines can be moved from one location to another by simply moving or copying the associated files. Using VMware vSphere features such as vMotion, Enhanced vMotion, or Storage vMotion, virtual machines can be migrated from host to host or datastore to datastore while a virtual machine is running. Virtual machines can also be exported to an OVF or OVA to be imported into another VMware vSphere environment.

Virtual infrastructure management

VMware vCenter Server provides a centralized management interface to manage and configure groups of ESXi hosts in the virtualized datacenter. The vCenter Server is required to configure and control many advanced features, such as the Distributed Resource Scheduler (DRS), Storage DRS, and VMware High Availability (HA). The vCenter Server is accessed using either the vSphere Client or the vSphere Web Client. Many vendors provide plugins that can be installed to allow third-party storage, network, and compute resources to be managed using the vSphere Client.

The vCenter Server can be installed on a 64-bit Windows server. It can be run on dedicated physical hardware or as a virtual machine. When the vCenter Server is deployed on the Windows server, it requires either a Microsoft SQL or an Oracle database to store configuration and performance information. IBM DB2 databases are supported with vSphere 5.1, but this support was removed in vSphere 5.5.

SQL Express can also be used for small environments of less than five hosts and 50 virtual machines. The database can either be installed locally on the same server as the vCenter Server or on a remote database server.

Another option for deploying the vCenter Server is the vCenter Server Appliance (VCSA). The VCSA is a preconfigured, Linux-based virtual machine preinstalled with the vCenter Server components. The appliance includes an embedded database suitable for small deployments or a remote Oracle database.

Several other management and automation tools are available to aid the day-to-day administration of a vSphere environment: the vSphere Command-Line Interface (vCLI); vSphere PowerCLI provides a Windows PowerShell interface; vCenter Orchestrator can be used to automate tasks; and the vSphere Management Assistant (vMA) is a Linux-based virtual appliance that is used to run management and automation scripts against hosts. These tools allow an administrator to use command-line utilities to manage hosts from remote workstations.

VMware provides a suite of other products that benefit the virtualized datacenter. These datacenter products, such as vCenter Operations Manager, Site Recovery Manager, and vCloud Director, can each be leveraged in the virtual datacenter to meet specific requirements related to management, disaster recovery, and cloud services. At the core of these products is vSphere suite, which includes ESXi and the vCenter Server.

Understanding the benefits of virtualization

The following table provides a matrix of some of the core VMware technologies and the benefits that can be realized by using them. This is not meant to be an exhaustive list of all VMware technologies and features, but it does provide an insight into many of the technologies commonly deployed in the enterprise virtual datacenter.

VMware technology

Primary benefits


vSphere ESXi

  • Server consolidation

  • Resource efficiency

ESXi is VMware's bare-metal hypervisor that hosts virtual machines, also known as guests, and schedules virtual hardware access to physical resources.


  • Increased availability

HA restarts virtual machines in the event of a host failure. It also monitors and restarts the virtual machines in the event of a guest operating system failure.

vMotion and the VMware DRS

  • Resource efficiency

  • Increased availability

vMotion allows virtual machines to be live-migrated between hosts in a virtual datacenter. DRS determines the initial placement of the virtual machine on the host resources within a cluster and makes recommendations, or automatically migrates the virtual machines to balance resources across all hosts in a cluster.

Resource pools

  • Resource efficiency

These are used to guarantee, reserve, or limit the virtual machine's CPU, memory, and disk resources.

VMware Fault Tolerance (FT)

  • Increased availability

FT provides 100 percent uptime for a virtual machine in the event of a host hardware failure. It creates a secondary virtual machine that mirrors all the operations of the primary. In the event of a hardware failure, the secondary virtual machine becomes the primary and a new secondary is created.

Thin provisioning

  • Resource efficiency

This allows for storage to be overprovisioned by presenting the configured space to a virtual machine but only consuming the space on the disk that the guest actually requires.

Hot add CPU and memory

  • Resource efficiency

This allows for the addition of CPU and memory resources to a virtual machine while the virtual machine is running.

Storage vMotion

  • Portability

This moves virtual machine configuration files and disks between storage locations that have been presented to a host.

vSphere Data Protection (VDP)

  • Disaster recovery

This provides agentless image-level backup and recovery of virtual machines.

vCenter Server

  • Simplified management

This provides a single management interface to configure and monitor the resources available to virtual datacenters.

vCenter Server Linked Mode

  • Simplified management

This links multiple vCenter Servers together to allow them to be managed from a single client.

Host Profiles

  • Simplified management

This maintains consistent configuration and configuration compliance across all the hosts in the environment.

There are many others, and each technology or feature may also have its own set of requirements that must be met in order to be implemented. The purpose here is to show how features or technologies can be mapped to benefits, that can then be mapped to requirements and ultimately mapped into a design. This is helpful in ensuring that the benefits and technologies that virtualization provides, satisfy the design requirements.

Identifying when not to virtualize

Not all applications or server workloads are good candidates for virtualization. It is important that these workloads are identified early on in the design process.

There are a number of reasons a server or application may not be suitable for virtualization. Some of these include the following:

  • Vendor support

  • Licensing issues

  • Specialized hardware dependencies

  • High resource demand

  • Lack of knowledge or skill sets

A common reason to not virtualize an application or workload is the reluctance of a vendor to support their application in a virtual environment. As virtualization has become more common in the enterprise datacenter, this has become uncommon; but, there are still application vendors that will not support their products once virtualized.

Software and operating systems licensing in a virtual environment can also be a challenge, especially when it comes to physical server to virtual machine conversions. Many physical servers are purchased with Original Equipment Manufacturer (OEM) licenses, and these licenses, in most cases, cannot be transferred to a virtual environment. Also, many licenses are tied to hardware-specific information, such as interface MAC addresses or drive signatures. Licensing issues can usually be overcome. Many times the primary risk becomes the cost to upgrade or acquire new licensing. As with other potential design risks, it is important that any issues and potential impacts licensing may have on the design be identified early on in the design process.

Some applications may require the use of specialized hardware. Fax boards, serial ports, and security dongles are common examples. There are ways to provide solutions for many of these, but often with the risks associated with the ability to support the application or with the loss of one or more of the potential benefits of virtualizing the application, the better solution may be to leave the application on dedicated physical hardware. Again, it is important that these types of applications be identified very early on in the design process.

Physical servers configured with a large amount of CPU and memory resources where applications are consuming a large amount of these resources may not be good candidates for virtualization. This also holds true for applications with high network utilization and large storage I/O requirements. vSphere 5.1 supports virtual machines configured with up to 64 Virtual CPUs (vCPUs) and 1 TB of RAM, but the high utilization of these configured resources can have a negative impact on other workloads in the virtual environment. These high-utilization workloads will also require more resources to be reserved for failover. The benefits of virtualizing resource-intensive applications must be weighed against the impact placed on the environment. In many cases, it may be better to leave these applications on dedicated physical hardware.

Many administrators may lack knowledge of the benefits or skills to manage a virtualized datacenter. The administrator of a virtual environment must be well-versed with storage, networking, and virtualization in order to successfully configure, maintain, and monitor a virtual environment. Though this may not necessarily be a reason not to leverage the benefits of a virtualized environment, it can be a substantial risk to the acceptance of a design and the implementation. This is especially true with smaller IT departments where the roles of the server, application, storage, and network administrators are combined.

Each of these can introduce risks in the design. We will discuss how risk impacts the design process in much more detail in Chapter 2, The Discovery Process, and Chapter 3, The Design Factors.


Becoming a virtual datacenter architect

The virtual datacenter architect or architect is someone who identifies requirements, designs a virtualization solution to meet those requirements, and then oversees the implementation of the solution. Sounds easy enough, right?

How to do it…

The primary role of the architect is to provide solutions that meet customer requirements. At times, this can be difficult since the architect may not always be part of the complete sales process. Many times customers may purchase hardware from other vendors and look to us to help them "make it all work". In situations such as this, the purchased hardware becomes a constraint on the design. Identifying and dealing with constraints and other design factors will be discussed in more detail in Chapter 2, The Discovery Process, and Chapter 3, The Design Factors.

The architect must also be able to identify requirements, both business and technical, by conducting stakeholder interviews and analyzing current configurations. Once the requirements have been identified, the architect must then map the requirements into a solution by creating a design. This design is then presented to the stakeholders, and if approved, it is implemented. During the implementation phase, the architect ensures that configurations are done to meet the design requirements and the work done stays within the scope of the design.

The architect must also understand best practice. Not just best practice for configuring the hypervisor, but for management, storage, security, and networking. Understanding best practice is the key. The architect not only knows best practice but understands why it is considered best practice. It is also important to understand when to deviate from what is considered best practice.

There's more…

The large part of an architect's work is "customer facing". This includes conducting interviews with stakeholders to identify requirements and ultimately presenting the design to decision makers. Besides creating a solid solution to match the customer's requirements, it is important that the architect gains and maintains the trust of the project stakeholders. A professional appearance and, more importantly, a professional attitude are both helpful in building this relationship.


Using a holistic approach to datacenter design

The virtual datacenter architect must be able to take a holistic approach to datacenter design. This means that for every decision made, the architect must understand how the environment as a whole will be impacted.

An architect is required to be, at the very least, familiar with all aspects of the datacenter. They must understand how the different components of a datacenter, such as storage, networking, computing, security, and management, are interconnected, as shown in the following diagram:

It has become very important to understand how any decision or change will impact the rest of the design. Identifying dependencies becomes an important part of the design process. If a change is made to the network, how are computing, management, and storage resources affected? What other dependencies will this introduce in the design? Failing to take a holistic approach to design can result in unnecessary complications during the design process and potentially costly fixes after the design is implemented.

How to do it...

You have been engaged to design a virtualization solution for a financial organization. The solution you are proposing is using 10 GB Converged Network Adapters (CNA) to provide connectivity to the organization's network in three 1U rackmount servers. The organization needs to separate a Virtual Local Area Network (VLAN) that is currently configured to be delivered over the CNA onto a physically separate network to satisfy a new compliance requirement. A 1 GB network will provide sufficient bandwidth for this network, and the network should be highly available. Single points of failure should be minimized.

To support this compliance requirement, you, the architect, must take a holistic approach to the design by answering a number of questions about each design decision, as follows:

  • Are there available network ports in the current rackmount servers or will a network card need to be added? If a card must be added, are there available Peripheral Component Interconnect (PCI) slots?

  • Will a dual port network card provide sufficient redundancy, or will the network need to be separated across physical cards? Are there onboard network ports available that can be used with a PCI network card to provide in-box redundancy?

  • Has the physical separate switch's hardware been obtained? If not, how long before the equipment is received and deployed? Will this have an impact on the implementation schedule?

  • How will the virtual switch need to be configured to provide the connectivity and redundancy that is required?

How it works...

The impact can be fairly significant, depending on some of the answers. For example, let's say the 1U rackmount server will not support the required network adapters needed to satisfy the requirement and a different 2U rackmount server must be used. This then raises more questions, such as is there sufficient space in the rackmount to support the new server footprint.

What if the requirement had been that the applications connected to this network be virtualized on separate physical server hardware and storage? What parts of the design would have to change? The architect must be able to understand the dependencies of each part of the design and how a change in one place may affect other areas of the design.

As you think through these questions, you should be able to see how a change to a requirement can have a deep impact on many other areas of the design. It becomes very important to identify requirements early on in the design process.


Passing the VMware VCAP-DCD exam

For vSphere 4 and vSphere 5, VMware released advanced exams testing the ability of a person to administer and design complex virtual environments. The exams for vSphere 5 are the VMware Certified Advanced Professional 5-Data Center Administration (VCAP5-DCA) exam, which focuses on administering a VMware vSphere environment, and the VMware Certified Advanced Professional 5-Data Center Design (VCAP5-DCD) exam, which focuses on designing a VMware vSphere environment.

The VCAP5-DCD exam tests your ability to design enterprise virtualized environments. To be successful, you must have an in-depth understanding of VMware's core components and the relationship they share with other components of the datacenter, such as storage, networking, and application services, along with a mastery of VMware's datacenter design methodologies and principles. All the exam objectives, including study resources, can be found in the exam blueprint. The exam blueprint can be found on the VCAP5-DCD page at http://www.vmware.com/go/vcapdcd.

Getting ready

Before you are eligible to take the VCAP5-DCD exam, you should have obtained the VMware Certified Professional 5–Data Center Virtualization (VCP5-DCV) certification. Besides the training required for the VCP5-DCV certification, there is no other required training that must be completed in order to sit the VCAP5-DCD exam. When you are ready to schedule your VCAP5-DCD exam, you must submit an exam authorization request to VMware. When you submit the exam authorization request, VMware will verify that you have met the certification prerequisites and provide you with the access necessary to schedule the exam. It typically takes less than two business days to receive the exam authorization.

The exam consists of 100 questions and is scored using a scaled scoring system. You have 4 hours to complete the exam, and the exam has a scoring scale of 100-500 with a minimum passing score of 300. The questions are comprised of a mixture of multiple-choice, drag-and-drop, and design scenarios. Once you have submitted an answer for a question, you are not allowed to return to that question. It is important to note that there is also no way to flag an item for later review.

How to do it…

The VCAP-DCD exam is one of the most challenging exams I have ever taken. Here are a few tips to help you prepare for and successfully sit the VCAP-DCD exam:

  • Study the material on the exam blueprint: The exam blueprint lists all the objectives of the exam, along with links to documentation related to each exam objective.

  • Schedule your exam: Scheduling your exam sets a goal date for you to work towards. Setting the date can provide motivation to help you stay on track with your studying efforts.

  • Watch the APAC vBrownBag DCD5 Series: The APAC vBrownBag did a series of podcasts focusing on the VCAP-DCD exam for vSphere 5 exam objectives. These podcasts are very helpful and provide a valuable study resource. The podcast can be found at the following link:


  • Get familiar with the exam design interface: On VMware's VCAP Certification page for the DCD exam, there is a UI Demo that will help get you familiar with the design interface that is used on the exam.

  • Practice time management: It is very important that you are aware of the amount of time you are taking on a question, and how much time remains. If you get hung up on a multiple-choice question, take your best guess and move on. Conserve time for the more complex, drag-and-drop and design scenario questions.

  • Answer every question: A question left unanswered will be marked incorrect and will not benefit your score in any way. A guess has some chance of being correct.

  • Study the material on the exam blueprint: I know this has already been mentioned once, but it is worth mentioning again. The exam blueprint contains all the testable objectives. Study it!

There's more…

For up-to-date information on the VCAP-DCD certification, to download the exam blueprint, to view the Exam UI Demo, and to book your VCAP-DCD exam, visit the VMware VCAP Certification page at http://www.vmware.com/go/vcap.

About the Author

  • Hersey Cartwright

    Hersey Cartwright has worked in the technology industry since 1996 in many roles, from help desk support to IT management. He first started working with VMware technologies in 2006. He is currently a Solutions Engineer for VMware, where he designs, sells, and supports VMware software-defined datacenter products in enterprise environments within the healthcare industry. He has experience working with a wide variety of server, storage, and network platforms.

    Browse publications by this author