Implementing Cisco UCS Solutions

1 (1 reviews total)
By Farhan Nadeem , Prasenjit Sarkar
  • Instant online access to over 7,500+ books and videos
  • Constantly updated with 100+ new titles each month
  • Breadth and depth in over 1,000+ technologies
  1. Cisco UCS Physical Architecture and Installing UCS Hardware

About this book

Cisco Unified Computing System(UCS) provides unique features for the contemporary data centres. Cisco UCS is a unified solution that consolidates computing, network and storage connectivity components along-with centralized management. Cisco UCS reduces TCO and improves scalability and flexibility. Stateless computing blade server's design simplifies the troubleshooting, and Cisco-patented extended memory technology provides higher virtualized servers consolidation results.

A hands-on guide to take you through deployment in Cisco UCS. With real-world examples for configuring and deploying Cisco UCS components, this book will prepare you for the practical deployments of Cisco UCS data centre solutions.

If you want to learn and enhance your hands-on skills with Cisco UCS solutions, this book is certainly for you.

Starting with the description of Cisco UCS equipment options, this hands-on guide then introduces Cisco UCS Emulator which is an excellent resource to practically learn Cisco UCS components’ deployment. You will also be introduced to all areas of UCS solutions with practical configuration examples.

You will also discover the Cisco UCS Manager, which is the centralized management interface for Cisco UCS. Once you get to know UCS Manager, the book dives deeper into configuring LAN, SAN, identity pools, resource pools, and service profiles for the servers. The book also presents other administration topics including Backup, Restore, user’s roles, and high availability cluster configuration. Finally, you will learn about virtualized networking, 3rd party integration tools and testing failure scenarios.

You will learn everything you need to know for the rapidly growing Cisco UCS deployments in the real-world.

Publication date:
December 2013
Publisher
Packt
Pages
370
ISBN
9781782170662

 

Chapter 1. Cisco UCS Physical Architecture and Installing UCS Hardware

In previous decades, computers evolved at a dramatic pace. Moore's law, which predicted that the density of transistors and integrated circuits would double every two years as computing components kept on shrinking in size while improving in computational capacity, has truly prevailed. This technological evolution led to three distinct generations of computing devices.

We witnessed the era of the following generations:

  • Gigantic mainframe computers

  • Commoditized personal computers and tower and rack servers (also known as pizza-box servers)

  • Blade servers (also known as modular servers)

Mainframes were monolithic systems often with proprietary hardware and software. With their enormous computing power, mainframes were able to run multiple applications; however, their major limitations were cost and many single points of failure. Due to their high cost and management complexity, mainframes remained mainly confined to military use, universities, and some very large organizations.

Tower and rack-mounted servers usually have limited computational resources as compared to mainframes; however, these are very cost effective. Because of the limited resources available on these servers, a one-to-one server-to-application ratio is usually the way to go. Because of this one server one application design, rack and tower servers need more rack space, separate cabling, individual power supplies, and more cooling in the datacenter, which makes management of the infrastructure quite complex. However, this second generation of computers is generally very cost effective. This led to the mass adoption of computers everywhere.

The latest trend in the ever evolving computer architecture space is to move away from tower and rack-mounted servers in favor of blade servers. In today's highly demanding enterprise applications, blade server architecture provides excellent features when compared with rack and tower servers. These features include the following:

  • Less rack space usage

  • Less cabling

  • Shared power

  • Consolidated I/O

  • Easy management

  • Excellent heating and cooling

Contemporary datacenters are facing unprecedented growth in computational demands alongside the need for reducing implementation and operational costs. Considering these factors, blade servers are designed to minimize the use of resources and space. Components in a blade chassis are either removed or shared between blade servers.

The minimum form factor of a rack server is 1 Rack Unit (RU). 1 RU is equal to 1.75 inches, and the most common server rack height is usually 42 RU. It is therefore possible to fit only 42 pizza-box servers in a standard rack. With blade servers, it is possible to achieve higher densities of servers per rack.

In a blade server, data connectivity interfaces and power supplies are also shared. Thus, blade servers also require less cabling, and hence less management.

In this chapter, we will discuss physical components of the Unified Computing System (UCS) equipment. The list of the topics covered in this chapter is as follows:

  • A quick look at the UCS equipment

  • Understanding the physical architecture of UCS

  • Understanding Fabric Interconnects (FIs)

  • Cisco UCS 5100 series blade server chassis

  • IOM modules

  • Blade servers and rack-mount servers

  • Getting started with mezzanine cards

  • Power capacity and power plugs

  • Installing UCS chassis components

  • Cabling FI and IOM

 

Looking at the UCS equipment


With the ever increasing demand on datacenters, vendors started focusing on different aspects of server and networking hardware consolidation; however, most of the ad hoc solutions were based on gluing together the existing products which were not designed grounds up to provide a cohesive infrastructure and failed to address the requirements of the datacenter as a whole. Hence, management of these amalgamated solutions was a nightmare for IT administrators.

Cisco entered into the blade server market with a holistic approach to the blade server design. With a strong background in networking and storage products, Cisco developed a cohesive solution consolidating the computing, network, and storage connectivity components along with centralized management of these resources. The purpose of Cisco UCS is to reduce the Total Cost of Ownership (TCO) and improve scalability and flexibility.

Salient features and benefits of the UCS solution include the following:

  • Stateless computing

  • Rapid provisioning of servers

  • Simplified troubleshooting

  • Virtualization readiness

  • Choice of industry-standard form factors

  • Extended memory technology for increased density

Stateless computing

Cisco introduced the idea of stateless computing with its blade server design. Cisco blade servers do not have any initial configuration. Universally Unique Identifiers (UUIDs) for blades, Network Interface Cards (NICs), Media Access Control (MAC) addresses, storage World Wide Node (WWN) numbers, firmware, and BIOS settings are all abstracted from Unified Computing System Manager (UCSM), the management software running on the FIs.

Rapid provisioning of servers

Provisioning of servers dramatically improves as the servers can be provisioned using the UCSM software even before they are physically available. Once the server is physically installed, it will abstract its identity from UCSM. Using server configuration templates, it is therefore possible to create a server template only once and apply the template to hundreds of servers.

Simplified troubleshooting

Replacement of servers also becomes very easy. Since the servers are stateless, as soon as a replacement server is installed, it will abstract all the configuration of the old server and will be available for use. Servers can also be easily migrated for different roles and workloads.

Virtualization readiness

Virtualization in the form of modern bare metal hypervisors is a major breakthrough for optimal utilization of computational resources. Cisco UCS solution supports all major hypervisor platforms including VMware ESX/ESXi, Microsoft Hyper-V, and Citrix Xen server. Support and integration with VMware vSphere solution is very strong. UCSM can be integrated with vCenter to abstract and manage features at the individual Virtual Machine (VM) level. By leveraging the benefits of virtualization and increasing the density of the physical server, the UCS solution can scale up to thousands of VMs.

Choice of industry-standard form factors

Cisco UCS servers are available in two categories: B-series blade servers and C-series rack-mount servers. Both form factors are designed using the same industry-standard components and can address different computational requirements. Both B-series blade servers and C-series rack-mount servers are designed using Intel Xeon CPUs. B-series servers are managed through UCSM, whereas C-series servers can either be individually managed or can be integrated to UCSM.

Extended memory technology for increased density

Cisco also introduced a patented extended memory technology for two CPU socket servers to increase the total amount of memory support; this could be more than double the amount of memory as compared to the industry standards for two socket servers. Virtualized workloads can leverage this extra memory to support an even greater density of VMs in a reduced physical footprint, resulting in reduced Capital Expenditure (CAPEX) and Operational Expenditure (OPEX) costs. Extended memory technology is available in both B-series blade servers and C-series rack-mount servers.

 

Understanding the physical architecture of UCS


The Cisco UCS solution can be divided into the following three main categories:

  • The Cisco UCS FIs

  • The Cisco UCS blade servers

  • The Cisco UCS rack-mount servers

The Cisco UCS FIs

The Cisco UCS FIs provide network connectivity and management for the connected servers. The UCS FIs run the UCSM control software and consist of the following components:

  • 6100 series and 6200 series FIs

  • Transceivers for network and storage connectivity

  • Expansion modules for both FI series

  • UCSM software

The Cisco UCS blade servers

The Cisco UCS blade servers require a mother chassis where these servers can be installed. The UCS blade server solution consists of the following components:

  • A blade server chassis, used for installing blades

  • B-series blade servers

  • IOM modules for connectivity to the FIs

  • Transceivers for connectivity to the FIs

  • Mezzanine cards for network and storage connectivity

The Cisco UCS rack-mount servers

The Cisco UCS rack-mount servers are standalone servers that can be installed and controlled individually. Cisco provides Fabric Extenders (FEXs) for the rack-mount servers. FEXs can be used to connect and manage rack-mount servers from FIs. The UCS rack-mount server solution consists of the following components:

  • UCS C-series rack-mount servers

  • FEXs for connecting rack-mount servers to FIs

In this chapter we will provide details about hardware options available for both blade servers and rack-mount servers. Most of the field deployments of UCS servers are based on blade servers. Therefore, our main focus will be on blade server configuration in this book. However, if proper connectivity is established between rack-mount servers and FIs, rack-mount servers can also be managed in the same fashion.

The following figure depicts the FIs running the UCSM software, a blade server chassis with IOM modules, and the main components of a blade server:

In this chapter, we will go into the details of various UCS components and will focus on their physical specifications and installation in the subsequent sections.

Note

The Cisco interactive 3D model for Cisco 5100 series chassis and blades is an excellent resource for exploring Cisco UCS components physically. It is also available for iPhone/iPad (search for UCS Tech Specs in the marketplace). More details on this are available at http://www.cisco.com/en/US/prod/ps10265/ps10279/ucs_kaon_model_preso.html.

 

Understanding FIs


An FI is the core component of a UCS solution. FIs are typically configured as highly available clustered pairs in production environments. It's possible to run a single FI-based design as a proof of concept test deployment before actually implementing it in production. FIs provide the following two capabilities:

  • Network connectivity to both LAN and SAN

  • UCS infrastructure management through the embedded management software, UCSM, for both hardware and software management

FIs are available in two generations, namely Cisco UCS 6100 series and Cisco UCS 6200 series. The core functionality is the same in both generations; however, UCS 6200 series has a newer generation Application Specific Integrated Circuit (ASIC), higher throughput, and increased number of physical ports. Both generations can be upgraded to the latest UCSM software.

FIs provide converged ports. Depending on the physical Small Form Factor Pluggable (SFP) transceivers and FI software configuration, each port can be configured in different ways. Cisco 6200 series FI ports can be configured as Ethernet ports, Fiber Channel over Ethernet (FCoE) ports, or Fiber Channel (FC) ports. On the other hand, 6100 series converged ports only support Ethernet and FCoE (they also support FC, but only in the expansion slot).

In production, FIs are deployed in clustered pairs to provide high availability. Cisco-supported implementation requires that clustered FIs be identical. The only possibility for having different FIs in a cluster is during a cluster upgrade.

Tip

Larger enterprises may consider deploying the Cisco UCS central software, which can manage multiple UCS domains across globally distributed datacenters.

The following are the specifications of all the available UCS FIs.

The Cisco 6296UP FI

The Cisco 6296UP FI (UP represents Unified Ports) is a 2 RU device with a maximum of 96 converged ports. Ports can be configured as 1 GB Ethernet, 10 GB Ethernet, 10 GB FCoE, and 2/4/8 GB FC. The specifications of this FI are as follows:

  • A maximum of 20 blade server chassis per FI

  • A fabric throughput of 1920 Gbps

  • A 48-port base unit with three expansion slots (each expansion slot can add 16 ports)

The Cisco 6248UP FI

The Cisco 6248UP FI is a 1 RU device with a maximum of 48 converged ports. Ports could be configured as 1 GB Ethernet, 10 GB Ethernet, 10 GB FCoE, and 2, 4, or 8 GB FC. The specifications of this FI are as follows:

  • A maximum of 20 blade server chassis per FI

  • A fabric throughput of 960 Gbps

  • A 32-port base unit with one expansion slot that can provide 16 extra ports

The Cisco 6140UP FI

The Cisco 6140UP FI is a 2 RU device with a maximum of 48 converged ports. Fixed ports can be configured as 10 GB Ethernet and 10 GB FCoE. Only the first 16 ports can be configured as 1 GB Ethernet. The FC is only supported in the expansion module ports. The specifications of this FI are as follows:

  • A maximum of 20 blade server chassis per FI

  • A fabric throughput of 1040 Gbps

  • A unit of 40 fixed ports with two expansion slots where each expansion slot can provide eight FC ports of 2, 4, or 8 Gbps or six 10 Gbps SFP+ ports

The Cisco 6120UP FI

The Cisco 6120UP FI is a 1 RU device with a maximum of 20 fixed 10 GB ports. Fixed ports can be configured as 10 GB Ethernet and 10 GB FCoE. Only the first eight ports can be configured as 1 GB Ethernet. The FC is only supported in the expansion module ports. The specifications of this FI are as follows:

  • A maximum of 20 blade server chassis per FI

  • A fabric throughput of 520 Gbps

  • A unit of 20 fixed ports with one expansion slot, which can provide eight 2/4/8 Gbps FC or six 10 Gbps ports

Note

Standard pricing of FIs provides a limited number of port licenses. To enable more ports, extra licenses can be purchased per port.

Exploring connectivity transceivers for FIs

A variety of SFP transceivers are available for both FI series. These transceivers provide south-bound IOM connectivity and north-bound network and storage connectivity. They are based on industry-standard SFP+ specifications.

Transceivers can be selected depending on the technology, for example, Ethernet or FC, and also according to the distance requirements. For shorter distances between FIs, IOMs, and north-bound network switches, twinax cables with integrated SFP is an economical alternative as compared to fiber optic SFP.

The most commonly used transceivers include the following:

  • Cisco SFP-10G-SR: This is a multimode optical fiber 10 Gbps Ethernet SFP that can be used for distances up to 400 meters.

  • Cisco SFP-10G-LR: This is a single-mode optical fiber 10 Gbps Ethernet SFP that can be used for distances up to 10 Km.

  • Cisco SFP-10G-FET: This is a low power consuming multimode fiber optic 10 Gbps Ethernet SFP that can be used for distances up to 100 meters.

  • Cisco SFP-H10GB-CUxM: These are the twinax cables providing low cost 10 Gbps Ethernet connectivity and are available in 1, 3, 5, and 7 meter fixed length configurations. The actual transceivers are named SFP-H10GB-CU1M, SFP-H10GB-CU3M, SFP-H10GB-CU5M, and SFP-H10GB-CU7M according to the length each twinax cable provides.

  • Cisco SFP-H10GB-ACU10M: This is a 10-meter-long twinax cable providing 10 Gbps Ethernet. At a length of 10 meters, this cable requires active transceivers at both ends.

  • DS-SFP-FCxG-xW: These are multi-mode and single-mode fiber optic FC transceivers that are available at 2, 4, and 8 Gbps transfer speeds. The actual transceivers are named DS-SFP-FC4G-SW and DS-SFP-FC8G-SW according to the speed and distance covered by the transceiver.

Note

A detailed list of FI-compatible SFPs is available at http://www.cisco.com/en/US/prod/collateral/ps10265/ps11544/data_sheet_c78-675245.html.

Distance and other detailed specifications of the Cisco SFPs are available at http://www.cisco.com/en/US/prod/collateral/modules/ps5455/data_sheet_c78-455693.html.

 

The Cisco UCS 5100 series blade server chassis


The Cisco 5100 series blade server chassis is a vital building block of the Cisco UCS solution. Currently, there is only one generation of UCS blade chassis, which is Cisco UCS 5108. The chassis form factor is 6 RU and it can host the following:

  • A maximum of eight half-width blade servers

  • A maximum of four full-width blade servers

  • Any other combination of half-width blade and full-width blade servers is also possible

A look at the chassis front

The UCS chassis front is used to insert blade servers into the chassis. The front of the chassis also holds UCS power supplies. The UCS chassis front can hold the following hardware:

  • Eight half-width empty slots for a maximum of eight half-width blade servers with a removable divider in the middle; this can be removed for installing a maximum of four full-width blades or any other combination of half-width and full-width servers

  • Four slots for single power supplies; these slots can be configured as nonredundant, N+1 redundant, and grid redundant

This has been demonstrated in the following image:

A look at the chassis back

The UCS chassis back provides slots for IOM modules, fan units, and power connectors. It provides the following connectors:

  • Two slots for IOM modules that act as remote line cards to the FIs

  • Eight fan units

  • Four power supply connectors

This has been demonstrated in the following image:

Environmental requirements

The UCS chassis is designed for the industry-standard rack form factor. The following are the environmental requirements for the UCS chassis:

  • An industry-standard, four-post, 19 inch rack or cabinet is required as the chassis cannot be mounted onto a two-post relay rack because of its weight and length

  • It requires a 6 RU rack space

  • Ensure proper cooling and ventilation for the rack as chassis air flow is front to back and should not be obstructed

  • The operating temperature of the UCS is 10 to 35 degrees centigrade

  • The acoustic power level is 85 dBA under normal operation

  • The weight of an empty chassis is 136 kg, which requires special consideration for floor loading

Note

Due to the weight and size of the chassis, at least two persons are required to mount it to the rails. It is highly recommended to mount the chassis first and then insert the power supplies, fan units, and blade servers.

The use of a server lift is highly recommended. Chassis side handles are only for moving and adjusting the chassis in the rack.

 

IOM modules


IOM modules are also known as Cisco FEXs or simply FEX modules. These modules serve as line cards to the FIs in the same way that Nexus series switches can have remote line cards. IOM modules also provide interface connections to blade servers. They multiplex data from blade servers and provide this data to FIs and do the same in the reverse direction as well. In production environments, IOM modules are always used in pairs to provide redundancy and failover.

Apart from data transfers between chassis and FIs, IOM modules provide the following two features:

  • Chassis Management Controller (CMC): This monitors chassis components such as fan units, power supplies, and chassis temperature. This also reads chassis component identification data and detects the insertion and removal of blade servers.

  • Chassis Management Switch (CMS): This provides fast Ethernet links to the embedded Cisco Integrated Management Controller (CIMC) on blade servers for Keyboard Video Mouse (KVM) access and S erial over LAN (SoL) access and for Intelligent Platform Management Interface (IPMI) data to travel from individual blades to FIs for monitoring and management purposes.

The following figure depicts components and connectivity inside an IOM module:

The Cisco 2208XP IOM card

The specifications of the Cisco 2208XP IOM card are as follows:

  • Eight 10 Gbps fabric ports for connecting to the FI

  • 32 10 Gbps, server-facing ports with FCoE

  • 80 Gbps throughput

The Cisco 2204XP IOM card

The specifications of the Cisco 2204XP IOM card are as follows:

  • Four 10 Gbps fabric ports for connecting to the FI

  • 16 10 Gbps, server-facing ports with FCoE

  • 40 Gbps throughput

The Cisco 2104XP IOM card

The specifications of the Cisco 2104XP IOM card are as follows:

  • Four 10 Gbps fabric ports for connecting to the FI

  • Eight 10 Gbps, server-facing ports with FCoE

  • 40 Gbps throughput

 

Blade servers and rack-mount servers


While blade servers are the dominant deployment of the UCS solution, Cisco has strategically extended its offering to include the industry-standard rack-mount form factor as well in order to provide users a choice against the competing rack-mount server vendors such as HP, IBM, and Dell.

Learning more about blade servers

Blade servers are at the heart of the UCS solution and come in various system resource configurations in terms of CPU, memory, and hard disk capacity. All blade servers are based on Intel Xeon processors and there is no AMD option available. Small and Medium Businesses (SMBs) can choose from different blade configurations as per business needs.

The B22 M3 blade server

The B22 M3 blade server is a new addition to the blade server series. It is a half-width blade server and has been shown in the following image:

The specifications of the B22 M3 blade server are as follows:

  • Two Intel® Xeon® E5-2400 series processor family CPUs

  • A maximum of 192 GB as the total memory capacity with a total of 12 memory slots

  • Two drive bays for internal SAS/SATA/SSD hard drives for up to 2 TB of maximum storage capacity with built-in RAID 0,1 controllers

  • Up to 80 Gbps of I/O throughput with a supported mezzanine card

The B22 series entry level blade server provides an excellent price-to-productivity balance. It is an excellent choice for IT, web infrastructure, and scale-out applications.

B200 M1/M2/M3 blade servers

B200 M1/M2/M3 blade servers are currently in the third generation. The B200 series is a half-width blade server. The latest B200 M3 series blade server further extends the capabilities of the B200 M2 servers. There are slight differences in the physical construction of M1 and M2/M3. Have a look at the following image for more details:

The specifications of the latest B200 M3 blade server are as follows:

  • Two Intel® Xeon® E5-2600 series processor family CPUs (a single processor is no longer supported with B200 M3)

  • A maximum of 768 GB in total memory capacity (using 32 GB Dual Inline Memory Modules (DIMMs)) with a total of 24 memory slots

  • Two drive bays for internal SAS/SATA/SSD hard drives for up to 2 TB of maximum storage capacity with built-in RAID 0,1 controllers

  • Up to 80 Gbps of I/O throughput with a supported mezzanine card

B200 series entry-level blade servers are an excellent choice for web server farms, distributed databases, and CRM applications.

B230 M1/M2 blade servers

B230 M1/M2 series servers differ from B200 series servers in terms of computational capabilities. This is also a half-width blade server with two CPUs; however, the B230 server is based on a slightly different CPU family. The B230 series memory slot density is higher as compared to that of the B200 series blade servers. Have a look at the following image for more details:

The specifications of the B230 M2 blade server are as follows:

  • Two Intel® Xeon® processors of the E7-2800/8800 product family CPUs

  • A maximum of 512 GB as the total memory capacity with a total of 32 memory slots

  • Two SSD drive bays with a maximum of 600 GB of storage and built-in RAID 0,1 controllers

  • Up to 20 Gbps of I/O throughput with a supported mezzanine card

B230 series servers are an excellent choice for virtualized loads and databases in compact form factor.

The B420 M3 blade server

B420 M3 series blade servers are full-width blade servers with the highest memory density ideal for demanding enterprise application deployment.

The specifications of the B420 M3 blade server are as follows:

  • Four Intel® Xeon® processors of the E4600 product family CPUs

  • A maximum of 1.5 TB as the total memory capacity with a total of 48 memory slots

  • Four drive bays for internal SAS/SATA/SSD hard drives for a maximum storage capacity of up to 4 TB with built-in RAID 0,1,5,10 controllers

  • Up to 160 Gbps of I/O throughput with supported mezzanine cards

B440 M1/M2 blade servers

B440 M1/M2 series blade servers are full-width blade servers with four CPUs and high memory density, ideal for enterprise applications deployment.

The specifications of the B440 M2 blade server are as follows:

  • Four Intel® Xeon® processors of the E7-4800/8800 product family CPUs

  • A maximum of 1 TB as the total memory capacity with a total of 32 memory slots

  • Four drive bays for internal SAS/SATA/SSD hard drives for up to 3.6 TB as the maximum storage capacity with built-in RAID 0,1, 5, 10 controllers

  • Up to 40 Gbps of I/O throughput with a supported mezzanine card

Note

For a quick comparison of B-series blade servers, please visit http://www.cisco.com/en/US/products/ps10280/prod_models_comparison.html. Select all the servers and click on the Compare button.

Cisco also had UCS 250 M1/M2 blade servers which were discontinued in November 2012. You may still see these servers deployed in the field.

Learning more about rack-mount servers

A majority of server vendors manufacture rack-mount servers (also known as pizza-box servers). Cisco UCS C-series servers are available in the industry-standard rack-mount form which can be deployed for SMB remote offices and applications requiring dedicated equipment. Like B-series blade servers, these servers are also based on Intel Xeon processors and there is no AMD option available.

C-series servers can be managed independently through an out of band (OOB) web interface or can be integrated with the UCSM software. The embedded CIMC provides the following services:

  • Remote KVM to server console

  • Remote power management

  • Remote virtual media for operating system installation

  • Industry-standard IPMI support for monitoring

  • Standard SNMP traps for monitoring

Connecting and managing C-series rack servers through UCSM requires a connection through Nexus 2200 series FEXs, which act as line cards to the FIs.

C-series servers are available in various CPUs, memory, I/O, and storage configurations to address the needs of differently sized organizations. The following sections cover the specifications of these servers as available on the Cisco website.

The C22 M3 rack-mount server

The C22 M3 rack-mount server is an entry-level rack-mount server. It is based on the Intel E2400 CPU product family.

The specifications of the C22 M3 server are as follows:

  • Two Intel® Xeon® processors of the E5-2400 product family of CPUs

  • A maximum total memory capacity of 192 GB with a total of 12 memory slots

  • Up to eight small form factor (SFF) or four large form factor (LFF) internal storage SAS/SATA/SSD hard disks with an 8 TB (SFF) or 12 TB (LFF) storage capacity and an optional RAID controller

  • Two 1 Gbps I/O ports with optional 10 Gbps unified fabric ports

  • Two PCIe third-generation expansion slots

The C22 series rack-mount server is an economical choice for SMBs, branch offices, departments for entry level virtualization, and web farms.

The C24 M3 rack-mount server

The C24 M3 rack-mount server is also an entry level rack-mount server with storage extensibility. It is based on the Intel E2400 CPU product family.

The specifications of the C24 M3 server are as follows:

  • Two Intel® Xeon® processors of the E5-2400 product family of CPUs

  • A maximum total memory capacity of 192 GB with a total of 12 memory slots

  • Up to 24 SFF or 12 LFF internal storage SAS/SATA hard disks with a 24 TB (SFF) or 26 TB (LFF) storage capacity and an optional RAID controller

  • Two 1 Gbps I/O ports with optional 10 Gbps unified fabric ports

  • Five PCIe third-generation expansion slots

The C24 M3 series rack-mount server is an economical choice for SMBs, branch offices, departments for entry level virtualization, and web farms.

The C220 M3 rack-mount server

The C220 M3 rack-mount server is based on the Intel E2600 CPU product family.

The specifications of the C220 M3 rack-mount server are as follows:

  • Two Intel® Xeon® processors of the E5-2600 product family of CPUs

  • A maximum total memory capacity of 512 GB with a total of 16 memory slots

  • Up to eight SFF or four LFF internal storage SAS/SATA/SSD hard disks with an 8 TB (SFF) or 12 TB (LFF) storage capacity and an optional RAID controller

  • Two 1 Gbps I/O ports with optional 10 Gbps unified fabric ports

  • Two PCIe third-generation expansion slots

The C220 M3 series server has great performance and density for distributed database applications and web farms.

The C240 M3 rack-mount server

The C240 M3 rack-mount server is based on the Intel E2600 CPU product family.

The specifications of the C240 M3 rack-mount server are as follows:

  • Two Intel® Xeon® processors of the E5-2600 product family of CPUs

  • A maximum total memory capacity of 768 GB with a total of 24 memory slots

  • Up to 24 SFF or 12 LFF internal storage SAS/SATA/SSD hard disks with a 24 TB (SFF) or 26 TB (LFF) storage capacity and an optional RAID controller

  • Four 1 Gbps I/O ports with optional 10 Gbps unified fabric ports

  • Five PCIe third-generation expansion slots

The C240 M3 series server has great performance along with storage extensibility for distributed database applications and web farms.

The C260 M2 rack-mount server

The C260 M2 rack-mount server is based on the Intel E2800 CPU product family.

The specifications of the C260 M2 rack-mount servers are as follows:

  • Two Intel® Xeon® processors of the E7-2800 product family of CPUs

  • A maximum total memory capacity of 1 TB with a total of 64 memory slots

  • Up to 16 LFF internal storage SAS/SATA/SSD hard disks with a 16 TB (LFF) storage capacity and no RAID controller

  • Two Gigabit Ethernet (GE) LAN on Motherboard (LoM) I/O ports with two optional 10 Gbps unified fabric ports

  • Six PCIe third-generation expansion slots

The C260 M2 series server provides one of the highest densities of computational resources for business-critical applications.

The C420 M3 rack-mount server

The C420 M3 rack-mount server is based on the Intel E4600 CPU product family.

The specifications of the C420 M3 rack-mount server are as follows:

  • Four Intel® Xeon® processors of the E5-4600 product family of CPUs

  • A maximum total memory capacity of 1.5 TB with a total of 48 memory slots

  • Up to 16 LFF internal storage SAS/SATA/SSD hard disks with a 16 TB (LFF) storage capacity and an optional RAID controller

  • Two 1 Gbps I/O ports with two optional 10 Gbps unified fabric ports

  • Four PCIe third-generation expansion slots

The C420 M3 series server provides computational resources for business-critical applications such as large databases both in bare metal and virtualized environments.

The C460 M2 rack-mount server

The C460 M2 rack-mount server is based on the Intel E4800 CPU product family.

The specifications of the C460 M2 rack-mount servers are as follows:

  • Four Intel® Xeon® processors of the E7-4800 product family of CPUs

  • A maximum total memory capacity of 2 TB with a total of 64 memory slots

  • Up to 12 LFF internal storage SAS/SATA/SSD hard disks with a 12 TB (LFF) storage capacity and no RAID controller

  • Two GE LoM I/O ports with two optional 10 Gbps unified fabric ports

  • 10 PCIe third-generation expansion slots with an optional LSI MegaRAID controller in the eleventh slot

The C460 M2 series server provides exceptional computational resources for business-critical applications such as large databases both in bare metal and virtualized environments.

Note

For a quick comparison of C-series servers, please visit http://www.cisco.com/en/US/products/ps10493/prod_models_comparison.html. Select all the servers and click on the Compare button to get the results.

 

Getting started with mezzanine adapters


A huge variety of mezzanine adapters, also known as Virtual Interface Cards (VICs), is available from Cisco for both B-series blade servers and C-series rack servers. Older adapters are of the fixed port type and are not optimized for contemporary virtualized server environments. There are some older third-party network cards also available as an option. Newer adapters are optimized for virtualization and can provide 128 or 256 dynamic virtual adapters. The number of virtual adapters is dependent on the VIC model. These virtual adapters can be configured as Ethernet (vNIC) or fiber channel (vHBA) devices. All virtualization-optimized VICs also support the VM-FEX technology. Our focus will be on those mezzanine adapters that are virtualization optimized.

VICs for blade servers

VICs are available in the form of a mezzanine card. All new VICs provide dynamic vNIC or vHBA interfaces for server-side connectivity.

VIC 1280

The specifications of VIC 1280 are as follows:

  • 256 dynamic vNIC (Ethernet) or vHBA (FC) interfaces

  • VM-FEX support for virtualized environments

  • Hardware failover without driver need

  • 80 Gbps network throughput

  • Mezzanine form factor

  • Compatible with UCS M2 (B230 and B440) and all M3 blade servers

VIC 1240

The specifications of VIC 1240 are as follows:

  • 256 dynamic vNIC (Ethernet) or vHBA (FC) interfaces

  • VM-FEX support for virtualized environments

  • Hardware failover without driver need

  • 40 Gbps network throughput with optional 80 GB throughput using optional port expander in the mezzanine slot

  • LoM form factor

  • Compatible with all M3 blade servers

VIC M81KR

The specifications of VIC M81KR are as follows:

  • 128 dynamic vNIC (Ethernet) or vHBA (FC) interfaces

  • VM-FEX support for virtualized environments

  • Hardware failover without driver need

  • 20 Gbps network throughput

  • Compatible with UCS M2 blade servers

VICs for rack-mount servers

VICs are available as PCIe cards as well. All new VICs provide dynamic vNIC or vHBA interfaces for server-side connectivity.

VIC 1225

The specifications of VIC 1225 are as follows:

  • 256 dynamic vNIC (Ethernet) or vHBA (FC) interfaces

  • VM-FEX support for virtualized environments

  • Hardware failover without driver need

  • 20 Gbps network throughput

  • Compatible with UCS M2 (C460 and C260) and all M3 rack-mount servers

VIC P81E

The specifications of VIC P81E are as follows:

  • 128 dynamic vNIC (Ethernet) or vHBA (FC) interfaces

  • VM-FEX support for virtualized environments

  • Hardware failover without driver need

  • 20 Gbps network throughput

  • Compatible with UCS M2 (C460 and C260) and all M3 rack-mount servers

Note

For a quick comparison of mezzanine card specifications, please visit http://www.cisco.com/en/US/products/ps10277/prod_models_comparison.html#~tab-a.

Cisco VICs are also famous by their code name, Palo.

 

Power capacity and power plug types


The UCS 5108 blade chassis comes with options of up to four power supply units. Each is a single phase unit and provides 2,500 watts. Depending on the total number of power supplies in the chassis and input power sources, UCS 5108 can be configured into the following three modes:

Nonredundant mode

Power supply units installed in the system provide adequate power. A power supply failure results in a chassis failure. Load is evenly distributed among power supplies; however, there is no power redundancy.

N+1 redundant mode

In this mode, at least one extra power supply unit is available in the system in addition to the units that are required for providing the power required for the chassis to be operational. The extra power supply unit is in standby mode and the load is evenly distributed among the operational power supply units. In case of single power supply failure, standby power will replace the failed power supply immediately.

Grid redundant mode

In this mode, all the four power supply units must be available in the system and power should be supplied from two different power sources. Power supply units must be configured in pairs. Units 1 and 2 form one pair, and 3 and 4 form the second pair. Ideally, separate physical power cabling from two independent utility grids is recommended to feed each pair of power supply. In case one power source fails, the remaining power supply units on the other circuit continue to provide power to the system.

The power connector inside the UCS blade server chassis is IEC 60320 C19. The connector on the other side of the power cable varies according to the country-specific electrical standards.

Note

More information regarding power and environmental requirements is available at http://www.cisco.com/en/US/docs/unified_computing/ucs/hw/site_prep/guide/siteprep_tech_specs.html#wp1064498.

 

Installing UCS chassis components


Now that we have a better understanding of the various components of the Cisco UCS platform, we can delve into the physical installation of the UCS chassis, that is, the installation of blade servers, IOM modules, fan units, power supply units, SFP+ modules, and physical cabling.

Care must be taken during the installation of all components as failure to follow the installation procedure may result in component malfunction and bodily injury.

Tip

UCS chassis don'ts

Do not try to lift even an empty chassis alone. At least two persons are required to handle the UCS chassis.

Do not handle internal components such as CPU, RAM, and mezzanine cards without the Electro Static Discharge (ESD) field kit.

Before the physical installation of the UCS solution, it is also imperative to consider other datacenter design factors including the following:

  • Building floor load bearing capacity

  • Rack requirements for UCS chassis and FIs

  • Rack airflow, heating, and ventilation (HVAC)

Physical installation is divided into the following three sections:

  • Blade server component (CPU, memory, hard drives, and mezzanine cards) installation

  • Chassis component (blade servers, IOMs, fan units, and power supply units) installation

  • FI installation and physical cabling

Blade server installation

Cisco UCS blade servers are designed on industry-standard components with some enhancements. Anyone with prior server installation experience should be comfortable installing internal components using the guidelines provided in the blade server manual and following the standard safety procedures. ESD transient charges may result in thousands of volts of charge building up, which can degrade or permanently damage electronic components.

Note

The Cisco ESD training course may be referred to at http://www.cisco.com/web/learning/le31/esd/WelcomeP.html.

All Cisco UCS blade servers have similar cover design with a button at the top front of the blade; this button needs to be pushed down. Then, there is a slight variation among models in the way that the cover slides; this could be either towards the rear and upwards or towards self and upwards.

Installation and removal of CPU

The following is the procedure to mount a CPU onto a UCS B-series blade server:

  1. Make sure you are wearing an ESD wrist wrap grounded to the blade server cover.

  2. To release the CPU clasp, first push it down and then to the side away from the CPU socket.

  3. Move the lever up and remove the CPU blank cover. Keep the blank cover in a safe place just in case you need to remove a CPU.

  4. Pick up the CPU with the plastic edges and align it with the socket. The CPU can only fit one way.

  5. Lower the mounting bracket with the side lever and secure the CPU into the socket.

  6. Align the heat sink with its fins in a position allowing unobstructed airflow from front to back.

  7. Gently tighten the heat sink screws on to the motherboard.

CPU removal is the reverse of the installation process. It is critical to place the socket blank cover back over the CPU socket. Damage could occur to the socket without the blank cover.

Note

For UCS B440, air blocker must be installed if you permanently remove a CPU.

Installation and removal of RAM

The following is the procedure to install RAM modules into the UCS B-series blade server:

  1. Make sure you are wearing an ESD wrist wrap grounded to the blade server cover.

  2. Move away the clips on the side of the memory slot.

  3. Hold the memory module with both edges in an upright position and firmly push straight down, matching the notch of the module to the socket.

  4. Close the side clips to hold the memory module.

Memory removal is the reverse of the installation process.

Note

The memory modules must be inserted in pairs and split equally between each CPU if all the memory slots are not populated. Refer to the server manual for identifying memory slot pairs and slot-CPU relationship.

Installation and removal of internal hard disks

UCS supports SFF serial attached Small Computer System Interface (SCSI) (SAS) hard drives. Blade servers B200, B240, and B440 support regular thickness (15 mm) hard drives whereas B230 supports thin (7 mm) hard drives.

To insert a hard disk into the B200, B250, and B440 blade servers, carry out the following steps:

  1. Make sure you are wearing an ESD wrist wrap grounded to the blade server cover.

  2. Remove the blank cover.

  3. Press the button on the catch lever on the ejector arm.

  4. Slide the hard disk completely into the slot.

  5. Push the ejector lever until it clicks to lock the hard disk.

To remove a hard disk press the release button, pull the catch lever outward, and slide the hard disk out.

To insert or release a thin hard drive into or from the B230 server, release the catch by pushing it inside while inserting or removing the hard disk.

Note

Do not leave a hard disk slot empty. If you do not intend to replace the hard disk, cover it with a blank plate to ensure proper airflow.

Installation of mezzanine cards

UCS B200 and B230 support single mezzanine cards whereas B250 and B440 support two cards. The procedure for installing these cards is the same for all servers, which is as follows:

  1. Make sure you are wearing an ESD wrist wrap grounded to the blade server cover.

  2. Open the server top cover.

  3. Grab the card with its edges and align the male molex connector, the female connector, and the motherboard.

  4. Press the card gently into the slot.

  5. Once the card is properly seated, secure it by tightening the screw on top.

Mezzanine card removal is the reverse of the installation process.

Installation of blade servers on the chassis

Installation and removal of half-width and full-width blade servers is almost identical with the only difference being the use of one ejector arm for half-width blade servers whereas for full-width blade servers, there are two ejector arms. Carry out the following steps:

  1. Make sure you are wearing an ESD wrist wrap grounded to the chassis.

  2. Open one ejector arm for the half-width blade servers or both ejector arms for full-width blade servers.

  3. Push the blade into the slot. Once firmly in, close the ejector arm on the face of the server and tighten the screw with your hands.

The removal of a blade server is the opposite of the installation process.

Note

In order to install a full-width blade, it is necessary to remove the central divider. This can be done with a Philips screwdriver to push two clips, one in the downward and the other in the upward direction, and sliding the divider out of the chassis.

 

Cabling FI and IOM


UCS is an integrated solution that handles both network traffic and management control. All management and data movement intelligence for chassis components and blade servers is present in the FIs and IOM modules (which are line cards for the FIs). Therefore, proper cabling between FIs and IOM modules is an important design consideration.

IOM – FI cabling topology

IOM modules are used to connect blade server chassis to the FIs and act as line cards to them. It is therefore necessary to maintain proper connectivity between IOMs and FIs. Since an IOM module becomes part of the FI, multiple links from a single IOM can only be connected to a single FI and not across to the other FI. Depending on the IOM model, there can be one, two, four, or eight links from IOM to a single FI. These links can be configured in the port channel for bandwidth aggregation. The chassis discovery process is initiated as soon as an IOM is connected to an FI.

In the following figure, on the left-hand side, all links from IOM 0 are connected to a single FI and can be combined into a single port channel. The figure on the right shows a configuration in which links from a single IOM are connected to different FIs. This is an invalid topology, and hence chassis discovery will fail.

Note

Chassis discovery will also fail if a high availability cluster is not established between FIs.

IOM – FI physical cabling

IOMs provide connectivity to the individual blade servers through I/O multiplexer and connectivity to the FI. IOM interface connectivity to blade servers does not require user configuration.

IOM to FI connectivity, however, requires physical cabling. Both IOM and FI have SFP+ slots. There are a variety of possibilities in terms of physical interfaces. Some of the common configurations include the following:

  • 10 GB FET SFP+ interface (special optical multimode fiber SFP+ module which can only be used with UCS and Nexus equipment)

  • 10 GB CU SFP+ (copper twinax cable)

  • 10 GB SR SFP+ (short range multimode optical fiber SFP+ module for up to 300 m)

  • 10 GB LR SFP+ (long-range, single-mode optical fiber SFP+ module for above 300 m)

The following figure shows eight connections from IOM 0 to Fabric Interconnect A and eight connections from IOM 1 to Fabric Interconnect B. Depending on the bandwidth requirements and model, it is possible to have only one, two, four, or eight connections from IOM to FI.

Although large numbers of links provide higher bandwidth for individual servers, as each link consumes a physical port on the FI, they also decrease the total number of UCS chassis which can be connected to the FIs.

As shown in the preceding figure, IOM to FI only supports direct connection. However, FI to north-bound Nexus switch connectivity can be direct and may use regular port channel (PC), or the connections from a single FI may traverse two different Nexus switches and use virtual PortChannel (vPC).

The following figure shows a direct connection between FIs and Nexus switches. All connections from FI A are connected to the Nexus Switch 1 and all connections from FI B are connected to Nexus Switch 2. These links can be aggregated into a PC.

The following are two other connections that need to be configured:

  • Cluster heart beat connectivity: Each FI has two fast Ethernet ports. These ports should be connected using a CAT6 UTP cable for cluster configuration

  • Management port: This is also a fast Ethernet port that can be configured using a CAT6 UTP cable for remote management of the FI

The following figure shows the FI to Nexus switch connectivity where links traverse Nexus switches. One network connection from FI A is connected to Nexus Switch 1 and the other to Nexus Switch 2. Both these connections are configured via vPC. Similarly, one connection from FI B is connected to Nexus Switch 2 and the other to Nexus Switch 1. Both these connections are also configured via vPC. It is also imperative to have vPC on a physical connection between both the Nexus switches. This is shown as two physical links between Nexus Switch 1 and Nexus Switch 2. Without this connectivity and configuration between Nexus Switches, vPC will not work.

Physical slots in Nexus switches also support the same set of SFP+ modules for connectivity that FIs and IOMs do.

Note

A complete list of SFP+ modules is available in Table 3 at http://www.cisco.com/en/US/prod/collateral/ps10265/ps10276/data_sheet_c78-524724.html.

 

Summary


The Gartner Magic Quadrant report for blade servers, published in 2012, placed Cisco as a market leader along with Dell, HP, and IBM. Cisco has already surpassed Dell from its number three position. According to this report, Cisco's entry into the blade server market is causing re-evaluation among the installed blade server bases. Cisco's clustered FI-based design provides a complete solution for converged data connectivity, blade server provisioning, and management whereas other vendor offerings acquire the same functionality with increased complexity.

Cisco UCS presented a paradigm shift for blade servers and datacenter design and management to the industry. In this chapter, we learned about the UCS solution's integral components and physical installation of the UCS solution.

We learned about the various available options for all the UCS components including FIs, blade chassis, blade servers, rack-mount servers, and the internal parts of the servers.

In the next chapter, we will learn about Cisco UCS Emulator which is an excellent tool for exploring UCSM software.

Note

The detailed Gartner Magic Quadrant report, 2012, is available at http://www.gartner.com/technology/reprints.do?id=1-19KZ0QR&ct=120306&st=sb.

About the Authors

  • Farhan Nadeem

    Farhan Nadeem has been in the IT field for over 19 years. He has a master's degree in electrical engineering and holds several industry-recognized certifications, including VCAP-DCA, VCP, CCNP DC, CISSP, CCA, and MCIP-EA. Farhan has proven experience in successfully engineering, deploying, administering, and troubleshooting heterogeneous infrastructure solutions. Starting with the MCSE-NT Microsoft certification in 1997, he's always stayed abreast of the latest technologies and server hardware through proactive learning and successful real-world deployments. He has extensive work experience in complex heterogeneous environments comprising various hardware platforms, operating systems, and applications. This exposure has given him broad knowledge in investigating, designing, implementing, and managing infrastructure solutions. He progressively started focusing on virtualization technologies and the Cisco UCS platform and has completed several successful UCS deployments with multiple virtualization platforms. When not working with computers, he enjoys spending time with his family. He has also technically reviewed the second edition of this book.

    Browse publications by this author
  • Prasenjit Sarkar

    Prasenjit Sarkar is a product manager at Oracle for their public cloud, with a focus on cloud strategy, Oracle Ravello, cloud-native applications, and the API platform. His primary focus is driving Oracle's cloud computing business with commercial and public sector customers, helping to shape and deliver a strategy to build broad use of Oracle's Infrastructure as a Service offerings, such as Compute, Storage, and Database as a Service. He is also responsible for developing public/private cloud integration strategies, customers' cloud computing architecture visions, future state architectures, and implementable architecture roadmaps in the context of the public, private, and hybrid cloud computing solutions that Oracle can offer.

    He has also authored six industry-leading books on virtualization, SDN, and physical compute, among others.

    He has six successful patents and six more patents pending at the US PTO. He has also authored numerous research articles.

    Browse publications by this author

Latest Reviews

(1 reviews total)
This book is full of errors. Unfortunately I cannot recommend this book to any reader.
Book Title
Access this book, plus 7,500 other titles for FREE
Start FREE trial