What do you get with Print?

Instant access to your digital copy whilst your Print order is Shipped

Paperback book shipped to your preferred address

Redeem a companion digital copy on all Print orders

Access this title in our online reader with advanced features

DRM FREE - Read whenever, wherever and however you want

OpenNebula 3 Cloud Computing

Chapter 1. OpenNebula and Why it Matters?

In the last few years, many IT environments have been facing profound changes in how hardware infrastructure is designed and how software is being managed. This is thanks to what is popularly called The Cloud.

A cloud provides users with computation power, storage, and software services. It does not require knowledge of the physical location or the specific hardware configuration of where the services are running. It runs on the the same logic that applies to a power grid providing energy to different houses, where a consumer does not need to know how the power is produced or how it is delivered to the house. Services provided by a cloud can be subdivided into three main service layers, as follows:

Software as a Service (SaaS)
Platform as a Service (PaaS)
Infrastructure as a Service (IaaS)

The Application layer is used when the user has access to a service without a need to know the physical infrastructure on which the application relies. The installation, configuration, and maintenance are completely managed by the provider itself, such as the popular Gmail service. The main advantages over legacy software are minor maintenance costs and no hardware costs at all.

The Platform layer is used when the user has access to a particular instance of an operating system or a software stack, with the desired amount of CPU power, memory, and storage available. Each instance is completely isolated from others running on the same hardware, and it is easy to scale out resources as they are needed. Take for example Google App Engine, where developers can deploy their own Python applications without the need to know how to configure a high-performance scalable stack. Also, they do not need to make long-term hardware provisioning plans to be prepared for future resource needs.

The Infrastructure layer is the most complex one. It involves different components that need to be orchestrated to be effective (for example, CPU power, memory allocation, storage, network, Virtual Memory (VM) instance management, and end-user access). It also allows complete outsourcing of IT infrastructure—the users pay for their effective usage, and even complex architectures can be managed without the need to have a bunch of physical systems. A typical example is Amazon EC2.

As there are situations where you cannot simply rely on external cloud providers (because of strict business policies), and you cannot afford proprietary virtualization solutions, cloud frameworks such as OpenNebula will be of great help. It will act as an effective open source toolkit, which can easily be adapted to work flawlessly even in heterogeneous environments, to build private, public, and hybrid IaaS.

OpenNebula started as a research project by Distributed Systems Architecture (DSA) Research Group (http://dsa-research.org) with a few of the current features. Based in Madrid in 2005, and later released in 2008 as completely open source, it is now actively developed by the community that has grown around it. Today, OpenNebula has achieved a very neat feature set thanks also to its great modularity, which eases the integration with other solutions.

The main difference between OpenNebula and other commercial cloud solutions is that its true open source blood guarantees users complete interoperability with every existing infrastructure component already available. Thus, it avoids the vendor lock-in using common open industrial standards, such as EC2 API and Open Cloud Computing Interface (OCCI).

Unlike other open source alternatives, OpenNebula does not embrace a particular hypervisor. It also does not have any specific infrastructure requirements, fitting well into any pre-existing environment, storage, network, or user-management policies.

The plugin model on which OpenNebula is implemented, gave system integrators the ability to customize every aspect including virtualization, storage, information, authentication, authorization, and remote cloud services. Every action is managed by a bash script that can easily be modified or plugged with some other custom script or software written in any language and supported by your operating system.

The following diagram will help you to understand which components are involved in OpenNebula and also the level at which they operate (lower ones interact directly with the host's resources, higher ones interact with user interfaces).

On the lowest level, there are drivers that directly talk to the underlying software OS components. They are as follows:

Transfer drivers: These are used to manage the disk images on the current storage system—a shared one, such as Network File System (NFS) or Internet Small Computer System Interface (iSCSI), or on a non-shared one such as a simple copy over Secure Shell (SSH).
Virtual Machine drivers: These are hypervisor-specific and they are used for managing the virtual machine instances on the current hosts.
Information drivers: These are used to retrieve the current status of virtual machine instances and hosts. They are hypervisor-specific, too—they are copied and remotely executed in every physical host through SSH.

All the monitoring information collected from physical hosts and VM instances, along with the configurations of every VM, the available disk images (images catalog) and the virtual networks, is stored in a simple SQLite database or a replicated MySQL database. It can easily be retrieved or altered by custom scripts or software, depending on your infrastructure needs.

The user can take advantage of the native OpenNebula cloud API, available as Java, Ruby, and XML-RCP API. It gives access to all the functions available and permits an easy integration of custom procedures before or after the standard one.

For performance reasons, the core of OpenNebula is written in highly optimized C++ code, giving good scalability. A good example of its robustness is the CERN infrastructure prototype, initially managing 480 server hosts; in spring 2010, they reached an impressive figure of 16,000 virtual instances.

A hook system is implemented to give users the ability to execute custom scripts after a predefined set of events, providing an easy way to send e-mail notifications upon changes, or to provide a simple failover mechanism.

Security is also taken into serious consideration. Host communication takes place exclusively through secured connections protected with the SSH RSA keypairs and Secure Socket Layer (SSL). Each virtual network is isolated with a firewall, ebtables (http://ebtables.sourceforge.net/). It works at the Address Resolution Protocol (ARP) level, also known as the data link layer in the OSI stack.

OpenNebula has a dedicated Quality Assurance (QA) team that deals with both unit and functional tests with a wide range of scenarios. This greatly reduces the number of bugs, or at least enables a greater user awareness of such bugs. There is also in use a continuous integration system, which carries on automatically building and testing every change-set pushed by OpenNebula developers, available at (http://hudson.opennebula.org/).

The OpenNebula project aims high, as its objectives are to develop the most advanced, scalable, and adaptable software toolkit with quality and stability checks for every release. It also actively searches for community support and contributes to the open source ecosystem—every bug request is analyzed by the team, and every bug found in the underlying software components is forwarded back to the respective project owner.

These kinds of operations are essential for maintaining a high profile and for attracting new people, to use and participate in the development of the project. Many third-party projects related to OpenNebula are directly hosted on the main site, emphasizing the work of every participating user, even for small contributions. The initial contributions arrived from the DSA research group (http://dsa-research.org) at the Universidad Complutense de Madrid, which has received funding from the European Union's Seventh Framework Programme, for the adoption of cloud solutions for scientific grid computing

Since mid-2010, OpenNebula has had official commercial support from C12G Labs, after numerous requests for support; this gave a boost in vitality to the project.

The core features

The first steps with OpenNebula are towards the realization of a private cloud. A private cloud does not expose any API, and every resource is used for internal purposes only. When cloud resources are available, whether exclusively or not, to third-party users through a predefined set of APIs, it is named a public cloud. When you use external resources to improve your cloud, or you expose to third-party users your own local resources or both, it is called a hybrid cloud.

Starting with the private cloud, we can identify a set of features, as follows:

User management: It is possible to configure multiple users, who will have access only to their own instances, the ability to account for used resources, and with limits enforced by quota
VM Image management: Every disk image is registered and managed by a centralized image catalog
Virtual Network management: It is possible to define multiple networks bonded to different physical interfaces, with either static or dynamic IP address assignment
Virtual Machine management: Every machine has its own set of characteristics (for example, CPU, memory, disk storage, and virtual network) and can be launched under every available hypervisor of our cluster
Service management: A group of virtual machines can be grouped for being deployed together at boot time, and every virtual machine can be configured at boot time, without the need to assign different disk images for similar machines
Infrastructure management: The physical hosts can be managed alone or grouped on independent clusters, and it is useful when you have a heterogeneous environment
Storage management: The support for most common storage solutions is found in data centers such as FibreChannel, iSCSI and shared storage such as Network Attached Storage (NAS) with specific support for optimal disk image management
Information management: Every host and every virtual machine is actively monitored every few seconds, and it is already available in integration with standard monitoring tools such as Ganglia
Scheduling: Virtual machines are deployed on host nodes following specific user requirements and resource-aware policies, such as packing, striping, or load-aware
User interface: It includes the command-line tools available for managing every aspect of OpenNebula (for example, cluster status, virtual machines status, image repository, and so on)
Operations center: Most of the information and tasks available from the command line are available on web interfaces browsable with any modern web browser on any operating system (even Android smartphones or tablets)

For a Hybrid cloud, which uses both local and remote resources, the two main features available are as follows:

Cloud-bursting: It is the ability to add computing resources to your local infrastructure, using external resources, in order to meet peak demands or implement high-availability/disaster recovery strategies. This is essential for having a flexible and reliable infrastructure.
Federation: It is the ability to combine together different clusters, dislocated in different physical positions, enabling higher levels of scalability and reliability.

For a Public cloud, the main feature is exposing the cloud resources to externals with one or more standard interfaces, such as the cloud interfaces that enable the reachability of cloud resources, in a secure way, to external users, with OCCI and EC2 standard API interfaces.

Standing on the shoulders of the giants

As in any other open source project, OpenNebula does not reinvent the wheel and makes efficient use of existing open source technologies for its foundations.

Xen

The first adopted OpenNebula hypervisor is Xen (http://www.xen.org/). It has been a unique leading open source virtualization technology for many years. Today, it is still one of the most advanced virtualization platforms, maintained by over 50 vendors, including AMD, Cisco, Dell, Fujitsu, HP, IBM, Intel, Novell, Red Hat, Samsung, and Citrix.

Besides its use as a hypervisor in OpenNebula, Xen is also used standalone by many Internet hosting companies such as Amazon EC2, Linode, and Rackspace Cloud. It was originally distributed as a Linux patchset, but is nowadays included in main GNU/Linux distributions such as SuSe, RedHat, and Debian.

Xen is composed of the following three modules:

Hypervisor: The core component responsible for scheduling and executing all the virtual machine instances currently running.
Dom0: It is a privileged virtual machine running the base system and having direct hardware access. It is used to manage all the other deprivileged instances.
DomU: An unprivileged virtual machine running on the hypervisor and having access only to virtual resources exposed by Dom0.

Before the introduction of specific Intel/AMD CPU support for virtualization, Xen used a form of virtualization known as paravirtualization, meaning that virtual machines required a modified operating system if they were to run well with negligible virtualization overhead.

Without specific Xen support it was not possible to execute arbitrary operating systems as virtual machines.

Nowadays, with the help of specific CPUs' virtualization support, the guest operating system does not need any modification to run. Hence, it is possible to run any Linux, Windows, *BSD, or Solaris version with good performance.

KVM

After the Xen dominance in the past, the Kernel-based Virtual Machine (KVM) has grabbed more attention from the public in recent years (http://www.linux-kvm.org/). It has been directly integrated into the mainline kernel sources from release 2.6.20 of February 2007 and has been readily available in every GNU/Linux distribution from that point on.

KVM was being initially developed by a techie start-up, Quramnet, bought in 2008 by RedHat, and is now actively maintained by Linux developers all around the world.

The KVM design is another interesting point, because KVM by itself is only an interface available to user space programs that can be called through the /dev/kvm special system file. For similar reasons, another open source project has been ported to support the KVM interface in gaining a full virtualization environment, QEMU.

QEMU/KVM

Quick Emulator (QEMU), which can be accessed at http://wiki.qemu.org/, is a famous CPU emulator developed by Fabrice Bellard (one of the most surprising open source developers in the world). It has also laid the foundation for many other virtualization platforms, such as Xen and VirtualBox.

Over time, we have been introduced to the QEMU features. It includes support for the most common hardware devices, such as serial port, network card, PCI-ATA interface, USB controller and so on. Now, it has quickly become one of the first user space virtual machine emulators.

Thanks to the virtualization hardware support implemented by both Intel and AMD on their CPUs, the QEMU project forked in KVM to natively support those hardware extensions.

As in Xen, KVM supports paravirtualization thanks to the implementation of VirtIO devices. Instead of emulating real hardware like standard QEMU does, special devices using an API for virtual I/O result in better performances. Thus, the objective to provide a standardized set of hardware for every hypervisor is fulfilled. VirtIO devices are supported natively by recent Linux releases, and their downloadable drivers are available even for Windows. The usage of VirtIO for storage and network devices is strongly suggested.

Libvirt

Finally, the latest abstraction layer in a KVM/XEN virtualization stack is Libvirt (http://libvirt.org/). Libvirt is a collection of software, developed by RedHat, which provides an API interface for any major programming language that should be used for the following:

VM management: To control the complete life cycle of a virtual machine, including monitoring and migration.
Remote machine support: To reach and control remote hosts with a simple SSH tunnel or a more advanced SSL protocol.
Storage management: To create disk images in various formats, management of LVM groups, raw devices, and iSCSI.
Network interfaces management: To set up new bridges, VLANs, and bond devices. Automatically manage iptables for Network Address Translation (NAT) virtual machines.

In recent releases, Libvirt has included support for the major hypervisors available, becoming the de facto standard tool for managing virtual machine instances.

VMware

Finally, OpenNebula includes support for a range of VMware hypervisor versions (http://www.vmware.com/products/datacenter-virtualization/), that including the following:

VMware ESXi: It is a free (not open source) hypervisor, the simplest of the whole family, and natively includes only a command-line interface, and runs on its own kernel (not on Linux, as Xen/KVM do). Hence, the hardware support is pretty limited or highly optimized, depending on how you see it.
VMware ESX: It was the mainline product before ESXi. It includes a Java web interface, and it is available under commercial license only.
VMware Server: It is a free (not open source) hypervisor, available for installation on Linux and Windows; it includes a Java web interface such as ESX, but with fewer features.

VMware hypervisors historically offer better performance and are better integrated with Windows virtual machines. However, nowadays there is practically no relevant difference from the other two open source hypervisors. Maybe, in some environments, VMware hypervisors are required due to strict software certification licenses, which are not so easy to circumvent in Enterprise software.

We will mainly take into consideration VMware ESXi because of its Xen-/KVM-like architecture than the other two VMware products considered independently. However, keep in mind that the infrastructure used by OpenNebula is common.

Note

Remember that VMware ESXi does not have out-of-the-box support for every kind of hardware that Xen/KVM has, thanks to the Linux kernel. Hence, you should check the online VMware compatibility guide at http://www.vmware.com/resources/compatibility/search.php or ask your hardware supplier before buying new hardware, otherwise you may not even be able to install it.

Key benefits

Take advantage of open source distributed file-systems for storage scalability and high-availability

Build-up, manage and maintain your Cloud without previous knowledge of virtualization and cloud computing

Install and configure every supported hypervisor: KVM, Xen, VMware

Step-by-step, focused on Ubuntu/Debian distributions, but with general how-to working with every GNU/Linux distribution

Description

OpenNebula is one of the most advanced and highly-scalable open source cloud computing toolkits. If you ever wanted to understand what Cloud Computing is and how to realize it, or if you need a handy way to manage your messy infrastructure in a simple and coherent manner, this is your way. OpenNebula 3 Cloud Computing guides you along the building and maintenance of your cloud infrastructure, providing real-world examples, step-by-step configuration and other critical information. The book keeps you a step ahead in dealing with the demanding nature of cloud computing and virtual infrastructure management using one of the most advanced cloud computing toolkitsñ OpenNebula. The book takes you from a basic knowledge of OpenNebula to expert understanding of the most advanced features.The book starts with a basic planning of hardware resources and presents the unique benefits of the supported hypervisors; you will go in deep with day-to-day management of virtual instances, infrastructure monitoring and integration with Public Clouds like Amazon EC2.With this book you will be able to get started with fast and cheap configuration recipes, but also go deeper for a correct integration with your existing infrastructure.You will deal with well-know virtualization technologies like Xen and VMware, but also with the promising KVM technology integrated in the Linux kernel. After the basic infrastructure set-up, you will learn how to create and manage virtual instance via both command-line and web interfaces, and how to monitor your existing resources.At the end, the book acquaints you with integrating your local infrastructure with external Cloud resources but also publishing your resources to others via common API interfaces.

Who is this book for?

If you are a GNU/Linux system administrator with no experience with virtualization or cloud computing but eager to learn about it, or you are thwarted by your current virtualized infrastructure, this book is for you. You are expected to have some basic knowledge of GNU/Linux, with knowledge of basic package management tools and system configuration.

What you will learn

Plan your hardware infrastructure and keep resources and hardware under monitoring

Install OpenNebula, from sources or binary distribution and configure it on your front-end host

Install and configure KVM, Xen and VMware ESXi on your hosts, building from sources when needed

Integrate with existing NAS/SAN infrastructure or provide flexible and scalable storage with distributed file-systems

Manage your day to day virtual instances via both command-line and web interfaces

Monitor your infrastructure continuously using OpenNebula

Extend your cloud with resources from remote Cloud providers using standard API interface

Provide Cloud resources to external facilities through standardized API interfaces

What do you get with Print?

Instant access to your digital copy whilst your Print order is Shipped

Paperback book shipped to your preferred address

Redeem a companion digital copy on all Print orders

Access this title in our online reader with advanced features

DRM FREE - Read whenever, wherever and however you want

Frequently bought together

$65.99

Responsive Web Design with HTML5 and CSS3

$43.99

Total $ 109.98

Filter reviews by

All

Amazon verified reviews

NK Aug 31, 2012

Book is very compact in size, taking into account so much of content about OpenNebula. Last two years I have been installing, configuring and running OpenNebula for the research with few dozens of servers. The book has very helpful theoretical and mostly technical content.For my work I referred to online documentation, blogs, OpenNebula forum and mailing lis. Finally, Giovanni Toraldo made an excellent work, collecting all of the content needed for very beginner to industry expert for set up, manage ad maintain OpenNebula cloud. It is very nice to have all content in one place. Book makes very easy to understand cloud internals, not only for OpenNebula but for all existing open source cloud solutions.Also, I found "Tips and tricks" boxes very useful. They remind me easy traps which beginner can fall into during initial set up. Notes in the boxes are very handy, since it might save your days of digging down to the problem, reading forum posts or Googling. Latter was the case for me. So, content of the book requires keen attention, especially notes boxes.It is great to have a book like this where author shares his insight experience and knowledge about open source project. Usually, thorough explanation of software modules, script and interaction between parts are not explained in the documentation, and almost impossible to find. This book includes in depth explanation of parts of the OpenNebula, such as image transfer manager, hooks, Ganglia integration and etc. I highly recommend this book for everyone getting into the cloud, regardless of their level of experience.

Amazon Verified review

C. Moeller Oct 01, 2012

If you have been looking into understanding, and seeing how to setup your own cloud solution, this is a great book. The book starts with an overview of OpenNebula, then into all the different things you'll have to consider when starting, and setting up Open Nebula. You'll learn how to everything up, what hardware you should consider, and how to install extra applications that it easier to interface and use OpenNebula, as well as how to administer and manage your datacenter, once it is setup.If you have been interested in cloud computing, and would like to see how to set it up using an open source solution like OpenNebula, this will be a great book to get you setup and working.

Javi Jul 23, 2012

This book provides a good reference for anyone that wants to learn about OpenNebula-based Infrastructure as a Service (IaaS) clouds. The book covers the different scenarios that you can have within an IaaS cloud. Thus, it can be used to just get a general introduction and deploy a testing infrastructure; or to plan and deploy a production infrastructure.Since the information is structure as a guide, it is easy to follow and learn how to manage OpenNebula. Moreover, every time that a related technology is mentioned, like GlusterFS, Glanglia, or Amazon Web Services, it goes with an explanation of how to install and configure it to ensure the integration with OpenNebula.I found particularly interesting the last two sections, where it is explained how to go beyond the limits of you private cloud by using resources offered by public clouds (such as Amazon Web Services). In addition, it details the way to convert your private cloud into a public one and how to manage, from a centralized place, a large infrastructure as a federation of clouds.

Stefan Freitag Jul 06, 2012

From 2006 to mid 2012 I was working in the field of Grid Computing and also administered a small high performance compute cluster at a university. One of my aims concerning this cluster was the virtualization of all physical machines. Then I heard of Cloud Computing and OpenNebula in particular. Cloud Computing implies very often virtualization and OpenNebula seemed to be a solution to the question "how do i manage the virtual machines".To sum up, I started very early with OpenNebula and accompanied its development over a long time. Unfortunately I had to shift my focus, so that I missed newer developments in OpenNebula, e.g. the cloud operations center "Sunstone". Hence, I am quite happy that Giovanni took the time to write this book! From my point of view, it is written for beginners as well as experts. Giovanni describes not only the internal architecture of OpenNebula with its core components, but also the basic ideas of virtualization and storage management. Beginners learn how to setup Xen-, KVM-, or VMware ESXi-based compute resources and how to integrate them into OpenNebula. Experts interested in shared storage solutions will find information about GlusterFS and MooseFS.

Rodrigue Chakode Aug 27, 2012

PACKT Publishing [1] recently contacted me for reviewing the Book "OpenNebula 3 Cloud Computing" [2] written by Giovanni Toraldo. Despite my agenda these last months and the size of the book (315 pages), I accepted this proposition.I like OpenNebula that I worked with more than three years during my PhD. From the version 1.2 the first one I downloaded in 2009, the software has been significantly improved. However, I was often frustrated that this nice software lacks of a handbook for learning tips and tricks to easily set up the clouds required for my experiments. That's, among other reasons, why I accepted to review this book.Globally, I really enjoyed reading the book. Well written, it covers the major aspects of OpenNebula. The author starts by introducing the basics of the virtualization technologies (e.g. Xen, KVM, VMware) so to help the reader to understand what OpenNebula is and how it works along with those virtualization technologies. The author then provides a step-by-step guide for installing the software. I particularly appreciated how the storage of virtual machine images as well as the applications' data are addressed. Two examples are provided with GlusterFS [3] and MooseFS [4], two major distributed file systems. The author goes deeper into various other OpenNebula's features such as Virtual Network Management, Web Management, Monitoring, coupling with public clouds [5] (e.g. Amazon EC2). I also appreciated the fact that he provides relevant examples for illustrating the functionalities that are introduced.I have some minor criticisms about the book however. I think that lacks of some conceptual considerations, to address for example situations where we need to build custom clouds based on OpenNebula. Indeed, instead of basis IaaS clouds, many people are often interested in building custom clouds. A scientist for instance will be mainly interested in how to customize OpenNebula to prototype the new models of clouds he is designing. Unfortunately, the book just briefly introduces the contextualization as well as the OpenNebula's APIs (OCCI, XML-RPC) that are two relevant features for addressing such a need. May be that will be covered in the next edition of the book.[...][...][...][...][...]

OpenNebula 3 Cloud Computing: This book will teach you to build and maintain a cloud infrastructure using OpenNebula, one of the most advanced, highly scalable toolkits for GNU/Linux. Walks you through from initial planning to advanced management techniques.

What do you get with Print?

OpenNebula 3 Cloud Computing

Chapter 1. OpenNebula and Why it Matters?

The core features

Standing on the shoulders of the giants

Xen

KVM

QEMU/KVM

Libvirt

VMware

Note

Summary

Page 1 of 3

Key benefits

Description

Who is this book for?

What you will learn

Product Details

What do you get with Print?

Product Details

Frequently bought together

Table of Contents

Recommendations for you

Customer reviews

Filter reviews by

People who bought this also bought

FAQs

OpenNebula 3 Cloud Computing: This book will teach you to build and maintain a cloud infrastructure using OpenNebula, one of the most advanced, highly scalable toolkits for GNU/Linux. Walks you through from initial planning to advanced management techniques.

What do you get with Print?

Contact Details

Shipping Address

Billing Address

Key benefits

Description

Who is this book for?

What you will learn

Product Details

What do you get with Print?

Contact Details

Shipping Address

Billing Address

Product Details

Packt Subscriptions

Frequently bought together

Table of Contents

Recommendations for you

Customer reviews

Filter reviews by

People who bought this also bought

FAQs

Create a Free Account To Continue Reading

Sign in to activate your 7-day free access