Proxmox Cookbook

By Wasim Ahmed
    What do you get with a Packt Subscription?

  • Instant access to this title and 7,500+ eBooks & Videos
  • Constantly updated with 100+ new titles each month
  • Breadth and depth in over 1,000+ technologies
  1. Free Chapter
    Installing Proxmox

About this book

Proxmox VE goes head to head with big virtualization platforms without bearing the overheads of high costs and vendor lock-ins. Rock solid stability and extremely easy manageability gives Proxmox VE an edge in the world of virtualization. The Proxmox cluster file system (pmxcfs) is designed uniquely for storage of configuration files with high security and reliability. Its simplicity and robustness make it easy for system administrators to manage their systems efficiently.

Proxmox VE's intuitive interface, high availability, and unique central management system puts it on par with the world’s best virtualization platforms. Its simplicity and high quality of service is what makes it the foremost choice for most system administrators.

Starting with a step-by-step installation of Proxmox nodes along with an illustrated tour of Proxmox graphical user interface where you will spend most of your time managing a cluster, this book will get you up and running with the mechanisms of Proxmox VE. Various entities such as Cluster, Storage, and Firewall are also covered in an easy to understand format. You will then explore various backup solutions and restore mechanisms, thus learning to keep your applications and servers safe. Next, you will see how to upgrade a Proxmox node with a new release and apply update patches through GUI or CLI.

Monitoring resources and virtual machines is required on an enterprise level, to maintain performance and uptime; to achieve this, we learn how to monitor host machine resources and troubleshoot common issues in the setup. Finally, we will walk through some advanced configurations for VM followed by a list of commands used for Proxmox and Ceph cluster through CLI.

With this focused and detailed guide you will learn to work your way around with Proxmox VE quickly and add to your skillset.

Publication date:
August 2015


Chapter 1. Installing Proxmox

In this chapter, we are going to cover the following Proxmox basics:

  • Hardware requirements

  • Preparing for installation

  • Installing Proxmox on a bare metal node

  • Installing Proxmox on a Debian system

  • A Proxmox subscription

  • Disabling a Proxmox subscription

  • Applying a Proxmox subscription

  • Setting up a Proxmox package repository

  • Seeking support



The Proxmox Virtual Environment (VE) is an open source multinode clustered hypervisor built on Debian Linux, and is able to run on commodity hardware, thus eliminating any vendor lock ins. Proxmox is freely available without any features locked. However, a subscription type license is available to enable a enterprise repository to receive well-tested patches and updates. Subscriptions are recommended for a production-level Proxmox environment.


A hypervisor is a software or firmware that creates a layer between native hardware and an operating system to form a virtual environment to create and run virtual machines. A hypervisor emulates the functions of physical hardware to enable virtual machines to see them as physical resources.

Proxmox can be configured to run a virtual environment of just a few nodes with virtual machines or an environment with thousands of nodes. Supporting both KVM and OpenVZ container-based virtual machines, Proxmox VE is a leading hypervisor today. Proxmox has an extremely vibrant community ready to provide help to any free Proxmox users. Also, the expert technical support team of Proxmox is equally capable of handling all corporate users with their mission critical virtual environment.

As mentioned earlier, Proxmox is a multinode environment, meaning that many nodes can form a single cluster where a virtual machine can be moved around to any node within the cluster, thereby allowing a redundant virtual environment. Through a robust Graphical User Interface (GUI), the entire Proxmox cluster can be managed. As of Proxmox VE 3.4, only one cluster is manageable through the GUI.

Here are some of the notable features of the Proxmox VE:

  • It provides a multinode cluster environment for virtualization. No single node acts as a master, thus eliminating single points of failure.

  • It provides High Availability (HA) of virtual machines.

  • It gives centralized web-based management and a single interface to manage an entire cluster.

  • A console can be accessed through secured VNC, SPICE, and HTML5-based noVNC.

  • It provides support for multiple authentication sources, such as local using Pluggable Authentication Module (PAM), Microsoft ADS, and LDAP.

  • A Proxmox cluster file system (pmxcfs) can be used to store configuration files for real-time replication on all nodes using corosync (

  • It provides role-based permission management for objects VMs, storages, nodes, pools, and so on.

  • Unlike SOAP, REST is not a protocol but combination of various standards such as HTTP, JSON, URI and XML. Visit for information on REST based APIs.

  • It provides a built-in powerful firewall for host nodes and virtual machines.

  • It provides migration of VMs between physical hosts with or without shared storage.

  • It supports mainstream storage types, such as Ceph, NFS, ZFS, Gluster, and iSCSI.

  • It provides cluster-wide logging.


Hardware requirements

Proxmox can be installed on just about any commodity hardware, but in order to have a stable platform, some attention is required when selecting nodes. A setup selected for Proxmox learning can be underpowered and less expansive. While this setup is acceptable for very small environments and home usage, it will not be adequate for production-level setups where stability and redundancy is the minimum requirement.

Minimum hardware

Here are the minimum requirements to install Proxmox on a hardware node:

  • Processor: Intel or AMD 64-bit

  • Motherboard: Intel VT or AMD-V capable (not required for OpenVZ)

  • Memory: 1 GB RAM

  • Network Interface Card (NIC): 1

Based on the minimum requirement of Proxmox, here are examples of Intel and AMD-based hardware configurations that are suitable for learning:

Processor →

Requirement ↓




i3-4160 3.0 GHz

FX-4300 3.8 GHz


Asus B85M-D Plus 6 x SATA

Asus M5A78L-M/USB3 6 x SATA


Corsair 1 x 4 GB DDR3

Corsair 1 x 4 GB DDR3

Recommended hardware

To have a stable performing Proxmox cluster, here are the hardware configurations that are recommended:

  • An Intel or AMD 64-bit processor

  • An Intel VT or AMD-V capable Dual or Quad CPU motherboard

  • 16 GB RAM memory

  • Two Network Interface Card (NIC)

  • RAID controller with Battery Backup Unit (BBU)

  • Solid State Drives (SSD) for operating system or SSD for shared storage node

  • Fencing hardware only if HA is needed


    For more details on fencing and HA visit

The following table lists the configurations of a server node that can be used in a production environment:




Intel Xeon E5-2630 v3 2.4 GHz


Intel S2600CP2 Dual LGA2011


Kingston 16 GB DDR3 Registered ECC

Power supply


Note that the example configurations are for reference only. Your requirement will vary depending on the work load and expected performance. Adjust the hardware requirements accordingly by keeping in mind that in a hypervisor more core counts will increase performance of virtual machines rather than higher clock counts of a processor that is used. With a higher core count, more threads can be distributed among processors.


It is worth mentioning here that it is better to select a CPU with a higher cache amount for a node with large amount of memory to minimize the main memory access and maximize the performance for each CPU cores.

Proxmox is a clustered hypervisor. In order to set up a cluster, a minimum of two nodes are required. For the purpose of following through this book, when selecting hardware, be sure to have enough components to set up two nodes.


For more details on Proxmox, please visit

Although a Proxmox cluster can be set up with just two nodes, a minimum of three nodes are recommended for a production environment.


In a cluster, a quorum is established with a minimum of three votes. Each node is counted as single vote. The cluster health depends on this democratic system where the majority vote wins. So, in a two node cluster, when one node fails the other node can only cast one vote, creating an unresolved situation. With a three node cluster, when one node fails, the total vote from the remaining nodes is two out of possible three votes. Thus, the cluster operation continues. By any means, a two node cluster is not recommended for a production cluster. However, it is still possible to create using instructions by visiting


Preparing for installation

Once the necessary hardware is assembled, in this recipe, we are going to see how to prepare ourselves before installing Proxmox.

Getting ready

We are going to use the Proxmox VE installation disk to install Proxmox. First, we need to prepare a disk with the Proxmox installation ISO image.

How to do it…

Here are the steps to install Proxmox:

  1. Download the Proxmox ISO image by visiting

  2. Use a burning software to copy the ISO image on to a DVD disk.

  3. Boot the physical node from the disk prepared from ISO to start the installation process.

There's more…

Some nodes may not have a ROM drive available to install from a disk. In such cases, it is possible to install Proxmox by transferring an ISO image onto a USB flash drive. Note that in order to boot from USB media, your motherboard must support the USB boot option. Check from the motherboard BIOS before proceeding to the following steps. If the motherboard does not support USB boot option, it may not also support an external USB ROM Drive. In this case, the best way to proceed is to install an ROM drive in the computer or use a newer motherboard if possible. Here are the instructions to transfer an ISO image to a USB and use it as bootable drive:

Use the following steps to do this on Windows:

  1. Download the Proxmox ISO installation image from

  2. Rename the file extension of the downloaded ISO from .iso to .raw.

  3. Download and install the USB image writing application from


    In some Windows 7 versions, the preceding writer may not work. In such scenarios, download and install the ImageUSB from

  4. Using the preceding applications, copy the ISO image onto a USB drive.

  5. Insert the USB media into the USB port of the physical node and boot from the USB media.

Use the following steps to do this on Linux:

  1. Download the Proxmox ISO installation image from

  2. Insert the USB media into the USB port of the node and find out the device name of the drive using #fdisk. The name should be in the /dev/XXX format.

  3. Use the following command to copy the ISO onto USB media. Use extra caution when using a device name in the following command. The wrong device name will destroy all the data on the device:

    #dd if=<Proxmox ISO file> of=/dev/XXX bs=1M
  4. Reboot the node from the USB media to start the installation.


Installing Proxmox on a bare metal node

The Proxmox installation process is guided by a fully graphical interface through various prompts. In this recipe, we are going to follow through creating our first Proxmox node.

Getting ready

Power up and boot the physical node using the installation disk or the USB media we created in the preceding recipe. The following screenshot is how the screen looks after the boot:

How to do it…

  1. At the main installation window after the booting process press Enter to start the installation.

  2. On the End User License Agreement (EULA) screen click on Agree.

  3. In this step, we have chosen the drive to install the hypervisor on. From Proxmox VE 3.4, a new feature to change the filesystem has been added. After selecting Target Harddisk from the drop-down menu, select Options to open the Filesystem selection dialog box. The following screenshot shows the drive selection screen with the Filesystem dialog box open:

  4. From the Filesystem drop-down menu, select the desired filesystem. Select the desired filesystem size in hdsize. In version 3.3, support for the ext3, ext4, and ZFS filesystems have been added. Add additional information, such as the swapsize, maxroot, minfree, and maxvz values as required or leave them blank to continue with default, then click on Next. The following table shows the functions of these four options. These values are usually alternatives to each other, meaning if values are entered for one of these options, values for other options are automatically calculated:




    This defines the size of a swap partition. There are complex formulas to calculate this value. However, it is generally left as the default value.


    This refers to the maximum size the root partition should be.


    This refers to the minimum free space to allocate for a pve partition.


    This refers to the maximum space to allocate to locally store VMs. The default path for local VM storage is /var/lib/vz.

    The following screenshot shows the filesystem that is supported after clicking on the drop-down menu:

  5. After selecting the drive for installation, we now have to select localization information, such as Country, Time Zone, and Keyboard Layout based on the desired language. Then, click on Next. The following screenshot displays the localization screen:

  6. Type in and confirm the password for the root login. Then, enter an e-mail address where all the Proxmox cluster notifications will go to. Click on Next.

  7. In this step, we need to enter the hostname and network information, such as IP Address, Gateway, DNS Server, and so on. Enter the necessary information for your environment, then click on Next. The following screenshot shows the network information screen:

  8. At this point, all the necessary information has been collected and the main installation has been started. After the installation is completed, eject the installation disk or the USB media, then click on Reboot.

There's more…

Follow steps 1 to 7 to set up the second node. A minimum of two nodes are required to form a Proxmox cluster. More information on cluster configurations is available in Chapter 3, Cluster and VM Management.


Installing Proxmox on a Debian system

Although installation of Proxmox on a native bare metal node is recommended, at times, it may be necessary to install Proxmox on an existing Debian node. In this recipe, we will see how to install the Proxmox VE on Debian. Note that Proxmox can only be installed on 64-bit hardware and a 64-bit operating system.

Getting ready

Install Debian from instructions from the official Debian site at Prior to installing Proxmox on the Debian node, ensure that the hostname can be resolved. Check the hostname using the following command:

#nano /etc/hosts

It should have these entries: localhost.localdomain localhost pmx1 pvelocalhost

If the entries are missing, type in the proper IP address associated with the hostname of the node. The pvelocalhost entry must be present at the end of the line.

How to do it...

Use the following steps to install Proxmox on a Debian system:

  1. Add the Proxmox VE repository in the source list as follows:

    #nano /etc/apt/sources.list
    deb wheezy main contrib
    #PVE repository for installation
    deb wheezy pve
    #PVE security updates
    deb wheezy/updates main contrib


    Due to a bug in the Debian OS, apt-get may display an error of not finding /binary-i386 with a Unable to find expected entry 'pve/binary-i386/packages' error message. The error may occur even though it is a 64-bit Debian installed operating system. In such cases, change the PVE repository entry to the following:

    deb [arch=amd64] wheezy pve
  2. Add the Proxmox VE repository key using the following command:

    #wget –O- | apt-key add –
  3. Update the repository using the following command:

    #apt-get update
  4. Update the operating system using the following command:

    #apt-get dist-upgrade
  5. Install the Proxmox VE Kernel:

    #apt-get install pve-firmware pve-kernel-2.6.32-37-pve
  6. Install Kernel headers:

    #apt-get install pve-header-2.6.32-37-pve
  7. Be sure to select the Proxmox VE Kernel on the boot loader grub2.

  8. Reboot the node to activate the new Proxmox VE Kernel.

  9. Verify the running kernel to ensure that the proper Proxmox VE Kernel is loaded:

    #uname –a
  10. Check the grub2 config using following command:

  11. Install the Proxmox VE packages:

    #apt-get install Proxmox-ve-2.6.32 ntp ssh lvm2 postfix ksm- control-daemon vzprocps open-iscsi bootlogd
  12. During installation, accept suggestions to remove Exim and set up Postfix. Exim can be installed later if required.

  13. Reboot the node after the Proxmox VE installation is done.

There's more…

Follow steps 1 to 13 to set up a second node. A minimum of two nodes are required to form a Proxmox cluster. We will take a good look at the Proxmox GUI in Chapter 2, Getting to know the Proxmox GUI.


A Proxmox subscription

The Proxmox VE itself is free. There are absolutely no costs involved in simply downloading the ISO image and installing a fully-functional Proxmox cluster without paying for a license or subscription. However, there is a Proxmox subscription option available to support the project, which enables enterprise repository. There are no feature differences between the Free and Subscription versions of Proxmox. Both are exactly the same. The difference is between the type of releases, updates, and patches they receive.


Packages from the enterprise repository go through an extra layer of scrutiny and testing. Thus, a subscription is recommended for a production-level Proxmox VE.

This not to be confused with the fact that a Proxmox cluster is built with no subscription and is not stable enough to use. New packages are usually released through a no-subscription repository first. Same packages are pushed through the Enterprise repository at a later time. This delay allows the Proxmox team to pin point and fix any lingering bugs or defects in the package. In a cluster running mission critical virtual machines may face unacceptable downtime due to the bug, which will not affect a smaller environment that is able to sustain downtime.

There are four levels of subscription at various price points, such as COMMUNITY, BASIC, STANDARD, and PREMIUM. For more information on the Proxmox subscription, visit


Disabling a Proxmox subscription

A fresh installation of the Proxmox VE without subscriptions will display a message upon login to Proxmox GUI, as shown in the following screenshot:

This is because an enterprise repository is enabled by default. If you decide not to get any Proxmox subscriptions and continue using the fully Free version, you will have to disable the enterprise repository.

Getting ready

Login to the Proxmox node through a console or SSH. An enterprise repository can only be disabled through CLI.

How to do it…

The enterprise repository is listed in /etc/apt/sources.list.d/pve-enterprise.list. We have to comment out the line to disable the repository:

# nano /etc/apt/sources.list.d/pve-enterprise.list
#deb wheezy pve-enterprise

Add the Proxmox No-Subscription repository as follows:

# nano /etc/apt/sources.list
deb wheezy pve-no-subscription

How it works…

No restart of services or reboot is required to apply the changes. Log out from the Proxmox GUI and log back in to see if the changes were applied correctly. The "no subscription" message box should not pop-up after login. Subscriptions can be managed from the Proxmox GUI under the Subscription tab. Here is a screenshot of a nonsubscribed Proxmox node:


Applying a Proxmox subscription

If you choose to get a Proxmox subscription of any type, a subscription key must be uploaded into the Proxmox node to activate it. The subscription key is sent by e-mail after successful payment. The key can be uploaded through the Proxmox GUI.

Getting ready

Login to the Proxmox GUI by accessing the link through a browser and enter the admin login credentials. The default username is root and the password is the same as was entered during installation: https://<pmx_node_IP>:8006.

How to do it…

  1. After logging into the Proxmox GUI, click on the node that the subscription is going to be applied to.

  2. Click on the Subscription tab.

  3. Click on the Upload Subscription Key button to open the key dialog box, as shown in the following screenshot:

  4. Copy/paste the subscription key from the e-mail received from Proxmox in the key box, then click on Ok.

  5. Click on Check. At this moment, the key will be verified with Proxmox to check the validity. After the activation is completed, it should look like the following screenshot:

There's more…

If a wrong subscription key has been entered, then the display should resemble the following screenshot:

In such cases, simply upload the correct key to activate again. You may also see the key invalid notice when the key has already been activated for another node, which is no longer in service. In such cases, contact Proxmox or other third-party license providers that the license was purchased from to request reissuing of the same license. Then, simply click on the Check button to reactivate the key. Each key is hard coded to each server ID. The key needs to be reissued by Proxmox in order to use it.


Please keep in mind that once a key is reissued, it will no longer work on the previous node it was activated for. If you have asked for reissue a license by mistake, request to reissue again and click on the Check button on the previous node.


Setting up a Proxmox package repository

The Proxmox VE offers three main repositories:






This is primarily used in the production of the Proxmox node. Packages in this repository go through additional scrutiny, bug fixes, and testing.


Not required

This is used in learning, training, and home Proxmox cluster nodes. Packages in this repository go through initial bug fixes and are stable enough to be referred as the final release.


Not required

This is used for the testing and development of Proxmox only. Packages in this repository are usually the very latest and are still going through final phases of the release cycle, such as beta testing and release candidate. Packages in this repository may contain a number of bugs and issues. Users of this repository are encouraged to share bug reports with developers.

The location and content of the Enterprise Repository source file is as follows:

#cat /etc/apt/sources.list.d/pve-enterprise.list
deb wheezy pve-enterprise

The location and content of the No-Subscription Repository source file is as follows:

#cat /etc/apt/sources.list
deb wheezy main contrib
deb wheezy pve-no-subscription
deb / wheezy/updates main contrib

Proxmox offers a Test Repository to allow users to try out new features or packages. As the name implies, the Test Repository should only be used for testing. All the new features of Proxmox are released in the Test Repository before they are available for the No-Subscription and Enterprise repositories. Packages in the Test Repository are not well-tested and may contain bugs. For this reason, the repository should never be used in a production-level cluster. The Test Repository is not enabled by default.

Getting ready

Log in to the Proxmox node through a console or SSH. The repository source file needs to be edited through CLI to enable the Test Repository.

How to do it…

Use the following steps to set up the Proxmox package repository:

  1. Open the repository source file using any favorite text editor:

    #nano /etc/apt/sources.list
  2. Make the necessary changes to make the entries look similar to the following:

    deb wheezy main contrib
    deb wheezy pvetest
    deb http://security wheezy/updates main contrib
  3. Save the file and exit the editor.

  4. Run the following command to update the repositories:

    #apt-get update

How it works…

Usually, the announcement of the availability of a new package is made on the official Proxmox forum ( The name of the package or the version information is included in the announcement. If you want to find information on a package, simply ask for it on the forum. Once you have the information, simply run the apt-get command to install it through CLI:

#apt-get install <package_name>

There's more…

Besides Enterprise, No-Subscription, and Test repositories there are two repositories that are outdated and are no longer supported or updated:

  • Outdated stable repository (pve): This repository has stopped receiving updates after the initial release of Proxmox VE 3.1:

    deb wheezy main contrib
    deb wheezy pve
    deb http://security wheezy/updates main contrib

    If you're upgrading from Proxmox VE 2.x to 3.x, the second entry in /etc/apt/sources.list needs to be changed from wheezy pve to wheezy pve-no-subscription.

  • Outdated Proxmox VE 2.x stable repository (squeeze pve): In order to have a stable Proxmox node, it is highly recommended to upgrade and update to the latest stable release of the Proxmox VE.

    deb squeeze main contrib
    deb squeeze pve
    deb squeeze/updates main contrib

Seeking support

As mentioned in the beginning of this chapter, Proxmox has a vibrant community of users ready to provide help to anybody in need. There are several ways that a new Proxmox user can seek help and support to learn and extend knowledge of the Proxmox VE:

Help and support






Customer portal and tickets

Requires subscription

Proxmox Mailing list


Proxmox Bug tracker


The forum

The official forum of Proxmox is a gathering place for newbies or experts alike to receive or provide help from the Proxmox community. It is also a vast information resource of past issues and solutions to many problems. A few of the Proxmox staff also spend a significant amount of time in this forum to offer help outside their job responsibilities. There is an array of Proxmox experts willing to help out without expecting anything in return. This is the only place for Free and Community-level subscriptions for Proxmox users to seek support. Visit to access the official Proxmox forum.


The Proxmox Wiki site contains a wealth of information on installation, configuration, and the management of Proxmox clusters. All documentation is written in a very easy to understand form with as many illustrations as possible. The use of the Proxmox Wiki page is completely free and requires no registration. Visit for the official Proxmox documentation.

Customer portals and tickets

A customer portal is only available for basic, standard, and premium subscription-level users. Supports tickets must be opened through a customer portal by visiting to receive technical support directly from the Proxmox staff.

Depending on the subscription level, a number of tickets that can be opened varies. Responses are guaranteed within one business day. See the different support levels for subscriptions by visiting

Proxmox mailing lists

There are two different mailing lists available from Proxmox:

The Proxmox bug tracker

The Proxmox bug tracker is available to submit and review any bugs found in Proxmox during day-to day-use ( It can also be used to submit new feature requests. Not all features get added to Proxmox, but developers of Proxmox are very forward thinking and apply time appropriate and main stream features to make the Proxmox VE even better.


The Proxmox bug tracker is NOT to be used to ask technical questions or to seek any sort of support.

About the Author

  • Wasim Ahmed

    Wasim Ahmed, born in Bangladesh and now a citizen of Canada, is a veteran of the IT world. He first came into close contact with computers in 1992 and never looked back. Wasim has a deep understanding of networks, virtualization, big data storage, and network security.

    By profession, Wasim is the CEO of a global IT support and cloud service provider based in Calgary, Alberta. He serves many companies and organizations through his company on a daily basis. Wasim's strength comes from his experience, which comes from learning and serving continually. Wasim strives to find the most effective solution at the most competitive price. He has built over 20 enterprise production virtual infrastructures using Proxmox and the Ceph storage system.

    Wasim and his team are notorious for not simply accepting a technology based on its description alone, but putting it through rigorous testing to check its validity. Any new technology that his company provides goes through months of continuous testing before it is accepted. Proxmox made the cut superbly.

    Browse publications by this author
Proxmox Cookbook
Unlock this book and the full library FREE for 7 days
Start now