KVM Virtualization Cookbook

Depending on your Linux distribution, the package name and installation commands will differ. You can use your system's package manager, such as apt, dnf, or yum to search for any packages containing the QEMU string and get familiar with what is available for your particular Linux variant. The source code can be downloaded from the official QEMU project website at http://www.qemu-project.org/download/#source.

Perform the following steps to install QEMU from packages on Ubuntu/Debian and RHEL/CentOS distributions:

root@kvm:~# apt-get update

root@kvm:~# apt-get install -y qemu

root@kvm:~# yum install qemu-kvm

To install from source, execute the following:

root@kvm:~#cd /usr/src && wget 
http://download.qemu-project.org/qemu-2.8.0.tar.xz

root@kvm:/usr/src# tar xvJf qemu-2.8.0.tar.xz && cd qemu-        2.8.0

root@kvm:/usr/src/qemu-2.8.0# ./configure  
root@kvm:/usr/src/qemu-2.8.0# make && make install

Installing QEMU is quite trivial, as we just saw. Let's have a look at what the QEMU metapackage installed on Ubuntu looks like:

root@kvm:~# dpkg --list | grep qemu
ii ipxe-qemu 1.0.0+git-20150424.a25a16d-1ubuntu1 all PXE boot firmware - ROM images for qemu
ii qemu 1:2.5+dfsg-5ubuntu10.8 amd64 fast processor emulator
ii qemu-block-extra:amd64 1:2.5+dfsg-5ubuntu10.8 amd64 extra block backend modules for qemu-system and qemu-utils
ii qemu-slof 20151103+dfsg-1ubuntu1 all Slimline Open Firmware -- QEMU PowerPC version
ii qemu-system 1:2.5+dfsg-5ubuntu10.8 amd64 QEMU full system emulation binaries
ii qemu-system-arm 1:2.5+dfsg-5ubuntu10.8 amd64 QEMU full system emulation binaries (arm)
ii qemu-system-common 1:2.5+dfsg-5ubuntu10.8 amd64 QEMU full system emulation binaries (common files)
ii qemu-system-mips 1:2.5+dfsg-5ubuntu10.8 amd64 QEMU full system emulation binaries (mips)
ii qemu-system-misc 1:2.5+dfsg-5ubuntu10.8 amd64 QEMU full system emulation binaries (miscelaneous)
ii qemu-system-ppc 1:2.5+dfsg-5ubuntu10.8 amd64 QEMU full system emulation binaries (ppc)
ii qemu-system-sparc 1:2.5+dfsg-5ubuntu10.8 amd64 QEMU full system emulation binaries (sparc)
ii qemu-system-x86 1:2.5+dfsg-5ubuntu10.8 amd64 QEMU full system emulation binaries (x86)
ii qemu-user 1:2.5+dfsg-5ubuntu10.8 amd64 QEMU user mode emulation binaries
ii qemu-user-binfmt 1:2.5+dfsg-5ubuntu10.8 amd64 QEMU user mode binfmt registration for qemu-user
ii qemu-utils 1:2.5+dfsg-5ubuntu10.8 amd64 QEMU utilities
root@kvm:~#

From the preceding output, we can see that there are few packages involved. If you are interested, you can read the individual description to get more familiar with what each package provides.

It's worth mentioning that all binaries provided from the earlier-mentioned packages start with the prefix QEMU. You can use tab completion to see the list of available executables:

root@kvm:~# qemu-
qemu-aarch64 qemu-io qemu-mips64el qemu-ppc64 qemu-sparc32plus qemu-system-lm32 qemu-system-mipsel qemu-system-sh4 qemu-system-xtensa
qemu-alpha qemu-m68k qemu-mipsel qemu-ppc64abi32 qemu-sparc64 qemu-system-m68k qemu-system-moxie qemu-system-sh4eb qemu-system-xtensaeb
qemu-arm qemu-make-debian-root qemu-mipsn32 qemu-ppc64le qemu-system-aarch64 qemu-system-microblaze qemu-system-or32 qemu-system-sparc qemu-tilegx
qemu-armeb qemu-microblaze qemu-mipsn32el qemu-s390x qemu-system-alpha qemu-system-microblazeel qemu-system-ppc qemu-system-sparc64 qemu-unicore32
qemu-cris qemu-microblazeel qemu-nbd qemu-sh4 qemu-system-arm qemu-system-mips qemu-system-ppc64 qemu-system-tricore qemu-x86_64
qemu-i386 qemu-mips qemu-or32 qemu-sh4eb qemu-system-cris qemu-system-mips64 qemu-system-ppc64le qemu-system-unicore32
qemu-img qemu-mips64 qemu-ppc qemu-sparc qemu-system-i386 qemu-system-mips64el qemu-system-ppcemb qemu-system-x86_64
root@kvm:~#

We can see that there's a single executable for each CPU architecture type that can be emulated.

To use this recipe, we need to have the qemu-img utility installed. If you followed the steps in the first recipe, you should have that covered. To check what image types are supported on your Linux distribution, run the following command:

root@kvm:~# qemu-img -h | grep Supported
Supported formats: bochs vvfat rbd vpc parallels tftp ftp ftps raw https qcow dmg http qcow2 quorum null-aio cloop vdi iscsi null-co vhdx blkverify file vmdk host_cdrom blkdebug host_device sheepdog qed nbd
root@kvm:~#

From the preceding output, we can see that there are many supported images on the test system that we are using. Make sure that your QEMU version supports the raw image type, as it's the default and that is what we are going to use in this recipe. One of the most commonly used image type is qcow2, which supports copy on write, compression, encryption, and snapshotting. We are going to leverage that in later recipes.

Perform the following steps to create a blank raw image of a specified size and to verify that the file was created on the host:

root@kvm:~# qemu-img create -f raw debian.img 10G    
Formatting 'debian.img', fmt=raw size=10737418240    
root@kvm:~#

root@kvm:~# ls -lah debian.img    
-rw-r--r-- 1 root root 10G Feb 10 16:58 debian.img    
 root@kvm:~#

root@kvm:~# file -s debian.img    
debian.img: data    
root@kvm:~#

root@kvm:~# qemu-img info debian.img
image: debian.img
file format: raw
virtual size: 10G (10737418240 bytes)
disk size: 0
root@kvm:~#

The qemu-img utility allows us to create, convert, and modify guest images.

In step 1, we used the -f flag specifying the image format; in this case, raw, the name of the image to be created and the size in gigabytes.

In step 4, we used the info subcommand to gather additional information about the existing image. Note how the disk size is showing as currently being zero. This is due to the fact that this is a blank image, not containing a filesystem. We are going to create one in the next recipe.

In this recipe, we listed the supported disk image formats by QEMU. The following is a brief description of the most common types that you might encounter:

In this book, we are going to use the raw and qcow2 disk formats, as they provide the best performance and toolset for running and manipulating them.

For this recipe, we are going to use the following tools:

Most Linux distributions should already have the tools installed. If that's not the case, consult your distribution's documentation on how to install them.

Perform the following steps outlined to partition and create a filesystem on the blank image:

    root@kvm:~# modprobe nbd
    root@kvm:~#

root@kvm:~# qemu-nbd --format=raw --connect=/dev/nbd0          debian.img  
root@kvm:~#

root@kvm:~# sfdisk /dev/nbd0 << EOF    
>,1024,82    
>;    
>EOF    
Checking that no-one is using this disk right now ...
OK
Disk /dev/nbd0: cannot get geometry

Disk /dev/nbd0: 1305 cylinders, 255 heads, 63 sectors/track

sfdisk: ERROR: sector 0 does not have an msdos signature
 /dev/nbd0: unrecognized partition table type
Old situation:
No partitions found
New situation:
Units = cylinders of 8225280 bytes, blocks of 1024 bytes, counting from 0

 Device Boot Start End #cyls #blocks Id System
/dev/nbd0p1 0+ 1023 1024- 8225279+ 82 Linux swap / Solaris
/dev/nbd0p2 1024 1304 281 2257132+ 83 Linux
/dev/nbd0p3 0 - 0 0 0 Empty
/dev/nbd0p4 0 - 0 0 0 Empty
Warning: no primary partition is marked bootable (active)
This does not matter for LILO, but the DOS MBR will not boot this disk.
Successfully wrote the new partition table

Re-reading the partition table ...

If you created or changed a DOS partition, /dev/foo7, say, then use dd(1)
to zero the first 512 bytes: dd if=/dev/zero of=/dev/foo7 bs=512 count=1
(See fdisk(8).)
root@kvm:~#

root@kvm:~# ls -la /dev/nbd0*    
brw-rw---- 1 root disk 43, 0 Feb 10 18:24 /dev/nbd0    
brw-rw---- 1 root disk 43, 1 Feb 10 18:24 /dev/nbd0p1    
brw-rw---- 1 root disk 43, 2 Feb 10 18:24 /dev/nbd0p2    
root@kvm:~#

root@kvm:~# mkswap /dev/nbd0p1    
Setting up swapspace version 1, size = 508 KiB (520192 bytes)    
no label, UUID=c246fe39-1bc5-4978-967c-806264771d69    
root@kvm:~#

root@kvm:~# mkfs.ext4 /dev/nbd0p2    
mke2fs 1.42.13 (17-May-2015)    
Discarding device blocks: failed - Input/output error    
Creating filesystem with 2620928 4k blocks and 655360 inodes    
Filesystem UUID: 2ffa23de-579a-45ad-abbc-2a179de67f11    
Superblock backups stored on blocks:    
    32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632    
Allocating group tables: done    
Writing inode tables: done    
Creating journal (32768 blocks): done    
Writing superblocks and filesystem accounting information: done    
root@kvm:~#

We take advantage of the functionality that the nbd kernel module provides by allowing us to associate a raw image file to a block device using the qemu-nbd utility. To get more information about the kernel module run the following code:

root@kvm:~# modinfo nbd
filename: /lib/modules/4.4.0-62-generic/kernel/drivers/block/nbd.ko
license: GPL
description: Network Block Device
srcversion: C67096AF2AE3C738DBE0B7E
depends:
intree: Y
vermagic: 4.4.0-62-generic SMP mod_unload modversions
parm: nbds_max:number of network block devices to initialize (default: 16) (int)
parm: max_part:number of partitions per device (default: 0) (int)
root@kvm:~#

We can examine the block device metadata created in step 2 by running the following command:

root@kvm:~# file -s /dev/nbd0
/dev/nbd0: x86 boot sector
root@kvm:~#

After creating the two new partitions in step 3, the type of the image file has changed. Let's examine it again:

root@kvm:~# file -s debian.img
debian.img: x86 boot sector
root@kvm:~#

Now that we have an image file that contains two partitions and a filesystem, we can proceed with installing the guest OS in the next recipe.

We are going to need the following in order to complete this recipe:

To ensure that the swap and root block devices are still present on the system, run the following:

root@kvm:~# ls -la /dev/nbd0*
brw-rw---- 1 root disk 43, 0 Feb 10 18:24 /dev/nbd0
brw-rw---- 1 root disk 43, 1 Feb 10 18:24 /dev/nbd0p1
brw-rw---- 1 root disk 43, 2 Feb 10 18:24 /dev/nbd0p2
root@kvm:~#

If that's not the case, please refer to the Preparing images for OS installation with qemu-nbd recipe on how to associate the raw image with the /deb/nbd0 block device.

To install the debootstrap utility, if not already present on your system, execute the following code:

root@kvm:~# apt install -y debootstrap
...
Setting up debootstrap (1.0.78+nmu1ubuntu1.2) ...
root@kvm:~#

Follow these steps outlined to install a new Debian Linux distribution on the raw image:

root@kvm:~# mount /dev/nbd0p2 /mnt/    
root@kvm:~# mount | grep mnt    
/dev/nbd0p2 on /mnt type ext4 (rw) 
root@kvm:~#

root@kvm:~# debootstrap --arch=amd64 --include="openssh-server vim" stable /mnt/ http://httpredir.debian.org/debian/    
...    
I: Base system installed successfully.    
root@kvm:~#

root@kvm:~# ls -lah /mnt/    
total 100K    drwxr-xr-x 22 root root 4.0K Feb 10 17:19 .    
drwxr-xr-x 23 root root 4.0K Feb 10 15:29 ..    
drwxr-xr-x 2 root root 4.0K Feb 10 17:19 bin    
drwxr-xr-x 2 root root 4.0K Dec 28 17:42 boot    
drwxr-xr-x 4 root root 4.0K Feb 10 17:18 dev    
drwxr-xr-x 55 root root 4.0K Feb 10 17:19 etc    
drwxr-xr-x 2 root root 4.0K Dec 28 17:42 home    
drwxr-xr-x 12 root root 4.0K Feb 10 17:19 lib    
drwxr-xr-x 2 root root 4.0K Feb 10 17:18 lib64    
drwx------ 2 root root 16K Feb 10 17:06 lost+found    
drwxr-xr-x 2 root root 4.0K Feb 10 17:18 media    
drwxr-xr-x 2 root root 4.0K Feb 10 17:18 mnt    
drwxr-xr-x 2 root root 4.0K Feb 10 17:18 opt    
drwxr-xr-x 2 root root 4.0K Dec 28 17:42 proc    
drwx------ 2 root root 4.0K Feb 10 17:18 root    
drwxr-xr-x 4 root root 4.0K Feb 10 17:19 run    
drwxr-xr-x 2 root root 4.0K Feb 10 17:19 sbin    
drwxr-xr-x 2 root root 4.0K Feb 10 17:18 srv    
drwxr-xr-x 2 root root 4.0K Apr 6 2015 sys    
drwxrwxrwt 2 root root 4.0K Feb 10 17:18 tmp    
drwxr-xr-x 10 root root 4.0K Feb 10 17:18 usr    
drwxr-xr-x 11 root root 4.0K Feb 10 17:18 var    
root@kvm:~#

root@kvm:~# mount --bind /dev/ /mnt/dev    
root@kvm:~#

root@kvm:~# ls -la /mnt/dev/ | grep nbd0    
brw-rw---- 1 root disk 43, 0 Feb 10 18:24 nbd0    
brw-rw---- 1 root disk 43, 1 Feb 10 18:26 nbd0p1    
brw-rw---- 1 root disk 43, 2 Feb 10 18:26 nbd0p2    
root@kvm:~#

root@kvm:~# chroot /mnt/    
root@kvm:/# pwd
/    
root@kvm:/#

root@kvm:/# cat /etc/debian_version
8.7    
root@kvm:/#

root@kvm:/# mount -t proc none /proc    
root@kvm:/# mount -t sysfs none /sys    
root@kvm:/#

root@kvm:/# apt-get install -y --force-yes linux-image-amd64 grub2

root@kvm:/# grub-install /dev/nbd0 --force   
Installing for i386-pc platform.
grub-install: warning: this msdos-style partition label has no post-MBR gap; embedding won't be possible.
grub-install: warning: Embedding is not possible. GRUB can only be installed in this setup by using blocklists. However, blocklists are UNRELIABLE and their use is discouraged..
Installation finished. No error reported.    
root@kvm:/#

root@kvm:/# update-grub2    
Generating grub configuration file ...
Found linux image: /boot/vmlinuz-3.16.0-4-amd64
Found initrd image: /boot/initrd.img-3.16.0-4-amd64
done   
root@kvm:/#

root@kvm:/# passwd    
Enter new UNIX password:    
Retype new UNIX password:    
passwd: password updated successfully    
root@kvm:/#

root@kvm:/# echo "pts/0" >> /etc/securetty    
root@kvm:/#

root@kvm:/# systemctl set-default multi-user.target    
Created symlink from /etc/systemd/system/default.target to /lib/systemd/system/multi-user.target.    
root@kvm:/#

root@kvm:/# echo "/dev/sda2 / ext4 defaults,discard 0 0" > /etc/fstab

root@kvm:/# umount /proc/ /sys/ /dev/

root@kvm:/# exit    
exit    
root@kvm:~#

root@kvm:~# grub-install /dev/nbd0 --root-directory=/mnt --modules="biosdisk part_msdos" --force 
Installing for i386-pc platform.
grub-install: warning: this msdos-style partition label has no post-MBR gap; embedding won't be possible.
grub-install: warning: Embedding is not possible. GRUB can only be installed in this setup by using blocklists. However, blocklists are UNRELIABLE and their use is discouraged..
Installation finished. No error reported.    
root@kvm:~#

root@kvm:~# sed -i 's/nbd0p2/sda2/g' /mnt/boot/grub/grub.cfg    
root@kvm:~#

root@kvm:~# umount /mnt    
root@kvm:~#

root@kvm:~# qemu-nbd --disconnect /dev/nbd0    
/dev/nbd0 disconnected    
root@kvm:~#

A lot has happened in the previous section, so let's step through the commands and talk a little bit more about what exactly was performed and why.

In step 1, we mounted the root partition we created earlier on the /dev/nbd0p2 device to /mnt, so we can use it. Once mounted, in step 2, we installed an entire Debian distribution on that device using the mount-point as the target.

In order to install the GRUB boot loader on the root partition of the image, we bind and mounted the /dev directory from the host filesystem to the image filesystem in /mnt in step 4.

Then in step 6, we used the chroot tool to change our directory namespace to be /mnt, so we can perform operations, as we are directly on the new OS.

In step 8, we mounted the proc and sysfs virtual filesystems inside the image because the GRUB bootloader tool expect them.

In step 9, we proceeded to install the kernel source and GRUB tools in preparation of installing the bootloader on the boot partition and in step 10 we installed the bootloader.

In step 11, the GRUB configuration files were generated and the boot ramdisk image was updated.

In steps 12, 13, and 14, we changed the root password and ensured we get access to the pseudo Terminal, so we can log into the VM later and change the run-level from the default graphical interface to the multiuser.

Since the fstab file is empty right after installing the Debian OS on the image, we have to add the root mount point, or the VM will not be able to start. This was accomplished in step 15.

In steps 16 and 17, we performed some cleaning up by unmounting the filesystems we mounted earlier and exited the chroot environment.

Back on the host filesystem in step 18, we installed GRUB on the nbd0 device by specifying the mounted location of the image.

In step 19, we updated the GRUB config device name to be sda2 because this is the name that will appear inside the virtual machine once we start it. The nbd0p2 name is only present while we have the association between the raw image and the network block device on the host OS. From the VM perspective, the second partition inside the image we created by is named sda2 by default.

And finally, in steps 20 and 21, we performed some cleaning by removing the mount point and disassociating the raw image from the network block device nbd0.

For this recipe, we are going to use the following tools:

Most of the utilities should already be installed on Ubuntu with the exception of kpartx. To install it, run the following:

root@kvm:~# apt install kpartx

The next steps demonstrate how to add additional space to the raw image we created earlier, extend the root partition, and resize the filesystem. By the end of this recipe, the original raw image filesystem size should have changed from 10G to 20G.

root@kvm:~# qemu-img info debian.img    
image: debian.img
file format: raw
virtual size: 10G (10737418240 bytes)
disk size: 848M    
root@kvm:~#

root@kvm:~# qemu-img resize -f raw debian.img +10GB    
Image resized.    
root@kvm:~#

root@kvm:~# qemu-img info debian.img    
image: debian.img
file format: raw
virtual size: 20G (21474836480 bytes)
disk size: 848M   
root@kvm:~#

root@kvm:~# losetup -f 
/dev/loop0    
root@kvm:~#

root@kvm:~# losetup /dev/loop1 debian.img    
root@kvm:~#

root@kvm:~# kpartx -av /dev/loop1    
add map loop1p1 (252:0): 0 1024 linear 7:1 2048    
add map loop1p2 (252:1): 0 20967424 linear 7:1 4096
root@kvm:~#

root@kvm:~# ls -la /dev/mapper    
total 0    
drwxr-xr-x 2 root root 100 Mar 9 19:10 .    
drwxr-xr-x 20 root root 4760 Mar 9 19:10 ..    
crw------- 1 root root 10, 236 Feb 10 23:25 control    
lrwxrwxrwx 1 root root 7 Mar 9 19:10 loop1p1    
lrwxrwxrwx 1 root root 7 Mar 9 19:10 loop1p2    
root@kvm:~#

root@kvm:~# tune2fs -l /dev/mapper/loop1p2    
tune2fs 1.42.13 (17-May-2015)    
Filesystem volume name: <none>    
Last mounted on: /    
Filesystem UUID: 96a73752-489a-435c-8aa0-8c5d1aba3e5f    
Filesystem magic number: 0xEF53    
Filesystem revision #: 1 (dynamic)    
Filesystem features: has_journal ext_attr resize_inode dir_index filetype needs_recovery extent flex_bg sparse_super            large_file huge_file uninit_bg dir_nlink extra_isize    Filesystem flags: signed_directory_hash    
Default mount options: user_xattr acl    
Filesystem state: clean    
Errors behavior: Continue    
Filesystem OS type: Linux    
Inode count: 655360    
Block count: 2620928    
Reserved block count: 131046    
Free blocks: 2362078    
Free inodes: 634148    
First block: 0    
Block size: 4096    
Fragment size: 4096    
Reserved GDT blocks: 639    
Blocks per group: 32768    
Fragments per group: 32768    
Inodes per group: 8192    
Inode blocks per group: 512    
Flex block group size: 16    
Filesystem created: Fri Feb 10 23:29:01 2017    
Last mount time: Thu Mar 9 19:09:25 2017    
Last write time: Thu Mar 9 19:08:23 2017    
Mount count: 12    
Maximum mount count: -1    
Last checked: Fri Feb 10 23:29:01 2017    
Check interval: 0 (<none>)    
Lifetime writes: 1621 MB    
Reserved blocks uid: 0 (user root)    
Reserved blocks gid: 0 (group root)    
First inode: 11    
Inode size: 256    
Required extra isize: 28    
Desired extra isize: 28    
Journal inode: 8    
Default directory hash: half_md4    
Directory Hash Seed: f101cccc-944e-4773-8644-91ebf4bd4f2d    
Journal backup: inode blocks    
root@kvm:~#

root@kvm:~# e2fsck /dev/mapper/loop1p2    
e2fsck 1.42.13 (17-May-2015)    
/dev/mapper/loop1p2: recovering journal    Setting free blocks count to 2362045 (was 2362078)    /dev/mapper/loop1p2: clean, 21212/655360 files, 258883/2620928 blocks    
root@kvm:~#

root@kvm:~# tune2fs -O ^has_journal /dev/mapper/loop1p2    
tune2fs 1.42.13 (17-May-2015)    
root@kvm:~#

root@kvm:~# tune2fs -l /dev/mapper/loop1p2 | grep "features"    
Filesystem features: ext_attr resize_inode dir_index filetype extent flex_bg sparse_super large_file huge_file uninit_bg     dir_nlink extra_isize    
root@kvm:~#

root@kvm:~# kpartx -dv /dev/loop1    
del devmap : loop1p2    
del devmap : loop1p1    
root@kvm:~#

root@kvm:~# losetup -d /dev/loop1    
root@kvm:~#

root@kvm:~# qemu-nbd --format=raw --connect=/dev/nbd0 debian.img  root@kvm:~#

root@kvm:~# fdisk /dev/nbd0    

Command (m for help): p

Disk /dev/nbd0: 21.5 GB, 21474836480 bytes
255 heads, 63 sectors/track, 2610 cylinders, total 41943040 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000

 Device Boot Start End Blocks Id System
/dev/nbd0p1 1 16450559 8225279+ 82 Linux swap / Solaris
/dev/nbd0p2 16450560 20964824 2257132+ 83 Linux

Command (m for help): d
Partition number (1-4): 2

Command (m for help): n
Partition type:
 p primary (1 primary, 0 extended, 3 free)
 e extended
Select (default p): p
Partition number (1-4, default 2): 2
First sector (16450560-41943039, default 16450560):
Using default value 16450560
Last sector, +sectors or +size{K,M,G} (16450560-41943039, default 41943039):
Using default value 41943039

Command (m for help): w
The partition table has been altered!

Calling ioctl() to re-read partition table.
Syncing disks.
root@kvm:~#

root@kvm:~# losetup /dev/loop1 debian.img

root@kvm:~# kpartx -av /dev/loop1    
add map loop1p1 (252:2): 0 1024 linear 7:1 2048    
add map loop1p2 (252:3): 0 41938944 linear 7:1 4096    
root@kvm:~#

root@kvm:~# e2fsck -f /dev/mapper/loop1p2    
e2fsck 1.42.13 (17-May-2015)    
Pass 1: Checking inodes, blocks, and sizes    
Pass 2: Checking directory structure    
Pass 3: Checking directory connectivity    
Pass 4: Checking reference counts    
Pass 5: Checking group summary information    
/dev/mapper/loop1p2: 21212/655360 files (0.2% non-contiguous), 226115/2620928 blocks    
root@kvm:~#

root@kvm:~# resize2fs /dev/nbd0p2 
resize2fs 1.42.13 (17-May-2015) 
Resizing the filesystem on /dev/mapper/loop1p2 to 5242368 (4k) blocks. 
The filesystem on /dev/mapper/loop1p2 is now 5242368 (4k) blocks long. 
root@kvm:~#

root@kvm:~# tune2fs -j /dev/mapper/loop1p2    
tune2fs 1.42.13 (17-May-2015)    
Creating journal inode: done    
root@kvm:~#

root@kvm:~# kpartx -dv /dev/loop1    
del devmap : loop1p2    
del devmap : loop1p1    
root@kvm:~# losetup -d /dev/loop1    
root@kvm:~#

Resizing an image for VM can be somewhat involving, as we saw from all the steps in the previous section. Things can get complicated when there are multiple Linux partitions inside the same image, even more so if we are not using Logical Volume Management (LVM). Let's step through all the commands we ran earlier and explain in more details why we ran them and what they do.

In step 1, we confirmed the current size of the image being 10 GB.

In step 2, we added 10 GB at the end of the image and confirm the new image size in step 3.

Recall that the image we built from earlier recipes contains two partitions, swap and root. We need a way to manipulate them individually. Particularly, we would like to allocate the extra space we added in step 2 to the root partition. To do that we need to expose it as a block device that we can easily manipulate with standard disk and filesystem utilities. We accomplished that using the losetup command in step 5, resulting in a mapping between the image and a new block device named /dev/loop1. In step 6, we exposed the individual partitions as two new device mappings. The /dev/mapper/loop1p2 is the root partition that we would like to append the unused disk space to.

Before we can resize the partitioned on the loop device, we need to check the integrity of the filesystem on it, and this is what we did in step 9. Because we are using a journaling filesystem, we need to remove the journal prior to resizing. We do that in step 10 and made sure that the has_journal attribute is not showing after running the tune2fs command in step 11.

Now, we need to work directly on the main block device and not the individual partitions. We remove the mappings in steps 12 and 13 and associated a new block device with the image file using the qemu-nbd command in step 14. The new /dev/nbd0 block device now represents the entire disk of the guest VM and it's a direct mapping to what's inside the raw image. We can use this block device just like any other regular disk, most importantly we can use tools such as fdisk to examine and manipulate the partitions residing on it.

In step 15, we use the fdisk utility to delete the root partition and recreate it. This does not destroy any filesystem data, but changes the metadata, allocating the extra space we added earlier as part of the root partition.

Now that the block device has all the disk space allocated to the root partition, we need to extend the filesystem that is on top of it. We do that by first recreating the individual partition mappings like we did earlier, to expose the root partition directly so that we can yet again manipulate it. We do that in steps 16 and 17.

In steps 18 and 19, we check the integrity of the root file system, then we resize it to the maximum available disk space on the root partition that it resides.

Finally, in step 20, we remove the mappings again. Now the image, the root partition inside the image, and the EXT4 filesystem on top of the Linux partition have been resized to 20 GB.

You can check the new root partition size by starting a new QEMU instance using the image. We are going to do just that in a separate recipe in this chapter.

For this recipe, we are going to need QEMU installed on the host OS. For instructions on how to install QEMU, please refer to the Installing and configuring QEMU recipe from this chapter. We are also going to need the wget utility to download the images from the upstream public repositories.

To obtain Debian Wheezy images for use with QEMU and KVM, perform the following:

root@kvm:~tmp# wget https://people.debian.org/~aurel32/qemu/amd64/debian_wheezy_amd64_standard.qcow2    
--2017-03-09 22:07:20-- 2    Resolving people.debian.org (people.debian.org)... 2001:41c8:1000:21::21:30, 5.153.231.30    Connecting to people.debian.org (people.debian.org)|2001:41c8:1000:21::21:30|:443... connected.    HTTP request sent, awaiting response... 200 OK    Length: 267064832 (255M)    Saving to: ‘debian_wheezy_amd64_standard.qcow2’    debian_wheezy_amd64_standard.qcow2 100% .  [===================================>] 254.69M 35.8MB/s in 8.3s    2017-03-09 22:07:29 (30.9 MB/s) - ‘debian_wheezy_amd64_standard.qcow2’ saved [267064832/267064832]    
root@kvm:~#

root@kvm:~# qemu-img info debian_wheezy_amd64_standard.qcow2    
image: debian_wheezy_amd64_standard.qcow2    
file format: qcow2    
virtual size: 25G (26843545600 bytes)    
disk size: 261M    
cluster_size: 65536    
Format specific information:    
compat: 1.1    
lazy refcounts: false    
refcount bits: 16    
corrupt: false    
root@kvm:~#

To download CentOS images run the following commands:

root@kvm:/tmp# wget         https://cloud.centos.org/centos/7/images/CentOS-7-x86_64-GenericCloud.qcow2    --2017-03-09 22:11:34-- https://cloud.centos.org/centos/7/images/CentOS-7-x86_64-GenericCloud.qcow2    Resolving cloud.centos.org (cloud.centos.org)... 2604:4500::2a8a, 136.243.75.209    Connecting to cloud.centos.org (cloud.centos.org)|2604:4500::2a8a|:443... connected.    HTTP request sent, awaiting response... 200 OK    Length: 1361182720 (1.3G)    Saving to: ‘CentOS-7-x86_64-GenericCloud.qcow2’
CentOS-7-x86_64-GenericCloud.qcow2 100%[=========================================>] 1.27G 22.3MB/s in 54s
2017-03-09 22:12:29 (24.0 MB/s) - ‘CentOS-7-x86_64-GenericCloud.qcow2’ saved [1361182720/1361182720]
FINISHED --2017-03-09 22:12:29--    Total wall clock time: 54s    Downloaded: 1 files, 1.3G in 54s (24.0 MB/s)    
root@kvm:/tmp#

root@kvm:~# qemu-img info CentOS-7-x86_64-GenericCloud.qcow2    
image: CentOS-7-x86_64-GenericCloud.qcow2    
file format: qcow2    
virtual size: 8.0G (8589934592 bytes)    
disk size: 1.3G    
cluster_size: 65536    
Format specific information:    
compat: 0.10    
refcount bits: 16    
root@kvm:~#

There are many public repositories on the Internet that provide images of various types, most commonly qcow2 for use with QEMU/KVM. In the previous section, we used the official CentOS repository to obtain the image and an another one containing prebuilt images for Debian.

Both images are in the qcow2 format, as we confirmed in step 2.

So far, we've only seen how to build, examine, manipulate, and download images. In the next recipe, we are going to focus on how to actually use the images to start QEMU/KVM instances.

Examine the following links to get more information about what prebuilt images are available for the listed distributions:

To complete this recipe, you will need the following:

Let's have a look at what CPU architectures QEMU supports on the host system:

root@kvm:~# qemu-system-x86_64 --cpu help
x86 qemu64 QEMU Virtual CPU version 2.5+
x86 phenom AMD Phenom(tm) 9550 Quad-Core Processor
x86 core2duo Intel(R) Core(TM)2 Duo CPU T7700 @ 2.40GHz
x86 kvm64 Common KVM processor
x86 qemu32 QEMU Virtual CPU version 2.5+
x86 kvm32 Common 32-bit KVM processor
x86 coreduo Genuine Intel(R) CPU T2600 @ 2.16GHz
x86 486
x86 pentium
x86 pentium2
x86 pentium3
x86 athlon QEMU Virtual CPU version 2.5+
x86 n270 Intel(R) Atom(TM) CPU N270 @ 1.60GHz
x86 Conroe Intel Celeron_4x0 (Conroe/Merom Class Core 2)
x86 Penryn Intel Core 2 Duo P9xxx (Penryn Class Core 2)
x86 Nehalem Intel Core i7 9xx (Nehalem Class Core i7)
x86 Westmere Westmere E56xx/L56xx/X56xx (Nehalem-C)
x86 SandyBridge Intel Xeon E312xx (Sandy Bridge)
x86 IvyBridge Intel Xeon E3-12xx v2 (Ivy Bridge)
x86 Haswell-noTSX Intel Core Processor (Haswell, no TSX)
x86 Haswell Intel Core Processor (Haswell)
x86 Broadwell-noTSX Intel Core Processor (Broadwell, no TSX)
x86 Broadwell Intel Core Processor (Broadwell)
x86 Opteron_G1 AMD Opteron 240 (Gen 1 Class Opteron)
x86 Opteron_G2 AMD Opteron 22xx (Gen 2 Class Opteron)
x86 Opteron_G3 AMD Opteron 23xx (Gen 3 Class Opteron)
x86 Opteron_G4 AMD Opteron 62xx class CPU
x86 Opteron_G5 AMD Opteron 63xx class CPU
x86 host KVM processor with all supported host features (only available in KVM mode)

Recognized CPUID flags:
fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 pn clflush ds acpi mmx fxsr sse sse2 ss ht tm ia64 pbe
pni|sse3 pclmulqdq|pclmuldq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cid fma cx16 xtpr pdcm pcid dca sse4.1|sse4_1 sse4.2|sse4_2 x2apic movbe popcnt tsc-deadline aes xsave osxsave avx f16c rdrand hypervisor
fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 erms invpcid rtm mpx avx512f rdseed adx smap pcommit clflushopt clwb avx512pf avx512er avx512cd
syscall nx|xd mmxext fxsr_opt|ffxsr pdpe1gb rdtscp lm|i64 3dnowext 3dnow
lahf_lm cmp_legacy svm extapic cr8legacy abm sse4a misalignsse 3dnowprefetch osvw ibs xop skinit wdt lwp fma4 tce nodeid_msr tbm topoext perfctr_core perfctr_nb
invtsc
xstore xstore-en xcrypt xcrypt-en ace2 ace2-en phe phe-en pmm pmm-en
kvmclock kvm_nopiodelay kvm_mmu kvmclock kvm_asyncpf kvm_steal_time kvm_pv_eoi kvm_pv_unhalt kvmclock-stable-bit
npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pause_filter pfthreshold
xsaveopt xsavec xgetbv1 xsaves
arat
root@kvm:~#

From the preceding output, we can see the list of CPUs that we can pass as parameters to the -cpu flag in order to emulate that CPU type inside our virtual machine.

To start a new virtual machine using the qemu-system utility, perform the following steps:

root@kvm:~# qemu-system-x86_64 -name debian -vnc 146.20.141.254:0 -cpu Nehalem -m 1024 -drive     format=raw,index=2,file=debian.img -daemonize    
root@kvm:~#

root@kvm:~# pgrep -lfa qemu
3527 qemu-system-x86_64 -name debian -vnc 146.20.141.254:0 -m 1024 -drive format=raw,index=2,file=debian.img -daemonize    
root@kvm:~#

root@kvm:~# pkill qemu    
root@kvm:~#

root@kvm:~# qemu-system-x86_64 -vnc 146.20.141.254:0 -m 1024 -hda CentOS-7-x86_64-GenericCloud.qcow2 -daemonize    
root@kvm:~#

root@kvm:~# pgrep -lfa qemu
3546 qemu-system-x86_64 -vnc 146.20.141.254:0 -m 1024 -hda CentOS-7-x86_64-GenericCloud.qcow2 -daemonize    
root@kvm:~#

root@kvm:~# pkill qemu    
root@kvm:~#

How to start a virtual machine with QEMU/KVM depends greatly on the type of image and how the partitions are structured inside that image.

We used two different image types with different partitioning schemes to demonstrate this concept.

In step 1, we used the qemu-system-x86_64 command to emulate a x86_64 CPU architecture, specifically we passed the -cpu Nehalem flag, emulating the Nehalem CPU model. We passed the IP address of our host as a parameter to the -vnc flag. This starts a VNC server in the VM so that we can later use a VNC client to connect to the QEMU instance. We specified the amount of memory to be allocated to the instance, in this case, 1GB with the -m flag. We instructed QEMU that we are going to use a raw image with the format=raw option and the name and location of the actual image with the file=debian.img parameter.

Recall that this raw image contains two partitions with the second partition containing the root filesystem where the bootloader is located. This is very important to remember because we need to specify from what partition index the guest OS should load. We do that with the index=2 flag. Finally, we pass the -daemonize parameter to background the QEMU process.

In step 4, we started another QEMU instance, this time using the qcow2 CentOS image we downloaded earlier. We did not have to specify from what partition we need to boot from this this time because most prebuilt images use the first partition, or only have one partition. We also used the -hda flag instead of the -drive parameter, just to demonstrate that both options can be used with the same result. The -hda flag tells QEMU the first disk for the instance should be loaded from the filename that follows it.

In order to start a KVM instance, you will need the following:

To check whether your CPU supports virtualization, run the following code:

root@kvm:~# cat /proc/cpuinfo | egrep "vmx|svm" | uniq
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm epb tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid cqm xsaveopt cqm_llc cqm_occup_llc dtherm arat pln pts
root@kvm:~#

The presence of the vmx (for Intel) or svm (for AMD) flags indicate that your CPU supports the virtualization extensions.

To manually load the KVM kernel module and ensure that it's been loaded, run the following code:

root@kvm:~# modprobe kvm
root@kvm:~# lsmod | grep kvm
kvm                   455843  0
root@kvm:~#

To start a KVM instance, ensure that it's running and finally terminate it, execute the following:

root@kvm:~# qemu-system-x86_64 -name debian -vnc 146.20.141.254:0 -m 1024 -drive format=raw,index=2,file=debian.img -enable-kvm -daemonize    
root@kvm:~#

root@kvm:~# pgrep -lfa qemu    
4895 qemu-system-x86_64 -name debian -vnc 146.20.141.254:0 -m 1024 -drive format=raw,index=2,file=debian.img -enable-kvm -daemonize    
root@kvm:~#

root@kvm:~# pkill qemu    
root@kvm:~#

To start a QEMU/KVM virtual machine, all we had to do differently from what we performed in the Installing and configuring QEMU recipe is pass the -enable-kvm flag to the qemu-system-x86_64 command.

In step 1, we specified a name for the VM with the -name flag, provided the IP address of our physical host to the -vnc flag, enabling VNC access for the virtual instance, allocated 1 GB of memory with the -m flag, specified the partition where the bootloader is located with the index=2 parameter, the image format, and name, and finally we enabled KVM hardware acceleration with the -enable-kvm parameter and deamonized the process with the -daemonize flag.

In step 2, we ensured that the instance is running and we terminated it in step 3.

As an alternative to directly running the qemu-system-* commands, on Ubuntu systems there's the qemu-kvm package that provides the /usr/bin/kvm binary. This file is a wrapper to the qemu-system-x86_64 command, and it passes the -enable-kvm parameter to it automatically.

To install the package and use the kvm command instead, run the following:

root@kvm:~# apt install qemu-kvm
...
root@kvm:~# kvm -name debian -vnc 146.20.141.254:0 -cpu Nehalem -m 1024 -drive format=raw,index=2,file=debian.img -daemonize
root@kvm:~# pgrep -lfa qemu
25343 qemu-system-x86_64 -enable-kvm -name debian -vnc 146.20.141.254:0 -cpu Nehalem -m 1024 -drive format=raw,index=2,file=debian.img -daemonize
root@kvm:~#

You might have noted that starting and stopping QEMU/KVM instances is somewhat of a manual process, especially having to kill the instance process in order to stop it. In Chapter 2, Using libvirt to Manage KVM, we are going to walk you through a set of recipes that will make managing the life cycle of KVM virtual machines much easier, with the userspace tools that the libvirt package provides.

In order to complete this recipe, you will need the following:

root@kvm:~# qemu-system-x86_64 -name debian -vnc 146.20.141.254:0 -cpu Nehalem -m 1024 -drive format=raw,index=2,file=debian.img -daemonize    
root@kvm:~#

root@kvm:~# pgrep -lfa qemu    
4987 qemu-system-x86_64 -name debian -vnc 146.20.141.254:0 -cpu Nehalem -m 1024 -drive format=raw,index=2,file=debian.img -daemonize    
root@kvm:~#

The VNC login screen

VNC session

In step 1, we started a new QEMU instance with KVM acceleration and enabled a VNC server on it with the specified IP address and display port. We specified the amount of available memory and the CPU model name.

In step 4, we logged in the instance using the root user and the password we created when building the image, then obtained the CPU information by running the lscpu command. Note how the CPU model name matches what we specified with the -cpu flag when we started the virtual machine. Next, we checked the allocated memory with the free command, which also matches what we previously specified with the -m parameter.