Chapter 1: Installation – Best Practices and Optimization
Veeam Backup & Replication v10 is part of Veeam Availability Suite, which is ready for the modern data center and allows you to back up all of your workloads, including Cloud, Virtual, and Physical. It is simple, yet flexible, when it comes to meeting your most challenging business requirements. In this chapter, we will discuss the installation of the software, what components make up Veeam Backup & Replication v10, and some best practices and optimizations. There will be practical examples throughout the chapter of how to optimize specific elements that make up the Veeam environment. We will also touch on some of the websites, including the Best Practices Guide for Veeam, among others, to give you the resources to help set up Veeam in your environment. As they say with Veeam – It Just Works.
In this chapter, we're going to cover the following main topics:
- Understanding the components of Veeam Backup & Replication
- Understanding the best practices for Veeam installation and setup
- Configuring and optimizing proxy servers
- Setting up repository servers for success
- Understanding the scale-out repository
To ensure a successful installation, you will require the following:
- A Windows 2016/2019 server with the required disk space to install the application (2012 R2 is also currently supported). Windows 10 and other modern Windows desktop operating systems are also supported.
- The latest ISO file from www.veeam.com, which requires registration on the site and allows you to obtain a trial license. As of the time of writing, version
10.0.1.4854is the current release.
Understanding the components of Veeam Backup & Replication
Veeam Backup & Replication has the following components:
- Backup Server: Installed on either a physical or virtual server, this is the core component of Veeam Backup & Replication that controls and coordinates backups, replication, recovery verification, and restore tasks. It manages job scheduling as well as resource allocation. It also contains global configuration settings for the environment.
- Proxy Server: These are the workhorses of the environment as they offload tasks from the backup server and are the data movers between the backup server and repositories. It is the proxy servers that you can scale to add processing tasks to the environment.
- Repository Server: This is the backup target where all backup files (VBM – metadata, VBK – full backup file, and VIB – incremental) get written. The repository servers can be Windows- or Linux-based and have different filesystems, such as NTFS, ReFS, and XFS.
- Enterprise Manager Server: This server is an optional component and gets installed when you want to manage multiple backup servers in a single pane of glass. You can see backup jobs within your environment from multiple backup servers. Enterprise Manager also allows you to search all Microsoft Windows guest OS files within all current and archived backups with one-click restore. This optional component uses a separate SQL server and backup catalog service for indexing the guest operating systems.
- Built-In WAN Acceleration: This is also an optional component that allows for better movement of data between sites. It helps minimize data transfer by comparing the data blocks before the transfer, so only new blocks are sent across the WAN. It also accelerates backups by up to 50x between sites.
- File Explorers: These are built-in applications used during restore and look very similar to Windows Explorer. They allow you to browse the restore point to select specific files or data to be restored. There are explorers for Active Directory, Microsoft SQL, Oracle, Microsoft Exchange, Microsoft SharePoint, and Microsoft OneDrive for Business.
You can reference the Veeam Backup & Replication File Explorers at the following website:
The following diagram will show all the aforementioned components:
When you have multiple offices, you may wish to deploy Veeam Backup & Replication in a more advanced setup, as illustrated here:
Now that you have a better understanding of the components that make up Veeam Backup & Replication, we will now get into the installation, as well as best practices and optimizations, in the next section.
Understanding the best practices and optimizations for Veeam installation
The installation of Veeam Backup & Replication v10 is a straightforward process, and this section will go through the operation of the install as well as touch on best practices and optimizations for your environment. Setting up Veeam, if not done right, can lead to components not working correctly and poor performance, among other things. However, if you set up Veeam correctly, it will protect your data and environment with little configuration.
Installation of Veeam Backup & Replication v10
Before installing Veeam, we need to ensure that you have a server deployed, either Windows 2016 or 2019, with enough disk space for the installation. The disk layout should be similar to the following:
- OS drive: This is where your operating system resides and should be used only for this purpose.
- Application drive: This will be your application installation drive for Veeam and all its components.
- Catalog drive: Veeam uses a catalog that can generate around 10 GB of data per 100 VMs backed up with file indexes. If this were to be a significant storage requirement for your deployment, it may be advisable to allocate to a separate drive.
Once your server is ready, and you have downloaded the ISO file and mounted it, complete the following steps for installation:
- Run the
setup.exefile on the mounted ISO drive:
- Click either on the Install button under the Veeam Backup & Replication 10a section on the left or the Install link on the right side under Standalone components. At this point, you will see the License Agreement window, so you need to select the two checkboxes to place a checkmark and then click Next to continue.
- You will now need to provide a valid license file, be it a purchase or a trial. If you do not have it at this part of the installation, you can click Next to continue, and Veeam will operate in the Community (Free) Edition. When you obtain the license file, you can install that within the application:
- The next screen is where you choose which components you want to install and which directory. Veeam recommends that all of them are selected:
– Veeam Backup & Replication: The main application.
– Veeam Backup Catalog: Used when you turn on Guest OS Indexing within your jobs. This option takes all the Guest files and stores them in a catalog where you are then able to use advanced searching across all restore points and conduct 1-click file restores from the Enterprise Manager console.
– Veeam Backup & Replication Console: This is where you go to view, create, and edit jobs, and manage the environment.
- After clicking Next, the installer will then do a system check for any pre-requisites required, and if something is missing, you will be prompted and have the option to install the missing components:
- Click the Install button to have the missing components installed.
- Once all the components have passed, you can click Next to move to the following screen. Unlike in previous versions of Veeam Backup & Replication, the next screen does not give you the option to input a user account to run the services. Instead, with version 10 of Veeam, you need to select the checkbox next to Let me specify different settings and then click Next.
- You will now have the opportunity to enter a user account for the Veeam services, better known as a Service Account. There are some recommended settings for this service account:
– You must have Local Administrator rights on the Veeam server.
– If you are using a separate SQL Server and not the Express edition that comes with the install, you require permissions to create the database.
– You will need full NTFS permissions to the folder containing the catalog.
For all the detailed permissions, please visit https://helpcenter.veeam.com/docs/backup/vsphere/required_permissions.html?ver=100:
For this setup, I am using an account that I created on my lab server. In contrast, in a production scenario, you would already have a service account set up in Active Directory to enter at this step:
- The next screen lets you select the type of SQL install you will be using, and in a lab scenario, using SQL Express is good enough. If you are in an enterprise environment, the recommended best practice is to use an external SQL Server for best performance. Also, note that you can use Windows authentication or SQL Server authentication:
- After selecting the appropriate options, click Next once again.
- The next window is the TCP/IP port configuration. Should you want to use different ports, you can adjust them here, but the default ports should suffice. You then click Next to move to the Data Locations screen:
- Here, you indicate the application drive for the Instant recovery write cache, which mounts restore points during recovery and the dedicated drive for Catalog folder for guest OS indexing.
The installer is now ready to first complete installation of the local SQL Express instance and then the application. Veeam will also set your user account that you selected to initiate all the services:
- After reviewing the setup, click Install to proceed with the installation and start setting up the components that work together with the backup server.
We will now proceed to configuring the required settings for Veeam to work with VMware:
- Repository Server: The server used to store the backup files.
- Proxy Servers: The servers that perform all the backup tasks.
- VMware vCenter Credentials: Used to connect and see your clusters, hosts, vApps, and Virtual Machines (VMs). vCenter Server is not required as standalone ESXi hosts are also supported if licensed in VMware:
- When you first launch the Veeam Backup & Replication console, you are taken directly to the Inventory tab, and Virtual Infrastructure will be highlighted:
- This screen is where we will begin adding the virtual center to enable the backup of your virtual machines. Click on the ADD SERVER option to start the process. You will then get prompted to select what kind of server to add. Choose VMware vSphere and then either vSphere or vCloud Director:
You would typically select vSphere; however, if you have vCloud Director in your environment, you may also want to choose this option. When you choose vSphere, you will get prompted for two things to complete the connection:
– The DNS or IP address of your vCenter server.
– Credentials; these can either be a
vsphere.localuser or a domain account set up for access.
You do have the option of selecting Microsoft Hyper-V and Nutanix AHV, but this book is focused mainly on VMware vSphere.
- Enter in the required credentials and then click Next, followed by Apply, to complete the VMware vSphere setup. You will now see your vCenter server listed under the Virtual Infrastructure section of the console and will be able to browse the hosts and virtual machines.
We now move on to the next piece required for the infrastructure, which is the proxy server. By default, the Veeam Backup & Replication server is to be your VMware Backup Proxy and File Backup Proxy. Due to the limitations of my lab, I am going to use this server as an example, but in the real world, you would add multiple proxy servers to your environment for better performance and according to best practice. Also, based on best practices, you would typically disable the Veeam Backup & Replication server from being a proxy server to allow the other proxy servers to handle the workload.
The next component you will require is a Repository Server, the location where Veeam Backup & Replication will store the backup files. By default, Veeam Backup & Replication creates a Default Backup Repository, typically on the most significant sized drive attached to your backup server. This location will be where the Configuration Backups usually get backed up. There are multiple options for adding a repository:
The first three selections would be for block storage, and the last one is object storage, which only works when creating a scale-out backup repository as the capacity tier for offloading data.
For further information on Veeam Backup & Replication best practices and the documentation, please visit the following websites:
- Veeam best practices website: https://bp.veeam.com/vbr/
- Veeam documentation website: https://helpcenter.veeam.com/docs/backup/vsphere/overview.html?ver=100
Configuring and optimizing proxy servers
Proxy servers are the workhorses of the Veeam Backup & Replication v10 application, and they do all the heavy lifting or processing of tasks for backup and restore jobs. When you set up Veeam, you need to ensure that the proxy servers get configured as per best practices:
- Veeam Installer Service: This is used to check the server and upgrade software as required.
- Veeam Data Mover: This is the processing engine for the proxy server and performs all the required tasks.
Veeam Backup & Replication proxy servers use what we call a Transport Mode to retrieve data during backup. Three standard modes are available, and they are listed in order, starting with the most efficient method:
- Direct Storage access: The proxy is placed in the same network as your storage arrays and can retrieve data directly from there. This method allows for two transport modes – Direct SAN access and Direct NFS access. The backup load is offloaded from the hypervisor to process the workloads.
- Virtual Appliance: This mode mounts the VMDK files to the proxy server for what we typically call Hot-Add Mode to back up the server data.
- Network: This mode is the least efficient, and is used when the Failover to network mode if primary mode fails, or is unavailable option is selected. It moves the data through your network stack, and it is recommended not to use 1 GB, but rather 10 GB or higher.
In addition to these standard transport modes provided natively for VMware environments, Veeam provides two other transport modes: Backup from Storage Snapshots, and Direct NFS. These provide storage-specific transport options for NFS systems and storage systems that integrate with Veeam.
Refer to the integration with storage systems guide: https://helpcenter.veeam.com/docs/backup/vsphere/storage_integration.html?ver=100.
Along with the transport modes, there are specific tasks that the proxy server performs:
- Retrieving the VM data from storage
- Sending the data to the backup repository server (backup job) or another backup proxy server (replication job)
The following are things you should consider in relation to your proxy servers:
- Operating System: Most software vendors will always recommend the latest and greatest, so if you are going to choose Windows, then 2019, or if you are going to choose Linux, then the newest flavor you have picked (Example – Ubuntu 20.04.1 LTS). Note that as regards Linux VMware backup proxies, only HotAdd mode is supported in Veeam Backup & Replication v10.
- Proxy Placement: Depending on the transport mode for the server, you will need to place it as close to the servers you want to back up, such as on a specific host in VMware, a physical server, or a blade enclosure. The closer to the source data, the better!
- Proxy Sizing: This can be tricky to determine and will be dependent on the server being physical or virtual. Veeam proxy servers complete what are called Tasks, which is the processing of one virtual disk for a VM or one physical disk for a server. Therefore, Veeam recommends one physical core or one vCPU as well as 2 GB of RAM per task.
Veeam has a formula used to calculate the required resources for a proxy server:
- D = source data in MB
- W = backup window in seconds
- T = throughput in MB/s, = D/W
- CR = change rate
- CF = cores required for full backup, = T/100
- CI = cores required for incremental backup, = (T * CR)/25
Based on these requirements, we can use a sample of data to perform the calculations:
- 500 virtual machines
- 200 TB of data
- 8-hour backup window
- 10% change rate
Using these numbers, we perform the following calculations:
We now use the numbers we calculated to determine the required number of cores needed to run both full backup and incremental backup to meet your defined SLA:
This formula takes the throughput from the previous formula and then calculates the number of CPU cores required.
Based on our calculations and considering you require 2 GB of RAM for each task, you would need a virtual server with 73 vCPUs and 146 GB of RAM. This size may seem like a considerable server, but keep in mind that it uses the sample data. Your calculations will likely be much smaller or possibly more extensive, depending on your dataset.
Should you decide to use a physical server as a proxy, you should have a server with 2 – 10 core CPUs. In the case of our sample data, two physical servers are what you require. If you are using virtual servers for proxies, then the best practice is to configure them with a maximum of 8 vCPUs and add as many as required for your environment – in this case, we would need nine servers.
Should you want to size things based on incremental backups only, your requirements are less than half of those for a full backup – 29 vCPUs and 58 GB of RAM.
There are limitations for proxy servers that you need to be aware of when it comes to job processing and performance. As noted above, a proxy server performs tasks, which are assigned CPU resources. Concurrent task processing is dependent on the resources you have available in your infrastructure and the number of proxy servers you have deployed. As seen here, when adding a proxy server to Veeam Backup & Replication, there is the Max concurrent tasks option, which correlates to the number of CPUs that are assigned:
The task limits can be viewed at the following link: https://helpcenter.veeam.com/docs/backup/vsphere/limiting_tasks.html?ver=100.
Job performance gets impacted based on the tasks of a proxy server. As an example, if you had a proxy server with 8 CPUs and you added 2 virtual machines for backup, one with 4 disks and the other with 6 disks, only 8 of 10 disks would get processed in parallel. The remaining 2 disks would have to wait on resources before backing up because tasks get assigned per virtual disk of a VM during the backup process.
You should now be able to size your proxy servers correctly regarding things such as CPUs and RAM and understand proxy placement and how it processes tasks. Proxy servers send data to repository servers, which is the focus of the next section.
Setting up repository servers for success
A repository server is a storage location for your backups, so setting them up correctly the first time will ensure that you have the best performance. When creating a repository, it is always a good idea to follow the Veeam Backup & Replication best practices:
- ReFS/XFS: With Windows 2016/2019, ensure you format your repository drive(s) as ReFS with 64k block sizing to take advantage of space savings for synthetic fulls and GFS. For Linux, you need to set up XFS and Reflink to take advantage of space saving and fast cloning. In both of these situations, a storage efficiency will be realized for synthetic full backups. This efficiency prevents duplication, but is not deduplication.
- Sizing: Ensure that you adhere to the Veeam Backup & Replication recommendation of 1 core and 4 GB of RAM per repository task. Just like proxy servers, your repository servers have task limits as well. At a minimum, you need 2 cores and 8 GB of RAM.
When you calculate out the sizing requirements, you need to take into account your proxy servers and the amount of CPUs configured. You then need to use a 3:1 ratio for the core count on a repository server:
Example: If your proxy server was configured with 8 CPUs, you would then need to configure the repository server with 2 CPUs based on this rule of 3:1. To configure the RAM, you multiply the CPU count by four, ending up with 8 GB of RAM.
When you use the Windows ReFS filesystem as your repository, you need to consider the overhead required for the filesystem and be sure to add another 0.5 GB of RAM per terabyte of ReFS.
Setting up your task limits for a repository server is different to a proxy server due to the way tasks are consumed. The setting chosen will be handled differently:
- Per-VM Backup Files: When selected, this creates a backup chain per VM located in a job. Therefore, if the backup job has 10 virtual machines, then it will consume 10 repository tasks as well as 10 proxy tasks.
- No Per-VM Selection: The backup job consumes one repository task because all VM data gets written to the same backup file, and the proxy task will remain the same with one task per each virtual disk.
The task limits can be viewed at the following link: https://helpcenter.veeam.com/docs/backup/vsphere/limiting_tasks.html?ver=100.
When setting up a repository for the first time, you can set the task limit as follows:
When you limit the number of tasks per repository, and you have jobs with many virtual machines requiring backup, this will be one of the bottlenecks in your environment. You also need to ensure that you do not set the limit too high as that could overwhelm your storage, causing performance degradation. Make sure to test all your components and resources available for your backup infrastructure.
You should, after this section, be able to choose which type of filesystem for your repository and also size it correctly based on CPU and RAM. We also discussed the per-VM versus no per-VM methods. Now we will use this knowledge to tie this into creating a scale-out repository.
Understanding the scale-out repository
So, what is a Scale-Out Backup Repository, or SOBR, you ask? A SOBR, uses multiple backup repositories called performance extents to create a sizeable horizontal scaling repository system. Veeam Backup & Replication can use multiple repositories of various types, such as the following:
- Windows Backup Repositories: NTFS or the recommended ReFS
- Linux Backup Repositories: XFS with Reflink
- Shared Folder: NFS, SMB, or CIFS
- Deduplication Storage Appliances: ExaGrid, EMC DataDomain, HPe StoreOnce
The SOBR can expand with on-premises storage, such as block storage, or even a cloud-based object repository known as Capacity Extent. Veeam Backup & Replication combines the Performance and Capacity extents into one to summarize their capacities:
- Enterprise: Allows for a total of two SOBRs with three extents
- Enterprise Plus: Provides for an unlimited number of SOBRs with as many performance extents as required, but only one capacity tier per SOBR
Should you happen to downgrade your licensing from Enterprise Plus or Enterprise to Standard, you will lose the ability to target your jobs to the SOBR. You can, however, restore data from the SOBR.
With the different license types, this limits you in terms of the number of both SOBRs and extents per SOBR you can configure. As noted above, there is a limit of two for Enterprise, and an unlimited number for Enterprise Plus.
For the best performance and manageability, it is best to keep your SOBR limited to 3-4 extents if possible. There is less movement of data and if you have a failed extent, replacing it is fairly simple. Also, when you have too many extents and need to move data around, this can become a bit of a challenge when extents start getting full. If you are using object storage, then it will be one of the components of the SOBR that will be the capacity tier.
The SOBR works with many types of jobs or tasks in Veeam Backup & Replication:
- Backup jobs
- Backup copy jobs
- VeeamZIP jobs
- Agent backups – Linux or Windows agent v2.0 or later
- NAS backup jobs
- Nutanix AHV backup jobs
The next thing to keep in mind is the limitations of using a SOBR as there are certain things you cannot do:
- Only Enterprise & Enterprise Plus License support using a SOBR.
- You cannot use it as a target for configuration backup jobs, replications jobs, VM copy jobs, Veeam Agents v1.5 or earlier for Windows, and v1.0 or earlier for Linux.
- Adding a repository as an extent to a SOBR will not be allowed if there is an unsupported job using the repository.
- Rotating drives are not supported – an example would be a drive attached by USB or serial cable.
- You are unable to use the same extent in two scale-out repositories.
Refer to the following page limitations on the Veeam Backup & Replication website: https://helpcFixedenter.veeam.com/docs/backup/vsphere/limitations-for-sobr.html?ver=100.
- Performance Tier: Fast storage and fastest access to data
- Capacity Tier: Typically object storage for long-term archival and offloading capabilities or a lower tier of storage
The performance tier you require will be the one that provides the quickest access to files and restores as and when necessary. When you create a standard repository before adding it to an SOBR, there are specific settings retained in the SOBR:
- The number of simultaneous tasks it can perform
- The storage read and write speeds
- Data decompression settings relating to storage
- The block alignment settings of the storage
What the SOBR will not inherit is a repository backed by rotating drives or if you selected to use the per-VM backup option. This option is on by default in a SOBR.
Another aspect to consider is the backup file placement policy that you will use. There are pros and cons to both, and specific operating systems such as ReFS and XFS require one over the other. The two types of placement policies are as follows:
- Data locality
Refer to the following performance tier page on the Veeam Backup & Replication website: https://helpcenter.veeam.com/docs/backup/vsphere/backup_repository_sobr_extents.html?ver=100.
Data locality allows the scale-out to place all backup files in the chain to the same extent within the SOBR, thereby keeping files together. In contrast, the performance policy will enable you to choose which extents to use for both full backup files (VBK) and incremental files (VIB). The metadata file (VBM) is located on all extents in the SOBR for consistency and in the case of files needing to move extents:
For further information on backup placement, refer to the following page on the Veeam Backup & Replication website: https://helpcenter.veeam.com/docs/backup/vsphere/backup_repository_sobr_placement.html?ver=100.
Using a capacity tier as part of your SOBR is suitable for the following:
- You can offload older data or when your SOBR reaches a specific percentage of capacity to allow you to free up storage space.
- Company policy stipulates that you have to keep a certain amount of data onsite, and that all older data is then tiered off after X days to the capacity tier.
- Using it falls into the 3-2-1 rule, where one copy of the information is offsite. Refer to the following blog post for more details on the 3-2-1 rule: https://www.veeam.com/blog/3-2-1-rule-for-ransomware-protection.html.
You specify the capacity tier after creating it as a standard repository and during the SOBR wizard at this stage:
Please visit the following capacity tier page on the Veeam Backup & Replication website: https://helpcenter.veeam.com/docs/backup/vsphere/capacity_tier.html?ver=100.
We are now going to tie all of the above information together and create an SOBR. First, you need to open the Veeam Backup & Replication console and select the Backup Infrastructure section on the bottom left. Once in this section in the tree, click on the Scale-out Repositories option on the left:
You will then name the scale-out and give it a thoughtful description. The default name is Scale-out Backup Repository 1. You then click Next, and you will come to the Performance Tier section of the wizard:
- Use per-VM backup files (recommended)
- Perform full backup when the required extent is offline
Click the Next button to proceed. You then will pick your placement policy, which is Data Locality or Performance. As noted, if using ReFS or XFS, you must select Data Locality to take advantage of the storage efficiency that each filesystem provides. Click Next after making your choice.
You can now choose to use the Capacity Tier option for your SOBR or just click the Apply button to finish. Note at this screen, when you select a capacity tier, that there are several options you can enable:
- Copy backups to object storage as soon as they get created in the performance tier.
- Move backups to object storage as they age out of the restore window – the default is 14 days, and you can also click on the Override button to specify offloading until available space is below a certain percentage.
- You can also encrypt your data upload to object storage as another level of security:
Note that some capacity tier targets support immutability. This feature is an essential attribute in the war on ransomware. In v10, capacity tier targets that support immutability include AWS S3 with object lock and some S3-compatible object storage systems. Immutability allows data to be protected and not deleted for the set period of days assigned.
Once complete, you will see your new SOBR, and when you select it, you will see the performance tier extents and capacity tier options if you chose it:
If you want further information on the SOBR, visit this page on the Veeam Backup & Replication website: https://helpcenter.veeam.com/docs/backup/vsphere/sobr_add.html?ver=100.
Once set up within Veeam Backup & Replication, the SOBR is pretty self-sufficient. Still, there is maintenance that you need to perform in order to ensure optimal performance, and plenty of storage is available for backups.
The final thing to discuss is the management of the SOBR after creation. Once created, you may need to do any of the following items:
- Edit the settings to change the performance policy, for example.
- Rescan the repository to update the configuration in the database.
- Extend the performance tier by adding another extent to the SOBR.
- Put an extent in maintenance mode to either perform maintenance on the server that holds it or evacuate the backups to remove the extent.
- Switch an extent into what is called Sealed Mode, where you do not want any more writes to it, but you can still restore from it; this allows you to replace the extent with a new one.
- Run a report on the SOBR.
- Remove an extent from the SOBR that requires maintenance mode.
- And lastly, remove the SOBR altogether.
For more information on SOBR management, please visit the following Veeam Backup & Replication website: https://helpcenter.veeam.com/docs/backup/vsphere/managing_sobr_data.html?ver=100.
This chapter has provided you with the tools required to complete the installation of Veeam Backup & Replication v10 and outlined what components make up the installation. We discussed the pre-requisites, including versions of SQL Server that you can use – Express or SQL Standard/Enterprise. We next addressed in detail how to set up proxy servers, as well as configuration best practices and optimal settings. This part was then followed by a discussion on repositories and how to create them, and also included best practices and optimizations for best performance. Lastly, we looked at the scale-out repository and how to set it up, including the performance and capacity tiers, as well as how to manage them after setup.
This chapter will ensure that you have all the basics covered and will help when we move to the next chapter, which covers the 3-2-1 rule—Chapter 2, The 3-2-1 Rule – Keeping Data Safe.