Chapter 9. Scaling-out RavenDB

As applications grow to support hundreds and thousands of users, database servers must handle more information. To avoid server bottlenecks, we need to scale-out applications and servers to efficiently use more resources in order to do more useful work.

This chapter explains the different options that are available for scaling-out RavenDB, and you will learn how to expand your databases to multiple servers rather than a single, bigger server. Also, in this chapter, we will focus on the expanding factors that go into improving your RavenDB performances.

In this chapter, we will cover:

• RavenDB sharding

• Sharding in blind and smart mode

• Mixing sharding and replication

What is scaling-out?

Scaling-out is a strategy for increasing capacity that involves distributing the load over multiple machines and making multiple servers perform the work of one logical server. A Web Farm, which generally involves using multiple machines and distributing the load across them, is an excellent example of scaling-out where each server in the farm is completely independent and hosts an identical copy of the entire website. In such a system, users are load balanced across the servers, and they rarely realize that there is more than one server behind the scenes.

Scaling-out is necessary because individual servers can handle only a finite number of connections. Once a server is processing thousands of requests, adding more RAM, network adapters, or processors may not increase the server's capacity enough to meet the demand, and there is always a performance bottleneck. Do not confuse scale-out with scale-up. Scale-up typically refers to architectures that use a single fixed...

RavenDB sharding

RavenDB sharding is the way to scale-out RavenDB databases horizontally. The idea is to distribute a single logical database across the RavenDB cluster servers. Unlike replication, where each server contains a complete (replicated) copy of data, RavenDB sharding will distribute data across servers and each server will hold just a portion of the data. RavenDB supports database sharding natively and allows you to split your data across servers in an easy way.

Sharding with RavenDB is as easy as 1-2-3. All you need is a list of connection strings to identify the nodes in the shard cluster. You will also create a shard strategy which is simply a set of instructions that define which node is responsible for what data. Finally, we use the list and the strategy to create a smart client. Let's look at the classes and interfaces that can make this happen.

The ShardStrategy object allows you to customize different aspects of the sharding behavior, giving you the option of fine grained...

Time for action – preparing RavenDB for sharding

To learn how to implement the RavenDB database sharding, we will create a new application (ShardRavenDB) which we will use to create and interact with shards. But before doing that, we need to prepare the shard infrastructure by simulating a RavenDB cluster and running more than one RavenDB server. Then, we will create some empty databases which we will populate when running the application.

Let's say that in our application, we have to handle data from the world's countries. Our choice would be to store countries data on a shard which depends on continents. For example, the countries in Asia will be stored on one shard named AS for Asia, the countries in Europe will be stored on a second shard named EU for Europe, the North American countries would be stored on a third shard named NA for North America, and so on. The next figure illustrates the basic shard infrastructure for the ShardRavenDB application which will communicate with shards using...

Time for action – implementing RavenDB sharding (the blind mode)

Now that you have prepared the shard infrastructure for the ShardRavenDB application, you are now ready to create it and learn how to create and populate shards. Once the application is created, you will run it and then observe and analyze the RavenDB logs:

1. In Visual Studio, create a new Console Application and name it ShardRavenDB.

2. Add the RavenDB Client reference to the application using the NuGet package manager.

3. Add a new class, name it City, and complete it using the following code snippet:

   

4. Add a new class, name it Country, and complete it using the following code snippet:

   

5. Add a new class, name it Shards, and complete it using the following code snippet:

   

6. Add the Shards class constructor and destructor then complete them as shown in the following code snippet:

   

7. Add a new class, name it PopulateShards, and complete it using the following code snippet:

   

8. Add the AddDocumentsToShards() method to the PopulateShards class and complete...

Time for action – implementing RavenDB sharding (the smart mode)

As you can see, blind mode sharding doesn't meet our target of storing country data on independent shards depending on their continents. You will need to modify the ShardRavenDB application to interact with shards in a smart way. Then, you will observe and analyze the RavenDB logs:

1. On each RavenDB shard, delete all documents and temporary indexes in the Asia, Europe, and NorthAmerican databases.

2. Open the ShardRavenDB project in Visual Studio.

3. Modify the PopulateShards class constructor to look like the following code snippet:

   

4. Ensure that the three RavenDB shard instances are running.

5. Save the ShardRavenDB project and press F5 to build the application and run it.

What just happened?

You just modified the ShardRavenDB application to interact with RavenDB shards in a smart way.

To interact with shards in the smart mode, you call the ShardingOn() method on the ShardStrategy object and tell RavenDB that you want to use a specific property...

Mixing sharding and replication

Mixing load balancing and availability is a common and simple way to scale-out your servers and databases. This can be done by distributing your data across several shards and mixing this distribution with replication by replicating every shard. Then, you can use the replicas to read queries. This technique works well for a read-heavy application. RavenDB does this out of the box.

You may choose to shard your database with dedicated failover nodes. In this configuration, each shard is configured as the replication master, and in front of each shard, a dedicated server is configured as the replication slave (for each master). In this case, the replication node numbers are at least the same shard nodes. The advantage is that if one of the primary nodes is failing, RavenDB will automatically switch to the replicated copy.

Another option is to use sharding primarily as a means of reducing load on the servers and set up replication between the different nodes without...

Summary

In this chapter, we learned how to scale-out RavenDB and how it is easy to implement sharding in RavenDB which is supported natively. We covered RavenDB sharding modes and database sharding techniques. Also, we learned how to initialize, populate, and query a sharded Document Store. Specifically, we covered RavenDB sharding in blind mode and smart mode, including when and how to use each of these sharding modes.

We learned how to mix RavenDB sharding and replication in order to improve the server performance and availability.

In the next chapter, we will talk about RavenDB profiling and how it can help you to improve your RavenDB server performances. Keep reading!