Cloud Storage Accelerator

The Cloud Storage Accelerator Overview

The Cloud Storage Accelerator was introduced in SP12 as a method for using cloud storage as a backup target. By installing and enabling the Storage Accelerator feature on the client, data now bypasses the MediaAgent and is written directly to cloud storage for complete communication with the storage target. This process speeds up the backup and reduces costs. When deduplication is used, the client communicates with the MediaAgent to reference and update the deduplication database (DDB), then sends only the unique blocks from the client directly to the cloud storage target.

NOTE: The original purpose of this design was for Endpoint protection. 

Roaming users protection using Cloud Storage Accelerator

Why use Cloud Storage Accelerator?

Data Center Storage in the Cloud

A company requires data protection even if its large sales team is on the road. The typical Commvault^® environment design requires all data on remote laptops to be transferred to a MediaAgent residing at a data center or in the cloud. With a data center design, there may be a situation where the throughput to the MediaAgent is limited or causes throttling. This can affect the data center WAN connection or increase the backup time beyond what is acceptable. Endpoint protection is also risky — it is the most common entryway for malware to attack a computer environment. If your primary data center storage is in the cloud, the Storage Accelerator can write directly to that cloud storage target and reduce any chance of a malware risk to your datacenter environment. 

Protecting Large VMs in the Cloud 

Using a VSA to protect very large VMs in the cloud may be problematic, which is why Commvault recommends using agents for this type of protection. Agents allow the Cloud Storage Accelerator (which is located on the VM) to write data directly to cloud storage. There is no need to configure a MediaAgent in the cloud just to protect a small amount of data. And due to the cost of egress, it's unlikely you'll protect the data with a MediaAgent that is external to the cloud. Using Commvault^® software rather than a native cloud tool is an efficient and cost-effective strategy for protecting your data. The only impact of egress cost is from DDB & Index updates, which are minimal.  

MediaAgent Instance Size

The MediaAgent instance size can be greatly reduced when compared to a traditional configuration. With the Storage Accelerator, the MediaAgent is only processing the DDB and Index updates. Higher throughput requirements are no longer needed since it does not process new data from the clients.

Restores using the Cloud Storage Accelerator

With Cloud Storage Accelerator, restores are handled through the direct transfer of data from the cloud storage target to the client.  This process bypasses the MediaAgent and reduces restore times. Using the Cloud Storage Accelerator is especially powerful when many restores process simultaneously to different clients. While processing, the MediaAgent is only queried for index information, which is also minimal.

Initially, the Cloud Storage Accelerator was only available for Windows clients in SP12, but as of SP14, Linux was added to the list of clients on which the Storage Accelerator can be installed.

Restoring data for Storage Accelerator enabled clients

Cost of Egress Data

When an environment has a MediaAgent outside of the cloud, there are egress charges associated with the communication for the DDB and Index Directory.  The following formula helps calculate those costs:

DDB Data =  (S/512 KB)*300 Bytes *
Index Directory Data = Number of Files*500 bytes *

*For the DDB calculation, the variable "S" is the size of data in kilobytes.  For the Index Directory calculation, the "Number of Files" is the actual number of files that are being backed up.  

For example, here is an environment protecting 100,000 files and 10 TBs of data. First, the 10 TBs needs to be converted into KBs, which is 10,000,000,000.

DDB Data = (10,000,000,000 / 512) * 300 = 5,859 MBs 

Index Directory Data = 100,000 * 500 = 50,000,000 bytes = 50 MBs

If the CommServe^® server is not in the cloud, there is traffic to consider. This is much more difficult to estimate considering the amount of traffic between the CommServe^® server and the client – depending on the environment and how it is configured. For example, use 100 MBs as an approximate data egress per backup job.

In this example, the approximate amount of egress data when the CommServe^® server and MediaAgent are not in the cloud is 5,859 + 50 + 100 = 6,009 MBs = 6 GBs

Now let's use these example figures and apply them to the cloud to determine the cost for an Azure Public Cloud. According to the current pricing (as of writing) for East US 2 in USD it's $0.087/GB for less than 10 TBs of data, and the first 5 GBs are free. Let's ignore the fact that the first 5 GBs are free since we'll assume this has already been used for another task.

The actual cost to protect 10 TBs of data would be:

6 GBs * 0.087 = $0.52

In this example, the entire egress costs for protecting 100,000 files and 10 TBs of data is $0.52 USD.

Keep in mind that this is for a FULL baseline backup, which usually only happens once per client. From then on we are only writing out incremental deduplicated data so the DDB and Index Directory data sizes will be much less.

Secondary Copies

When creating secondary copies, or auxiliary copies, the MediaAgent handles the movement of data, which means it is important to consider where the MediaAgent is located. If it's on-premises, the MediaAgent and an additional copy are used to send to another region in the same cloud. Note that the cloud-based MediaAgent is recommended so as not to endure egress charges and slow throughput. 

Cloud Storage Accelerator Metrics 

This section provides metrics in a test environment in Azure Public Cloud. The following tables compare the performance of backup and restore times for different scenarios. The tests were all done on VM instances in Azure. The test specifications are listed below. The Azure's native backup tool, 'Azure Backup,' is also added in this comparison. There are obvious benefits to using Commvault vs Azure Backup. For instance, Commvault^® software writes to all three tiers of blob storage with deduplication, unlike Azure Backup, which only writes to Hot Blob without deduplication.

Cloud storage accelerator backups comparison chart

Test specifications

Clients:

Azure: 5 Windows Server 2016 Datacenter clients & one Windows Server 2016 Datacenter MediaAgent

Location: Clients and MediaAgent in Azure. CommServe^® server on-premises.

Client specifications: Unmanaged disk VMs (D4S VMs)(4 CPU cores and 16GB of RAM)

Operating system disk: 1x126GB Premium or Standard (SSD) (IOPs limit: 500, throughput limit in MB/s: 100)

Data disk: 1x250GB Premium or Standard (SSD) or Standard (HDD) (IOPs limit: 1100 or 500, throughput limit in MB/s: 125 or 60)

CommServe server and MediaAgent:

Virtual Machine: Standard DS4 v3 (4 CPU cores and 16GB of RAM) Max IOPs: 6400

Operating system disk: 1x126GB Premium (SSD) (IOPs limit: 500, throughput limit in MB/s: 100)

Deduplication database disk: 1x400GB Premium (SSD) Cache: none (IOPs limit: 2300, throughput limit in MB/s: 150)

Index cache disk: 1x400GB Standard (SSD) Cache: none (IOPs limit: 500, throughput limit in MB/s: 60)

Storage: Azure Hot Blob in Region 'East US 2'

Software configuration: Two streams for each client/instance. Encryption On. Software compression enabled with source-side deduplication

Summary

Cloud Storage Accelerator provides the following benefits:

Pros

• Reduces network hops by writing directly to cloud storage
• Reduces compute load on the MediaAgent
• Restores directly from cloud storage and bypasses MediaAgent
• Backups are faster (Full, Incremental, Synthetic Full)
• Restores are faster (Granular)
• Reduces chances of introducing malware into the data center
• Reduces the cost of MediaAgent running in the cloud
• Writes data to all available cloud storage tiers

Cons

• Includes an extra step to configure the Reg Key
• Requires additional compute on the client
• Restores for Full VM are much slower
• Requires managing Client CPU/Network overhead as to not interfere with production use

Cloud Storage Accelerator Installation and Configuration

With date mover capability akin to the MediaAgent, the Cloud Storage Accelerator package installation requires just a few steps:

The Cloud Storage Accelerator package is a MediaAgent package that is stripped down to the role of data mover. The installation only requires a few manual steps:

Cloud storage accelerator installation steps:

1. First, install the Storage Accelerator package on the client computer. This is done by running the installation media and from the Select Packages dialog box. Click Tools and then select Storage Accelerator. 
   
   Note: Make sure to install only the Storage Accelerator package. Do not install the entire MediaAgent software on the client.
   
   
2. Configure a MediaAgent that will handle the DDB and the Index Directory updates and create a cloud storage target that the client will send data to.
3. Create a storage policy using the MediaAgent and cloud storage target as the data path.

Note: The Storage Accelerator package is supported with libraries that have a single mount path.

1. Associate the storage policy with the subclient used to backup data from the client.
2. Now, run a backup from the subclient.  

To verify that the backup job is writing directly to Cloud Storage and bypassing the MediaAgent, access Job Controller or your Job History. Verify that the MediaAgent is listed with the same name as the client (instead of the MediaAgent).

In the example below, if data from Winserver is currently protected, then the MediaAgent must be listed as 'Winserver.'