In a previous article we have explored the options and the steps on how to integrate Veeam Backup with AWS VTL tapes. As a final part of this series the last step is to configure a Veeam Tape Backup job to AWS VTL tapes. This configuration provides the option to send data directly to the Amazon Cloud taking advantage of the durability and availability option of the Cloud Provider. In addition, the option to lower cost over the longer periods when storing data to “colder” tiers like the Amazon Glacier and similar. This applies not just to backups but also to standard files, for example from NAS shares and similar.
The case for using Veeam Tape backup jobs to AWS VTL is an interesting one especially when paired with advanced retention scenarios like GrandFather-Father-Son (GFS) use cases. In general the idea would be to structure a Backup strategy which allows for the maximum amount of Data to be kept On-Premises for reduced RTO times and automatically send weekly, monthly, quarterly and yearly Fulls to the Cloud for a much lower cost per gigabyte. Surely, each Company has it’s own preference on how to best operate based on CAPEX and OPEX filtered by SLAs in place. Especially when comparing the cost with a de-dupe appliance. With so much data sprawling everywhere and on every platform, ensuring data availability across different environments is key. VTL Tapes still provide the flexibility of the physical ones without the “physical” approach of moving them to a tape center when these should be archived. Everything happens seamlessly in the Cloud. Last but not least the air-gap approach from tapes which provide that offsite and offline requirements for these type of media.
This and a lot more is exactly what the Veeam Tape backup job configuration is bringing: the ability to send / copy data to the Cloud taking advantage of the virtual tape infrastructures. All VTL tapes management is completed from a single Veeam console to create Media Pools, Media Sets, Data Retention and additional options like encryption, parallel processing and more.
Veeam Tape Backup job to AWS VTL configuration
From the Tape Infrastructure it is possible to create a Media Pool. There are “system” and “custom” Media Pools. The first ones are automatically created to seamlessly manage new tapes and their statuses like online and offline. These cannot be deleted. The custom ones can be added and editable at any time. As soon as a Tape Library is scanned all the available tapes will be included in the Free Media Pool. Let’s create one. In my case I will create a Media Pool per scope like Production VMs, Demo and similar.
Next step is to add the number of desired tapes to the Pool. This should be done in conjunction with the amount of data it is expected to be sent to tape. For example the backup chain consisting of the Full backup plus all pertinent incremental. Should these tapes in the Media Pool not be enough there is also the option to add them from the “Free Pool”, when available of course.
In the Media Set is possible to create the rules for automatically labelling the tapes used to send the backup to. Very handy it also allows to use variables for a more consistent naming convention.
In the retention section the options to overwrite tapes or even protect them for the desired period of time. In addition, the option to “store” them into a Vault. This feature also works with Veeam Location option.
In a final step before reviewing the options, the wizard provides the ability to choose if parallel processing on multiple tapes and encryption should be used. Hardware encryption is preferred when available as it improving performances compared to software encryption.
And finally a quick summary to review the main options.
With the same principle it is possible to create multiple Media Pools destined to different targets. For example backup jobs for Agents, Production VMs, other Hypervisors like Nutanix AHV and even other types of workloads.
At this point everything is ready in the infrastructure to create a Veeam Tape backup job to AWS VTLs. Let’s create a new Backup to Tape job with the wizard and start by adding the Job name and quick description.
Next is to add objects from the infrastructure. The explorer in the wizard now shows the existing backup jobs for which we want to send a copy to the AWS Cloud.
The wizard detects the backup chains in the job and prompts for an entire copy or just the last backup chain. This setting can be changed later on in the advanced settings as well. For example for several backup jobs to the same Media Pool we might want to use different settings.
The wizard now prompts where to send the Full Backup files. Let’s choose one of the Media Pools created previously.
Let’s repeat the same steps for incremental backups if this should be saved on tapes as well. I would advice generally using the same Media Pool.
These options are important. As a best practice Veeam offers the option to “eject the VTL tape from the slot” preventing accidental overwrites or deletion that could occur. Even better against ransomware attacks. The export option emulates the action of sending the tape to a tape center. In this case when the media set is exported will instruct the AWS Storage Gateway to “archive” the tape. When this happens the cost of retaining the tape in the AWS Cloud will be a lot lower. Of course it is still possible to retrieve archived VTL tapes. Depending on services this operation can take up to 3-5 hours.
Last but not least the option to schedule the Veeam Tape backup job with various options as per screenshot below.
A final summary to review and wrap up the job configuration.
When Veeam is running the backup to tape job it also show the information about the tape used along with other useful information.
Moving to the AWS Storage Gateway VM Monitor tab in VMware vCenter we can take a look at the actual resources. Definitely not consuming that much. Even though I must say this ws just at the beginning of the job. It was consuming more later on during the job. Subsequent ones were less resource intensive. I wonder if there is any caching involved across different jobs on the AWS Gateway.
In the meantime from the AWS Console it is possible to check in real time data coming through and populate the tape as shown by Veeam console. Data is now landing to AWS Storage Gateway.
As soon as the Veeam Tape backup job is completed the tape is ejected and exported. For a short period the status will be updated to in “transit to VTS”. Essentially there are 3 stages as soon as the upload from the AWS Gateway is successfully completed:
- IN TRANSIT TO VTS: Gateway is uploading data to AWS Cloud Storage
- ARCHIVING: Data upload is complete and the AWS Gateway is moving the tape to the archive
- ARCHIVED: The tape is moved and archived and is also available for retrieval
The tape is now archived and available for retrieval.
For each backup Veeam creates a catalog that is also helping for faster restores and to browse the content of the tapes in this case. As expected the Veeam Tape backup job included the Full and Incremental files from the chain.
From Backup jobs to tape it is possible to restore Full VMs and even run file level recovery with auxiliary staging areas that can be any network share and even existing Backup Repositories.
Once I have tapes archived in AWS for weekly, monthly, etc, does that tape/tape drive in the Veeam server disappear? What if I need more tapes for the upcoming weekly tapes. Do I have to create more tapes, thus creating more tape drives? Is it possible for me to have 50+ tape drives one day?
Hi Justin,
Thanks for your comment. Tapes do not disappear from Veeam server and you can still see the content in there (eg file share, backup jobs..)
You can always create/edit the VTL tape configuration to fit your needs in terms of space and rotation. For example by reusing VTL that have expired content. I’m not sure there is a limit of the max number of tapes per day. In the end depends on the backup to complete in the desired time window.
Hope this helps,
Michele
Thanks. That does help with the used tapes and how I would see them to restore. Do you have to have the same number of drives that you do tapes? It seems kind of clunky to have 25+ tape drives in the library, when I only need about 3/4 drives to complete the jobs.
Hi Justin,
Glad it helps. By all means you don’t need to have 25+ tapes drives 🙂 AWS Storage Gateway just comes with 10 Drives by default and up to 1600 Tapes if I’m not mistaken. Of course you can adjust these to your requirements! Same applies to traditional Tapes Libraries.
Kind Regards,
Michele
Perfect, thanks for the clarification! This guide was very helpful.
Thanks for your comment Justin, really appreciated!