September 22, 2016
A Closer Look At Datto’s Linux Backup Agent
Datto’s Linux backup agent allows users to take live, image-based backups of servers running the popular open source operating system. Creating the agent was a considerable undertaking that involved months of development. “Here’s the challenge,” said Tom Caputi, a software engineer at Datto. “You have this large set of 0s and 1s, which is constantly changing. How do you capture a point in time, while the server is still in use?”
There are a few basic open source tools available for creating instant copy-on-write snapshots of Linux servers. However, these tools did not meet our needs, because they do not support live server snapshotting. To create a backup using one of these tools, you must first unmount the data volume you are protecting, take and mount the snapshot, then copy the volume to a secondary location. This is obviously impractical, because production servers typically can not be taken offline for the amount of time required to perform these tasks. These tools also require a spare volume on the server for snapshots. Many production servers lack a pre-configured spare volume.
To overcome these issues, Caputi developed the part of the agent known as the Datto Block Driver. The driver creates live snapshots of production servers, while keeping the underlying volume running and available—think Microsoft VSS on Windows but for Linux. “Each individual write in the pipeline is stopped briefly while the driver reads the data that is about to be overwritten. This data is then moved to our copy-on-write file which allows us to keep track of all the original data,” said Caputi. “After the initial snapshot, the driver tracks incremental changes and copies the block addresses that change, not the blocks themselves, which is a lot more efficient.” The copy-on-write file resides on the source block device itself, eliminating the need for a spare volume.
The Datto Block Driver, a library for the agent, and several utilities have been open sourced, and you can download the source code free on GitHub. “It’s picked up some popularity,” said Caputi. “It’s got a couple hundred stars on GitHub.” The open source community involvement has lead to better support for Linux distributions, as well as faster maintenance and improvements for supported distributions. The Datto Linux agent currently supports a wide variety of Linux distributions, including:
- CentOS 6 and 7
- Debian 7 and 8
- Fedora 20, 21, 22, 23, and 24
- openSUSE 13.1, 13.2, and Leap 42.1
- Red Hat Enterprise Linux 6 and 7
- RHEL 6 and 7
- SUSE Linux Enterprise 11 sp4 and 12
- Ubuntu 12.04, 14.04, and 16.04
Datto’s Linux backup agent has become an important piece of Datto’s Total Data Protection platform. “Many of our partners are primarily Windows shops, but lots of them have a Linux server or two they need to protect,” Caputi said. “It also allows them to protect larger servers, which frequently Linux based.” The agent offers file restore, local virtualisation, bare metal restore and image export capabilities.