Datto uses ZFS both on our devices and in the Datto Cloud for data integrity and efficient data compression.
Datto is a leading adopter of ZFS (the Zettabyte File System) both for backup storage in Datto appliances and in the Datto Cloud. ZFS is an advanced file system that is combined with a logical volume manager that, unlike a conventional disk file system, is specifically engineered to overcome the performance and data integrity limitations that are unique to each type of storage device.
ZFS provides copy-on-write snapshots, zero-copy writable clones, data compression and deduplication, support for massive storage capacities, as well as continuous integrity checking and automatic data repair. With ZFS and Datto’s Inverse Chain Technology working together, Datto delivers the most advanced and complete BDR appliances and DraaS solutions available.
When it came time to select the most capable file system to embed into all of its systems, Datto chose ZFS. Physical and virtual Datto SIRIS, ALTO, and Datto NAS all use ZFS for internal storage. In addition, the nodes utilized in the Datto Cloud for DRaaS, Datto Drive, and Backupify, all use ZFS as well.
Integration between ZFS and backup management enables Datto to create an inverse backup chain to dramatically reduce capacity consumption by utilizing snapshots in the backup storage. VMs can be instantly virtualized and spun up quickly since Datto keeps recovery point in a fully constructed state, allowing VMs to be created using writable clones.
Protecting data demands more than RAID and replication. When backup data is accessed, it must be certain that the data retrieved is exactly the same data that was written. Conventional RAID approaches, file systems, and drive block checksums fail to handle such scenarios. Backup storage demands an advanced disk file system that can handle the reality that drives (HDDs and SSDs) are highly complex and function in an imperfect world.
Datto uses ZFS, which includes end-to-end checksums and data authentication at multiple levels in its file structure. ZFS is key to protecting data integrity by detecting and addressing silent data corruption scenarios, including:
ZFS is a self-healing file system that can automatically detect silent data corruption and immediately repair the damaged copy on the drive.
ZFS data protection and authentication protects data integrity along the entire I/O path. End-to-end checksums ensure that data integrity is not only protected on the drives, but also while it moves from drives into main memory. Controller bugs and DMA parity errors have met their match with ZFS.
Hard drives, when managed properly, support the high IO throughput that is demanded from backup storage. But BDR appliances do not use their storage simply for backup and restore. Instant Virtualization and screenshot backup verification demand more random reads from the Datto BDR - a workload profile that is more aligned with production than backup. To balance the needs of both, Datto uses ZFS to provide SIRIS appliances that are both hybrid flash and all-flash.
ZFS uses several layers (or levels) of caching for performance optimization of random reads. At the top is the Level 1 Adaptive Replacement Cache (“ARC”), which is data cached within the system’s DRAM. This cache is accessible in nanoseconds, but is the most limited and expensive. The next cache level is the Level 2 ARC (“L2ARC”), which uses flash storage such as an SSD for a much larger cache that is not subject to the limited random IOPS a hard drive can support.
The end result of using hybrid flash within Datto BDR and NAS appliances is average latency reductions of up to 90% compared to HDDs alone - with 5x to 10x improvement for 8K random read IOPS. Naturally, result will vary by model, and by the size of the working set.
In 2016, Datto was the first IT vendor to design and ship all-flash BDR appliances, effectively also being the world’s first all-flash backup appliance. These appliances handle the high-throughput demands of backup while also providing maximum random read IOPS, for local virtualization that keeps pace with almost any primary storage.
Backup is a write-demanding IO profile that can be taxing on SSD life if not properly managed. Datto’s unique all-flash advantage is how ICT only writes base full backup images once, instead of making full copies week after week. ICT also avoids the excessive wear that would occur with reverse incrementals, and their associated regular synthesis of backups.
File sharing on Datto BDR appliances, including NAS shares, iSCSI shares, and Datto Drive, all will have active workloads that benefit from flash performance. And now, Datto has added all-flash NAS with the Datto NAS 3.
Datto S3X1 - The world’s first all-flash backup and BDR appliance
Terabytes are not all created equal. Datto can often protect the same number of backup clients as an alternative solution, using less than half the capacity, while actually keeping more recovery points on premises within the BDR appliance (for better RPO). ZFS is the foundation of achieving this storage efficiency on Datto BDR appliances and NAS appliances, due these characteristics and capabilities:
The complete story of Datto’s storage efficiency is clearer when considering how Inverse Chain Technology functions, and how it integrates with and leverages ZFS. Those benefits include:
Discover how a proper BCDR not only offers peace of mind for the end user, but for the MSP as well:
"With an all-in-one solution, bolstered by technology like Screenshot Backup Verification, Addictive Technologies takes the guessing game out of BCDR. Now that we have implemented Datto, we know that the restores are happening and being tested daily by the Screenshot technology... Naturally, we all sleep a lot better now."
— Robert Marsden, Managing Director of Addictive Technology Solutions Southport, Queensland Australia, serving Small to Medium-size Businesses.
We'd love to help