Advanced Storage

Datto uses ZFS both on our devices and in the Datto Cloud for data integrity and efficient data compression.

How does it work?

Datto is a leading adopter of ZFS (the Zettabyte File System) both for backup storage in Datto appliances and in the Datto Cloud

ZFS is an advanced file system that is combined with a logical volume manager that, unlike a conventional disk file system, is specifically engineered to overcome the performance and data integrity limitations that are unique to each type of storage device.

ZFS provides copy-on-write snapshots, zero-copy writable clones, data compression and deduplication, support for massive storage capacities, as well as continuous integrity checking and automatic data repair.

With ZFS and Datto’s Inverse Chain Technology working together, Datto delivers the most advanced and complete BDR appliances and DraaS solutions available.

Datto's Advanced Storage technology with ZFS provides:

Data Integrity Protection

End-to-end checksums and data authentication to detect silent data corruption and automatically repair integrity

SSD accelerated and flash optimized:

Production-level storage performance from backup storage, while maximizing SSD service life for the world’s first all-flash backup

Maximum Backup Throughput:

Fast backups, even with a limited number of drives, via Write coalescing and an intelligent RAID

Storage Efficiency:

Including inline data compression, variable-length blocks, and inline block deduplication (Datto NAS)

ZFS Architecture

The most powerful, flexible and trusted file system available.

A commitment to ZFS

When it came time to select the most capable file system to embed into all of its systems, Datto chose ZFS

Physical and virtual Datto SIRIS, ALTO, and Datto NAS all use ZFS for internal storage. In addition, the nodes utilized in the Datto Cloud for DRaaS, Datto Drive, and Backupify, all use ZFS as well.

Integration between ZFS and backup management enables Datto to create an inverse backup chain to dramatically reduce capacity consumption by utilizing snapshots in the backup storage. VMs can be instantly virtualized and spun up quickly since Datto keeps recovery point in a fully constructed state, allowing VMs to be created using writable clones

Data Integration Protection

ZFS Integrity Protection

The system checks data at multiple levels to ensure the retrieved files are perfect copies of those that were backed up

Protecting data demands more than RAID and replication. When backup data is accessed, it must be certain that the data retrieved is exactly the same data that was written.

Conventional RAID approaches, file systems, and drive block checksums fail to handle such scenarios. Backup storage demands an advanced disk file system that can handle the reality that drives (HDDs and SSDs) are highly complex and function in an imperfect world.

Datto uses ZFS, which includes end-to-end checksums and data authentication at multiple levels in its file structure. ZFS is key to protecting data integrity by detecting and addressing silent data corruption scenarios, including:

  • Phantom writes
  • Data corruption on the drive
  • Misdirected reads
  • Accidental overwrites

ZFS is a self-healing file system that can automatically detect silent data corruption and immediately repair the damaged copy on the drive.

Silent data corruption is discovered when block A is read

The mirrored version of block A is read, and is determined to be healthy

ZFS sends verified version of block A to the application, and fixes failed copy on the drive

ZFS data protection and authentication protects data integrity along the entire I/O path. End-to-end checksums ensure that data integrity is not only protected on the drives, but also while it moves from drives into main memory. Controller bugs and DMA parity errors have met their match with ZFS.

ZFS Caching Levels

Using multiple levels of caching, performance of random reads is optimized.

SSD accelerated and flash optimized

Hard drives, when managed properly, support the high IO throughput that is demanded from backup storage. But BDR appliances do not use their storage simply for backup and restore.

Instant Virtualization and screenshot backup verification demand more random reads from the Datto BDR - a workload profile that is more aligned with production than backup. To balance the needs of both, Datto uses ZFS to provide SIRIS appliances that are both hybrid flash and all-flash.

ZFS uses several layers (or levels) of caching for performance optimization of random reads. At the top is the Level 1 Adaptive Replacement Cache (“ARC”), which is data cached within the system’s DRAM. This cache is accessible in nanoseconds, but is the most limited and expensive. The next cache level is the Level 2 ARC (“L2ARC”), which uses flash storage such as an SSD for a much larger cache that is not subject to the limited random IOPS a hard drive can support.

Datto S3X1

The world’s first all-flash backup and BDR appliance

The end result of using hybrid flash within Datto BDR and NAS appliances is average latency reductions of up to 90% compared to HDDs alone - with 5x to 10x improvement for 8K random read IOPS. Naturally, result will vary by model, and by the size of the working set.

In 2016, Datto was the first IT vendor to design and ship all-flash BDR appliances, effectively also being the world’s first all-flash backup appliance. These appliances handle the high-throughput demands of backup while also providing maximum random read IOPS, for local virtualization that keeps pace with almost any primary storage.

Backup is a write-demanding IO profile that can be taxing on SSD life if not properly managed. Datto’s unique all-flash advantage is how ICT only writes base full backup images once, instead of making full copies week after week. ICT also avoids the excessive wear that would occur with reverse incrementals, and their associated regular synthesis of backups.

File sharing on Datto BDR appliances, including NAS shares, iSCSI shares, and Datto Drive, all will have active workloads that benefit from flash performance. And now, Datto has added all-flash NAS with the Datto NAS 3.

Storage Efficiency

Terabytes are not all created equal. Datto can often protect the same number of backup clients as an alternative solution, using less than half the capacity, while actually keeping more recovery points on premises within the BDR appliance (for better RPO)

TDatto BDR devices maintain months of backups on premises, protected by pre-existing firewalls, security protocols, and encryption. Devices also replicate all backups (near and long-term) to two remote Datto Cloud locations. The hybrid cloud solution ensures classic “3-2-1 rule” resiliency, but without the architecture costs and planning that traditionally put this out of reach of most businesses.

All Datto appliances and cloud nodes uses the advanced ZFS file system both onsite and in the Datto Cloud. ZFS guarantees protection from drive failures, but more importantly protects against silent data corruption. As a DRaaS solution, the Datto’s Hybrid Cloud Platform also provides the flexibility of Instant Virtualization of backup clients using the compute from the local BDR appliance, or in the Datto Cloud, for simple yet maximum availability for BC/DR. The robust 300 PB Datto Cloud also provides cloud storage for Datto Backupify and Datto Drive.

ZFS is the foundation of achieving this storage efficiency on Datto BDR appliances and NAS appliances, due these characteristics and capabilities:

  • ZFS data compression
  • Variable length blocks on drives
  • Block-level data deduplication (now for Datto NAS 3)
  • No more RAID mirroring (RAID 10) with a large number of drives

The complete story of Datto’s storage efficiency is clearer when considering how Inverse Chain Technology functions, and how it integrates with and leverages ZFS. Those benefits include:

  • No more wasteful duplication or copying of weekly full backups
  • One copy provides both backup and instant virtualization or “instant restore”
  • Unified storage and backup with Datto NAS 3 and NAS shares
  • Backup Insights can locate any file within backup images without a file index database

Case Study: Avoiding Downtime Down Under

Discover how a proper BCDR not only offers peace of mind for the end user, but for the MSP as well:

An Insurance Policy for your Data

With an all-in-one solution, bolstered by technology like Screenshot Backup Verification, Addictive Technologies takes the guessing game out of BCDR. Now that we have implemented Datto, we know that the restores are happening and being tested daily by the Screenshot technology... Naturally, we all sleep a lot better now.

Mark Calzone

Robert Marsden

Managing Director of Addictive Technology Solutions Southport, Queensland Australia, serving Small to Medium-size Businesses.