Windows 8 Storage Improvements

At the end of November the Microsoft Server and Cloud Platform Team posted two articles about the upcoming storage changes in Windows 8. The highlights from the blog are as follows, the links below will take you to the original posts which have more information.

Windows 8 Platform Storage – Part 1
Windows 8 Platform Storage – Part 2

  • Improved NTFS metadata integrity on commodity storage – Beginning with Windows 8, NTFS has been enhanced to solely rely on the “flush” command in lieu of “forced unit access” (aka write-through) for all operations that require write ordering to ensure file system metadata integrity. This enhancement decreases the possibility of metadata inconsistency due to unexpected power loss on commodity hard disk drives while enabling the disk to cache associated data as long as safely possible. Cost-effective industry standard SATA drives can, therefore, be more reliably utilized.
  • Maximized NTFS availability through online file system scan and repair – very large volumes (e.g. 64TB) can now be confidently deployed without fear of service downtime associated with file system check/repair operations. Windows 8 performs online scanning for corruptions, online identification of error conditions for on-disk file system data structures, and online repair. Windows 8 now incurs file system downtime only when absolutely needed and such downtime is proportional only to the specific number of (online) detected corruptions. All of these enhancements together deliver significantly improved NTFS uptime.
  • Storage Spaces – Windows Server 8 delivers powerful platform-based storage virtualization capabilities enabling utilization of cost-effective commodity storage hardware for business critical application deployments. Two new concepts are introduced: (a) Storage Pools: these are units of capacity aggregation, administration, and workload isolation (b) Spaces (virtual disks): functionally equivalent to physical disks from the perspective of all users and applications, spaces deliver additional sophisticated capabilities including just-in-time allocation as well as resiliency to physical disk failures
  • Data Deduplication – Windows Server 8 implements state of the art (sub-file) data chunking and compression to deliver optimal data deduplication, and thereby maximize storage capacity utilization for a variety of workloads. The implementation is transparent to the primary workload on the server – this is achieved by minimizing I/O impact through scheduled and selective optimization, and minimizing server impact through low resource (CPU/memory) usage. Further, the implementation strongly optimizes for high data integrity – this is achieved by performing data integrity validation on all data as well as metadata, and by ensuring redundancy for all metadata as well as “popular” data chunks. Windows 8 servers that enable deduplication as well as BranchCache apply the same algorithms both to on-disk and on-the-wire savings, thereby benefiting from faster download times as well as reduced bandwidth utilization over wide area networks.
  • Windows Storage Management API – a comprehensive and extensible WMI v2 based storage management API enables easier “single point” administration and management of heterogeneous storage infrastructure (including Storage Spaces). In-box PowerShell command-lets enable simplicity of end-to-end provisioning and ongoing administration, as well as inherent support for remote execution and scripting. Application developers including those delivering sophisticated storage management solutions (e.g. System Center or equivalent) can now rely on a single Windows API. Storage hardware vendors can enable easy administration of their gear either through the industry standard Storage Management Initiative Specification (SMI-S) or via a Windows host-based Storage Management Provider interface.
  • Offloaded Data Transfer – joint innovation with major external storage array vendors has resulted in Windows Server 8 providing an incredibly efficient mechanism to perform cross-application and cross-machine data transfers.
  • Awareness and utilization of thinly provisioned disks – beginning with Windows Server 8, NTFS identifies thinly provisioned disks and enables transparent capacity reclamation (e.g. when files are deleted) for better capacity utilization. Further, sophisticated applications can also identify thinly provisioned disks through both WMI or via PowerShell, and can invoke commands to enable capacity reclamation when appropriate. Applications are also notified via standard events whenever capacity exhaustion thresholds are crossed on such thinly-provisioned storage, thereby enabling them to take appropriate action.
  • Storage Optimizer – Periodically, without any user intervention, a storage optimizer task completes maintenance activities including intelligent compaction of file system allocation to enable capacity reclamation on thinly provisioned disks (including on thinly provisioned spaces).

Overall there are some big changes coming, it will be very interesting to see these new features in action.