VMware vSAN 8 – What’s New? Technical Deep Dive

VMware vSAN represents one of the most significant improvements of VMware vSAN since it was introduced in 2014. The industry has evolved in storage and across the board from a technology perspective. With the hardware of today and the focus on the cloud, businesses have new strategies and storage requirements. VMware vSAN is uniquely positioned to help organizations to meet the modern demands of the modern enterprise data center. Let’s take a deep dive into VMware vSAN 8, what’s new, and see how it meets the new demands of the data center.

vSAN Express Storage Architecture (ESA) in vSAN 8

What is this? It is an optional alternative architecture that actually is available in parallel to the traditional vSAN 8 storage architecture that we have all known in previous versions of vSAN. It unlocks the capabilities of modern hardware. Using today’s modern hardware and hardware architectures, founding vSAN ready-node configurations, vSAN 8 Express storage architecture offers superior levels of performance, scalability, resilience, and data services capabilities. In fulfilling all these missions, performance is not compromised in the least.

Comparing vSAN Express Storage Architecture (ESA) in vSAN 8 with traditional vSAN architecture

Let’s take a step back and see how the traditional vSAN architecture compares with the new vSAN Express Storage Architecture. Let’s look at the traditional storage configuration architecture of traditional vSAN. Note the following characteristics:

Traditional vSAN Storage architecture

• Dedicated cache and capacity devices
• Accommodates older and a wide range of storage devices
• It uses the concept of disk groups to provide performance using caching and capacity drives

Enhanced vSAN Express Storage Architecture

• Flexible architecture using a single tier in vSAN 8
• It is an optimized solution for high-performance NVMe-based flash devices with QLC-based flash
• No disk groups – With the new vSAN architecture, all devices contribute to the caching and capacity storage

The cool thing with the vSAN Express storage architecture is that businesses can transition to the new architecture at the pace that suits their security use cases as needed. How do organizations make use of the new express architecture?

• You must use approved hardware – Ready nodes with the supported hardware
• Verify licensing as it requires vSAN Advanced or Enterprise licensing
• Migrate your virtual machines – you can migrate your virtual machines to a new cluster that supports the vSAN express architecture

VMware recommends that organizations upgrade to vSAN 8 regardless of whether they upgrade to the vSAN express architecture from the start. Even though the vSAN express architecture represents the ultimate performance that can be achieved with the vSAN HCI solution and modern hardware, customers upgrading to vSAN 8, even with traditional vSAN storage architecture, will receive many benefits to their vSAN HCI environment.

What is vSan Express Architecture?

The new vSAN Express architecture is a modern way to process and store data providing a new fast and efficient datapath paired with efficient data structure. Note the following advantages and features:

• It is highly adaptable to modern hardware
• it stores data efficiently and meets the performance requirements needed
• It provides more resilience
• It provides new scalable, high-performance snapshots with a new snapshot engine
• It simplifies administration and serviceability

Did VMware totally rewrite vSAN to achieve this new vSAN Express architecture? No, this was built on the same platform of vSAN that we know traditionally. However, using the robust architecture that vSAN already is, allowed for these significant improvements to be introduced.

New Log-Structured File System

VMware has introduced a patented file system they are calling the Log-structured file system. It is a new file system specifically writtent to ingest writes very quickly and efficiently, especially on flash devices. It reduces the compute and network resources needed to implement the vSAN 8 solution. Note the features:

• It reduces I/O amplification
• It provides low overhead
• It is compatible with future device types coming down the line

Optimized Data Structure

The next part of the architecture is the optimized data structure and I/O engine. It allows storing data in a fast and efficient way. Combined with the new log-structured file system, it allows vSAN to store data with resilience and efficiency.

Again it is not a rewrite of vSAN but rather patented enhancements that offer these new capabilities, including native snapshots.

vSAN LFS – vSAN Structured Log File system

It is a new layer that ingests new data quickly and efficiently. It prepares data to be saved in an optimized, space-efficient way, leading to reduced I/O amplification, minimized wear of devices, and is already compressed, etc.

It allows for writing many small writes efficiently, combining them, and writing a large block efficiently. It is compressed, combined, and packaged with metadata and written to a persistent log, and written to what is known as the performance leg, returning the write acknowledgment to the VM immediately. This keeps latency low.

As data accumulates in the durable log, the payload from the previous log entries are written to the capacity leg, using RAID 6 erasure coding. The metadata is sent to the metadata log keeping it available. When it fills up, it is retained in a B-tree structure on the capacity leg. The new LFS enables delivering RAID6 efficiency with RAID1 perfromance.

New Log Structured Object Manager in vSAN ESA (LSOM)

As you can imagine, delivering highest levels of performance with NVMe device required several enhancements. These need to happen at the lowest level of the stack, the Log Structured Object Manager.

It is built around a high-performance block engine and a key-value store. It delivers large write payloads while minimizing the overhead of the metadata writes. These are critical to allow higher-level upper levels of vSAN to send data to devices without contention.

It also removes the concept of disk groups. It eliminates a failure domain and makes adding, removing, and servicing the storage after a failure much easier and more efficient.

New object format for performance and capacity components

The new vSAN Express storage architecture stores and distributes the data in a slightly different way than when using the traditional vSAN architecture. It uses two “legs” to store data. These are both contained within the same object and uses the same underlying distributed object manager.

• Performance leg
  • Ingests writes quickly
  • Prepares data for efficient full stripe write
  • Always a RAID-1 mirror
  • FTT of mirror equal to prescribed storage policy
  • Resides on same hosts as capacity leg components
• Capacity leg
  • Combined data is written as an efficient full stripe write
  • Reflects prescribed storage policy (RAID-5, RAID-6, etc)

Again it uses the same tier of storage without disk groups.

vSAN Express Architecture and Capabilities using Modern Hardware

The vSAN Express Architecture is more than just a discreet set of enhancements delivered in vSAN 8. VMware has taken a step back and looked at how data could be processed more efficiently in the stack overall.

Data services have been moved to the very top of the vSAN stack to remove some of the context-switching and resources needed in previous versions of vSAN. Now, data is compressed and encrypted at the highest layer to minimize process amplification. It takes advantage of the new log-structured file system by ingesting small I/Os and writing these as one large I/O. This combination makes the Express storage architecture really great.

Single tier of storage instead of the two tiers previously

The new Express Storage architecture is optimized for today’s hardware. It has been re-engineered for today’s storage devices, and means that all devices used will contribute to both the capacity and performance (caching) tiers of a vSAN cluster.

The admin simply claims the devices and creates a single storage pool per host, contributing its resources to the storage resources of the cluster. It streamlines the administration of a vSAN cluster and leads to better serviceability, simple user experience, and availability of storage devices.

Space Efficient RAID-5/6 coding at the performance of RAID-1

Administrators can now use RAID 6 to gain enhanced levels of resilience. The new Express Storage Architecture provides the best capacity and performance. It delivers on that with vSAN 8. Admins can use RAID-6 for resilience and space and still achieve the performance of RAID-1.

Adaptive RAID-5 to accommodate cluster conditions

VMware has introduced a new adaptive RAID-5 erasure coding with the new Express Storage Architecture. It provides much more intelligence and efficiency compared to previous vSAN releases before vSAN 8. It is a more intelligent approach.

When choosing RAID-5 erasure coding, ESA looks at the host count. It decides which one of two erasure coding policies it will use for data placement in the vSAN 8 cluster. A new 4+1 RAID-5 placement scheme spreads data with parity across 5 hosts. It is used in clusters with 6 or more hosts.

A new 2+1 RAID-5 data placement scheme spreads data with parity across 3 hosts. If a cluster host count is reduced below 6 hosts for a while, the ESA will automatically adjust itself to the new 2+1 scheme to ensure it retains resiliency with one spare host in line with the number of hosts left to protect against subsequent failures.

Since it delivers the same performance using erasure coding as RAID-1 mirroring policies, customers can by default choose this setting at a more efficient level with higher resilience that adapts to the cluster.

Improved space efficiency in smaller clusters:

• 4+1 scheme uses 1.25x capacity for resilient data storage
• 2+1 scheme uses 1.5x capacity for resilient data storage

Automatically adjusts RAID-5 scheme to accommodate cluster size

• 4+1 scheme – used with 6 or more hosts
• 2+1 scheme – used with 3-5 hosts

Storage Policy-based Compression New Enhancements

In vSAN 8, VMware has redesigned how vSAN compression works with the Express Storage Architecture. New technologies are in place to improve data reduction of the compression feature to as high as 8 to 1 compression ratio. This is for every 4K block written, which is a 4X improvement.

It occurs in the upper layers of the vSAN 8 stack as it receives the incoming writes. This reduces the CPU effort for compression and reduces network traffic. Replica traffic is always compressed. It is enabled by default. However, it can be turned off on a per VM or per VMDK basis using storage policies.

Cluster-based encryption with vSAN ESA

The new Express Storage Architecture takes a new approach to encryption. It is still a per-cluster feature. Now it happens in the upper layers of the storage stack in the vSAN 8 architecture, like the new compression functionality. Encryption happens after compression.

It means all vSAN traffic between hosts will be encrypted as well. It means better security and less performance impact leading to improvements all around.

Availability and serviceability improvements

The new vSAN 8 Express Storage Architecture, deployments, and vSAN operations are easier and more flexible. The new storage policies will be more resilient and space-efficient data placement schemes without performance compromise. The ESA eliminates disk groups, fault domains have been reduced, leading to less impact when a failure occurs. It promotes the serviceability of the solution.

Additional checks ensure it runs on certified hardware to provide a simpler management experience for vSAN 8 administration.

vSAN Adaptive network traffic shaping for resynchronization

With the space efficiency of the ESA, storage can be processed at higher rates. This can lead to the network being the bottleneck when running high-performance workloads. In the Express Storage Architecture in vSAN 8.

The Express Storage Architecture in vSAN 8 includes an adaptive traffic shaping feature helping to ensure that when network contention occurs, vSAN 8 will prioritize virtual machine traffic over the resynchronization activity to prevent saturating a network link.

New Native vSAN Snapshots!

For quite some time now, we have been waiting for more efficient snapshot mechanisms for vSAN. This has now been delivered using the Express Storage Architecture. VMware vSAN 8 can manage the underlying data in new, more efficient ways.

There is a new native snapshot engine that is highly scalable and efficient. VI admins can now create point-in-time data states with minimal performance degradation for a virtual machine, no matter how many snapshots exist on a virtual machine.

It continues to support the traditional VADP backup functionality that will continue to allow broad backup partner community vendors to allow leveraging the new native vSAN 8 snapshot functionality as part of backups.

It is a foundational part of the new vSAN 8 and will allow more agility with data and it will undoubtedly introduce new use cases.

vSAN Proactive Insights – New features

One of the tremendous strengths of the VMware vSAN solution is the proactive health insights it provides to administrators. It allows vSAN to be a seamless, proactive, and easy-to-use HCI solution providing many benefits.

In past versions of vSAN, the customer experience improvement program received cloud-connected health checks. Not all environments can enroll in CEIP. With the vSAN proactive insights capabilities, customers who have a vCenter Server connected to the Internet but are not enrolled in the CEIP receive the latest cloud-connected health checks.

Increased maximum capacity on devices in cache/buffer tier for original architecture

So far, we have concentrated on the new Express Storage Architecture. However, there are vast improvements to the original architecture, including increased maximum capacity on devices in cache/buffer tier.

Note the following:

• The logical buffer capacity limit has been increased from 600 GB to 1.6TB per disk group
• Ingests larger amounts of write operations at burst rate levels of performance
• Improves performance consistency for environments with larger working sets
• Reduces I/O demand on capacity tier
• Applies to vSAN OSA all-flash configurations

Wrapping Up

The new VMware vSAN 8 release represents a powerful new release of VMware vSAN that transcends even the great performance we have seen with the previous releases of vSAN, including vSAN 7.0 Update 3. The new modern Express Storage Architecture, departing from the legacy disk group technology will be a great new architecture allowing businesses to take advantage of modern storage hardware and capabilities. It will help to simplify storage configuration and offers many new features as noted above with vSAN 8. However, VMware has not left out the original architecture in terms of improvements. Even for customers who will stick with the original architecture, there will be many performance, capacity, and space efficiency improvements. The new Express Storage Architecture is certainly the way forward for vSAN storage.

Read more about vSAN 8 here: What’s New with vSAN 8 | HCI | VMware

Take a look at my other coverage of VMware Explore announcements here:

• VMware vSphere 8 vSAN 8 and VMware Cloud Foundation+ Released New Features – Virtualization Howto
• What is VMware Aria? – Virtualization Howto
• What is VMware Project Northstar VMware Project Trinidad and VMware Project Watch? – Virtualization Howto
• VMware Anywhere Workspace Announcements at VMware Explore 2022 – Virtualization Howto