vSAN 8 Original Storage Architecture vs Express Storage Architecture differences

VMware vSAN 8 is revolutionary on many fronts and represents the most modern and technilogically advanced vSAN to date for running producton critical workloads. One of the new advancements with vSAN 8 is the new Express Storage Architecture. It is a new way that vSAN 8 handles storage for vSAN, providing many benefits and new capabilities for customers. It also unlocks new hardware capabilities found in modern hardware. However, questions may arise in how the new Express Storage Architecture is different from the Original Storage Architecture. Let’s look at vSAN 8 Original Storage Architecture vs Express Storage Architecture differences and see what we can learn about this new storage architecture.

Look at my vSAN post here:

• VMware vSAN 8 – What’s New? Technical Deep Dive

Why the changes between Original Storage Architecture vs Express Storage Architecture differences

One of the primary differences you will note about the Original Storage Architecture vs. the new Express Storage Architecture is the older Original Storage Architecture is based on a two-tier approach to configuring your storage.

As we know with the current configuration of vSAN, we configure both a cache tier and a capacity tier. VMware makes note that with the Original Storage Architecture, it was designed for traditional hardware devices and not with today’s modern hardware devices and capabilities.

As we remember, a few years ago, spinning disks were still very common in the enterprise data center, while flash devices were more limited and very expensive. However, fast forwarding a few years down the road, flash devices are commonplace and even cheaper in many cases than traditional hard disks.

Today is all about Flash, NVMe in particular

Now organizations have access to high-performance NVMe devices and NVMe-based TLC flash. NVMe flash is extremely performant and provides access to the highest levels of IOP capabilities. With the Original Storage Architecture, since it was designed for the traditional types of storage technologies, it does not provide the fullest capabilities and performance when using these modern storage devices, such as NVMe.

ESA designed from the ground up for NVMe

The Express Storage Architecture was designed from the ground up to take advantage of the newest storage technologies. Since most organizations are now able to use flash devices across the board, the two-tier architecture is not as efficient and it makes less sense to keep this architecture when using these modern devices.

However, with vSAN 8, VMware allows customers to choose which architecture they want to use to move forward. The beauty of vSAN 8 is you can continue to run the Original Storage Architecture side-by-side with clusters that are running Express Storage Architecture. You get to choose.

Change from Disk Groups vs. Storage Pools

A major pivot from the Original Storage Architecture vs. Express Storage Architecture is the change from Disk Groups vs. Storage Pools. What that means with the Express Storage Architecture is you no longer need to have a dedicated cache device as we do today in a traditional disk group.

As we mentioned earlier, the shift to new extremely fast storage devices, such as NVMe, makes the use of a cache drive per se less useful or altogether irrelevant. In addition, VMware has introduced a new log-structured file system and other optimizations into the storage stack that greatly reduces write amplification and reduces write latency.

This provides the tremendous benefit of being able to use the previous cache devices as part of the capacity contributed to vSAN. With no cache device, you can use the capacity of all disks used by vSAN. it also brings the additional benefit of more resiliency for the vSAN datastore.

Previously, with the Original Storage Architecture, when you lose a cache device, you look the entire disk group. Now, since the cache tier is eliminated, the scope of a drive failure is now limited down to the single failed device, instead of the entire disk group.

New vSAN 8 Express Storage Architecture requirements

There are additional requirements from a hardware perspective for the new Express Storage Architecture that customers need to be aware of for meeting the performance criteria VMware has set forth for the Express Storage Architecture. Note the table below:

As you see in the table above, there are now a minimum of (4) storage devices in the Express Storage Architecture (ESA) vs the default of (2) in the Original Storage Architecture model. VMware has chosen the new minimums for a reason with the ESA. They want to ensure customers achieve the highest levels of performance possible with the new architecture and maximize the benefits of ESA.

What about home lab environments?

If you are like me, you are chomping at the bit to get this in your lab environment. However, with the new requirements and the continued emphasis on this new architecture only being supported on “ready nodes,” is it going to be possible to run this on lab gear?

I had posted a question out on Twitter in a group with guys like John Nicholson and William Lam – what about home lab hardware? Can we ignore the hardware checks for the “supported” use of Express Storage Architecture for running this new architecture in home lab environments? The answer is yes.

You can for non-supported lab usage ignore the hardware checks for the new Express Storage Architecture and run your lab gear with the new ESA and realize the performance and other benefits for your lab usage.

vSAN 8 Express Storage Architecture Performance improvements

Let’s talk about the performance of the new architecture when compared to the original storage architecture found in traditional versions of vSAN. The new Express Storage Architecture contains dramatic performance improvements. When you combine the dramatic improvements frmo the use of the new Log Structured File System and the change of where things exist in the storage stack, vSAN 8 performance with the ESA is exponentially faster than the Original Storage Architecture.

Per VM compression

You can now introduce compression on a per-virtual machine basis, instead of only at the cluster level. Data is also compressed before it traverses the network which leads to increased throughput and less networking overhead since the data payload is smaller.

New native snapshots (more detail later)

We can’t talk about performance improvements with the new vSAN 8 Express Storage Architecture without mentioning the new native snapshot feature built into the new vSAN 8 offering. The vSAN 8 Express Storage Architecture native snapshot feature is up to 100x improved performance and is said to have minimal performance impact when creating or running virtual machines with snapshots.

Capacity improvements with the Express Storage Architecture

Capacity benefits have also been greatly enhanced with the new vSAN 8 Express Storage Architecture. First, with the new vSAN 8 Express Storage Architecture, VMware is now able to deliver RAID-1 performance with a RAID-6 datastore.

New 3-node RAID 5!!!

It also now opens the capability to have RAID 5 support for clusters as small as a 3-node cluster and more capacity efficiency on larger clusters. With the ability to use RAID 5/6 in most use cases and the elimination of the cache drive altogether, VMware has brought down the total cost of ownership and cost per gigabyte with vSAN 8 and the new Express Storage Architecture.

VMware has also improved the compression algorithms such that you can see as much as an 8:1 compression ration for ever 4KB block written. How does this compare with the Original Storage Architecture? With the OSA, 4:1 was a typical compression ratio seen by customers.

Express Storage Architecture – New Native Snapshots

The new native snapshot capability is worth talking about in a detailed way since this will be a game changer when it comes to how snapshots are used on virtual machines in production. The Original Storage Architecture still makes use of the original snapshot techology of early vSphere releases in the form of the redo-based log snapshot architecture.

With vSAN 6, VMware did introduce improvements in the form of the vsanSparse snapshots which helped with performance and other challenges. However, it did not eliminate all the challenges seen with snapshots in general on vSAN.

Redesigned snapshots in ESA

With the new Express Storage Architecture in vSAN 8, VMware has totally redesigned the snapshot mechanism so that snapshots can be created much faster with as much as a 100X improvement in consolidation times. What’s more, the new native snapshot mechanism does not require retooling backup tools and other solutions that leverage snapshots.

It is a drop-in replacement to traditional snapshot mechanisms used with earlier vSphere releases. Also, VMware has wired these into the same backup APIs, making it a seamless replacement once customers upgrade to vSAN 8 with ESA.

Security improvements with vSAN 8 Express Storage Architecture

Security has also seen benefits with the new Express Storage Architecture. Like compression, VMware has moved the vSAN Encryption process much higher in the storage stack so that data only needs encrypted once at rest. It no longer has to encrypt and decrypt multiple times as was required for example when data needed to be decrypted to perform compression when moving from cache to capacity devices in the Original Storage Architecture.

This change helps to reduce the impact on the CPU and lowers the I/O amplification of using encryption with vSAN. You can also add in data encryption on top of the vSAN datastore encryption for additional security.

Video lightboard series on the new vSAN 8 Express Storage Architecture

Wrapping Up

When looking at vSAN 8 Original Storage Architecture vs Express Storage Architecture differences, there are many to note. There are so many new enhancements and features with this release, it is going to be a major overhaul for running vSAN in the environment. However, it is great to see that VMware is not forcing customers to adopt ESA vs OSA. Rather, customers can choose which model they want to use. They can stick with OSA for now and choose to go up to ESA later when they choose.