Move VMware vSAN 6.7 Stretched Cluster to Different vCenter Server

In my home lab I have had several projects going on that have been slowly taking place over the past few weeks. One of those projects was getting my “prodution” vSphere vSAN cluster upgraded to vSphere 6.7 across the board – VCSA appliance, ESXi hosts, and vSAN disk format. After getting that accomplished, I wanted to switch out the VCSA appliance due to some legacy naming and configuration that I had in place from past lab environments and such that I wanted to get rid of. VMware has made the ease of deploying, upgrading, and migrating the VCSA appliance extremely easy. My plan of action after having already upgraded the vSAN environment to vSAN 6.7, including on-disk format version, etc, was to change out the VCSA appliance for a new, fresh VCSA appliance, different SSO domain, etc. Let’s take a look at the process to Move VMware vSAN 6.7 stretched cluster to different vCenter Server and see how this process went in the home lab.

Move VMware vSAN 6.7 Stretched Cluster to Different vCenter Server

To begin with, there were a few really great resources that I relied on for the initial information gathering phase of the process involved:

• William Lam’s Post – How to move a VSAN Cluster from one vCenter Server to another?
• Virtual Blocks Blog – Replacing a vCenter server for existing vSAN hosts
• Moving an ESX/ESXi host with vDS from one vCenter Server to another (1029498)

With the above informational resources, you should have a pretty complete picture of what to expect with moving a vSAN cluster.

It may seem counter intuitive that moving the vSAN cluster from one vCenter to another vCenter server (VCSA to VCSA) without disrupting running workloads.  However, the process should be fairly uneventful for your environment as it was for mine.  Below are a few things to note and consider, including a stretched cluster configuration.

• Take heed if you are running distributed switches as these will need to be restored in the new VCSA environment
• Keep your backup solution in mind.  Most tie into vCenter Server and so changing your VCSA appliance will change where your backup solution is pointing
• Make sure you have backups – even when things seem relatively non disruptive, always count on the worst before you do anything major.
• Right before the move, disable/pause your backup jobs during the move as most likely you would encounter failures, errors during the move.  This helps to keep your data protection environment clean.

As far as the steps for the process, I will defer to the steps that William Lam posted from VMware support:

1. Copy all VDS settings to new cluster – ***Note*** see the above listed VMware KB
2. Enable vSAN on new cluster
3. Disable stretched cluster
4. Move each host
5. Move witness
6. Re-enable stretched cluster

Also mentioned from William’s post was the disabling and enabling of the VSAN.IgnoreClusterMemberListUpdates setting with PowerCLI.  I did not find any information on if this is still needed with vSAN 6.7, however, I went ahead and disabled the setting before moving and re-enabled the setting after the hosts were moved and didn’t see any issues.

Before moving the hosts (I did this connected to the “old” VCSA appliance):

Foreach ($vmhost in (Get-Cluster -Name VSAN-Cluster | Get-VMHost)) {
$vmhost | Get-AdvancedSetting -Name "VSAN.IgnoreClusterMemberListUpdates" | Set-AdvancedSetting -Value 1 -Confirm:$false
}

Disable the VMware vSAN Stretched Cluster

Since in this environment I was not running vDS switches, I was able to get straight to the vSAN specific operations.  Firstly, I disabled (removed) the Witness host from the vSAN stretched cluster.  You can do this by clicking your cluster and clicking Configure >> vSAN >> Fault Domains & Stretched Cluster >> Disable.

This process does not take too long and the vSAN cluster is reconfigured.  You should eventually see the Status as Disabled.

Moving VMware vSAN ESXi Hosts to a new vCenter Server

The ESXi vSAN hosts in the source cluster resided in a vSphere 6.7 environment.  The new VCSA appliance that I deployed was the VCSA 6.7b server – VMware-VCSA-all-6.7.0-8832884.iso.  After deploying the new VCSA appliance and verifying connectivity, logging in, etc, I was ready to begin configuring the new environment.

Configure an empty vSAN cluster

As outlined earlier, I created a new vSAN enabled cluster in the new vCenter environment.  One to remember is it is best to mirror your settings from your previous vSAN cluster, including Deduplication and Compression, etc.  If you don’t, you will see vSAN cluster configuration consistency errors – however, this is not a show stopper as I found out, you can simply bring the settings into consistency in the new vSAN cluster and the error will go away.  As you can see below, I have configured a new vsancluster1 cluster with vSAN.

To move a host from one vCenter to another, you can simply Add host in the new vCenter server and the host will be disconnected and removed from the source vCenter in the process (this is the quickest way to do this and involves one operation instead of two).

Here, I added the first host to the new vSAN cluster and as you can see it adds successfully, with production workloads still running, unaffected.

However, as expected, the vSAN Monitor starts bleeding with problems.  For now though, ignoring this until the configuration is complete on the new vCenter Server.

Next, I addeded the second ESXi vSAN host to the new cluster.

Now, for the witness host.  Note Prior to the steps below, I added the Witness Host to the new vCenter environment.  Navigate to Configure >> Fault Domains & Stretched Cluster >> Configure.  This will begin the Configure vSAN Stretched Cluster wizard.

Running through the wizard, you configure fault domains, and select the Witness Host.  Compatibility checks should be successful.

Ok, now for some cleanup.  After all the hosts were added, I began looking at the failures.  After the hosts were added, the Data failure and vSAN object health failure went away after some time once objects resynced, etc.  However, I still had the vCenter state is authorized failure along with the Different VC error showing.  To resolve this, click the Update ESXi configuration from the screen below.

A note here as described in the dialog box, this overrides existing vSAN cluster configuration on all the hosts in the vSAN cluster.  You want to make sure everything is configured as you want along with all the hosts added to the cluster as this will result in hosts being removed by forcing the configuration update.

After a couple of minutes, the Failed states went away and I was back to minor warnings, at least for a lab environment – Hardware compatibility and vSAN Build Recommendation due to my.vmware.com login credentails not being configured.  Everything is running healthy and back to normal for the lab.

Finally, (again not 100% sure this is needed with vSAN 6.7) I ran the PowerCLI script to set the VSAN.IgnoreClusterMemberListUpdates setting back to “0”.

After moving the hosts (I did this connected to the new VCSA appliance):

Foreach ($vmhost in (Get-Cluster -Name VSAN-Cluster | Get-VMHost)) {
$vmhost | Get-AdvancedSetting -Name "VSAN.IgnoreClusterMemberListUpdates" | Set-AdvancedSetting -Value 0 -Confirm:$false
}

Takeaways

It is extremely cool to see software defined storage work this well and be resilient to drastic changes in the environment.  VMware has certainly thought through everything extremely well when it comes to vSAN and your data.  The reliance on vCenter is actually fairly minor once vSAN is up and running, so it makes the process of changing out the vCenter server fairly easy.  However, keep in mind that every environment is different needs to be approached as such.  Make sure you have current backups of everything and that you have a game plan and have covered all the bases such as restoring your distributed switches, etc.  As shown here the process to Move VMware vSAN 6.7 Stretched Cluster to Different vCenter Server was uneventful and went well for me in the home lab environment.  More home lab snippets to come as projects are tackled!