I am a big vSAN fan and use it in my own Home Lab for most of my VM’s (main exception being VM’s used for backing up … they are on my QNAP fileserver connected via iSCSI). My vSAN cluster configuration is quite static and the only thing that might change in the near future is increasing the capacity by adding an additional ESXi host to the cluster.
Currently I am running with vSAN version 6.2 and since the environment is very stable and it is my “production” environment I don’t plan to upgrade to the latest and greatest version yet. Still, I do want to work with the newer versions and functions (like iSCSI target) to become familiar with them and stay up-to-date with my vSAN knowledge, so I have a test (virtual) vSphere 6.5 Cluster with vSAN 6.5 installed, currently in a 2-node (ROBO) setup with an additional witness appliance.
With the release of vSAN 6.6 (check out the release notes here) I wanted to upgrade my vSAN 6.5 environment. Actually I decided to create a new vSAN 6.6 cluster from scratch with my existing ESXi hosts, which means I first had to delete my existing vSAN 6.5 datastore.
Deleting a vSAN datastore is not the same as simply disabling the vSAN feature from the cluster object. If that is all you would do (and the same applies to hosts that you would “just” move to a different or new vSAN cluster) you would notice at the time you want to create diskgroups in your new vSAN cluster that no disks would be available on the host(s). This is caused by the fact that the disks are still claimed by vSAN so not available for anything else. You can verify this by using the “vdq -q” command on the ESXi console which will show that the previous cache and capacity devices have a state called “In-use for VSAN”.
The proper way to delete a vSAN datastore (or remove an inidividual host from a vSAN cluster) is to first remove the diskgroups from the host(s). This can be done from the WebClient :
Select the disk group of the host you want to remove from a vSAN cluster (and in the case of deleting the vSAN datastore this means all hosts – one by one) and click the “Remove the disk group” button. You will be presented with the following window asking you what you want to do with the existing objects on the disk group :
In my case I chose “No data evacuation” since I simply wanted to delete the vSAN datastore. If you want to remove a host from a vSAN cluster you would probably choose “Evacuate all data to other hosts” to ensure all objects remain compliant to their respective spolicies.
Once all disk groups from all hosts are removed it is safe to disable the vSAN functionality from the cluster, which will reconfigure all hosts and remove the vSAN datastore from the cluster.
In case you did not properly remove a host and still have disks that can not be claimed by vSAN because they still show a state of “In-use for VSAN” you can always remove the disk group manually from the ESXi command line. First you have to show the still existing vSAN storage configuration on the host. You do this by typing on the command line : “esxcli vsan storage list” which might show something like :
Note the name of the SSD device that is used as a cache device for the disk group you want to remove. The cache device is identified by checking the parameter “Is Capacity Tier” (which obviously is “false” for the cache device). You simply remove the entire disk group by typing :
“esxcli vsan storage remove -s <SSD Device Name> -m noAction”
The parameter “-m noAction” means the same as the option “no data evacuation” that was shown after clicking “Remove the disk group” in the WebClient.
Typing the command “esxcli vsan storage list” again will now result in the command prompt returning right away (in other words … there is nothing to show anymore). You can verify the result of the disk group removal with the “vdq -q” command which will show that the former cache and capacity devices now have a state called “Eligible for use by VSAN”.
Now your host is ready to be used in a new vSAN cluster. In my case this will be a new vSAN 6.6 cluster, but that will be the subject of my next post.