Host cannot download files from VMware vSphere Lifecycle Manager patch store

Well, I had a bit of a mishap in the home lab that, unfortunately, led to the need to restore some data 🙂 I actually had to reload a couple of hosts in the vSAN cluster. I ran into a pesky little issue that turned out to be self-inflicted when it was all said and done. However, I wanted to document the issue here as the exact wording has yet to find itself into an official KB from VMware. The error I ran into after rebuilding a couple of lab hosts was “host cannot download files from VMware vSphere Lifecycle Manager patch store. Check the network connectivity and firewall setup, and check esxupdate logs for details. Let’s get to the bottom of this quick little issue that was encountered after reloading a couple of hosts.

Host cannot download files from VMware vSphere Lifecycle Manager patch store error

The issue was a message that I kept seeing appear in the recent vCenter tasks history and looked as though there was an issue with the vSphere Lifecycle Manager. This is a vSphere 7 environment with ESXi 7.0b servers with VCSA 7.0 16620013.

In looking a bit closer at the error message, two of the three hosts in the vSAN cluster looked to be the culprits in the error message that I was seeing in the vCenter Server tasks.

Sure enough, I could visit these two hosts manually and attempt to scan against the baselines and they would error out with the same error message.

The next step taken was to look at the esxupdate.log file and see any more detailed errors there. This is what I saw:

2020-08-17T20:18:11Z esxupdate: 2122079: esxupdate: ERROR: vmware.esximage.Errors.MetadataDownloadError: ('http://vcsa.cloud.local:9084/vum/repository/hostupdate/CIS/CIS-ESXi-7.0-Addon-cumulative_metadata.zip', None, "('http://vcsa.cloud.local:9084/vum/repository/hostupdate/CIS/CIS-ESXi-7.0-Addon-cumulative_metadata.zip', '/tmp/tmp3lnrm01e', '<urlopen error [Errno -2] Name or service not known>')")
2020-08-17T20:18:11Z esxupdate: 2122079: esxupdate: DEBUG: <<<

So the error that stands out is the Name or service not known. Could this be a simple resolution issue? I found a directly related VMware KB article here that describes the similar issue with the vSphere Update Manager:

• https://kb.vmware.com/s/article/67705

The KB however, is more focused on firewall issues for port 9084 which is required. Since I was on the same layer 2 segment as my VCSA appliance, I was not applying any firewall rules that would be filtering traffic to this port with my hosts. I suspected DNS resolution.

After this, I SSH’ed into each ESXi host and simply tried to ping my VCSA appliance. Low and behold – “name or service unknown”. In checking the DNS settings for my ESXi hosts I had fat fingered by DNS server on two of the hosts in haste and simply put the Host IP instead of my DNS server.

If you receive this error, make sure you take a look at the DNS server settings on your problematic ESXi host(s). This helps to rule out any issues with DNS configuration.

After correcting my DNS issue, I was able to then ping my VCSA by the FQDN which verified that DNS was working correctly.

Troubleshooting Steps Recapped and Resolution

To recap the steps performed to troubleshoot the issue, take a look at the following steps:

1. Examine the vCenter task error – see if it is specific hosts that are listed as having issues “scanning”.
2. Check the esxupdate.log on the specific ESXi host(s) having issues being scanned for updates to verify the issue with connectivity.
3. SSH into your ESXi host and simply ping your VCSA appliance from the command line by FQDN. See if you have issues with name resolution.
4. If there is a resolution issue, login to your DCUI and check your management network DNS settings.
5. Correct any issues with DNS settings here and restart your management network.
6. Verify you can then visit the Updates tab and scan the host for available updates

A note here, if name resolution is working correctly, then I would suggest taking a look at some possible firewall filtering, especially if you are crossing routed boundaries between your ESXi hosts and vCenter Server.

As in my case though, I suspect for the majority that see this issue, it will be DNS related in most cases. It’s always DNS right? It seems like it is a majority of the time.

Wrapping Up

If you see the error “Host cannot download files from VMware vSphere Lifecycle Manager patch store”, like me, you may have ran into issues getting your DNS server configuration populated correctly.

Following the troubleshooting steps above, you will be able to quickly diagnose the issue as either DNS-related or potentially firewall related.