The much anticipated release of vSphere 6.5 has officially GA'ed and you can find all the release notes and downloads over here. Just like prior releases, I have created a new Nested ESXi Virtual Appliance to aide in quickly setting up a vSphere 6.5 environment for both educational as well as lab purposes. If you have not used this Virtual Appliance before, I strongly recommend you thoroughly review this blog post here for the background before proceeding further.
Disclaimer: Nested ESXi and Nested Virtualization is not officially supported by VMware, please use this at your own risk (the usual).
The new ESXi 6.5 Virtual Appliance includes the following configuration:
- ESXi 6.5 OS [New]
- GuestType: ESXi 6.5[New]
- vHW 11 [New]
- 2 vCPU
- 6GB vMEM
- 2 x VMXNET vNIC
- 1 x PVSCSI Adapter [New]
- 1 x 2GB HDD (ESXi Installation)
- 1 x 4GB SSD (for use w/VSAN, empty by default)
- 1 x 8GB SSD (for use w/VSAN, empty by default)
- VHV added (more info here)
- dvFilter Mac Learn VMX params added (more info here)
- disk.enableUUID VMX param added
- VSAN traffic tagged on vmk0
- Disabled VSAN device monitoring for home labs (more info here)
- VMFS6 will be used if user selects to create VMFS volume [New]
- Enabled sparse swap (more info here) [New]
To be able to import and use this new ESXi VA, you will need to be running at least vSphere 6.0 Update 2 in your environment as I take advantage of some of the new Nested ESXi enhancements in vSphere 6.5. If you need to run ESXi 6.5 on earlier versions of vSphere, then you can take my existing 5.5 or 6.0 VAs and manually upgrade to 6.5.
Now that you made it this far, here is download: Nested_ESXi6.5d_Appliance_Template_v1.ova
Lastly, I have also spent some time building some new automation scripts which takes advantage of my Nested ESXi VAs and deploys a fully functional vSphere lab environment without even breaking a sweat. Below is a little sneak peak at what you can expect 😀 Watch the blog for more details!
You are the man, ive been waiting! Great stuff!
Awesome, thanks for blazing the trail. If we want to make our own nested hypervisor VMs, is there any documentation or guidance on what VM configurations or advanced settings need to be adjusted for use in vSphere 6.0 and 6.5? I'm putting together some nested clusters to compare differences, for a pitch to upgrade.
Great news , Finally ESXi 6.5 Virtual Appliance is available to download. I am very exited to work on this ESXi 6.5 Virtual Appliance. Can you please tell me one thing , Is there any change in ESXi 6.5 Virtual Appliance installation and configuration ?
Hi William,
Any idea creating VMFS5/6 failed on the 4GB or 8GB device?
0b0597f)LVM: 12525: LVMProbeDevice failed with status "Device does not contain a logical volume".
viceIO: 2962: Cmd(0x439500993d80) 0x16, CmdSN 0xa51b from world 0 to dev "naa.6000c2910e5baeebfff76aa088cbaa85" failed H:0x0 D:0x2 P:0x0 Valid sense data: 0x5 0x24 0x0.
0b0597f)LVM: 10306: LVMProbeDevice failed on (3658013760, naa.6000c2910e5baeebfff76aa088cbaa85:1): Device does not contain a logical volume
0b0597f)FSS: 2295: Failed to create FS on dev [naa.6000c2910e5baeebfff76aa088cbaa85:1] fs [123] type [vmfs3] fbSize 1048576 => Not supported
410731a)World: 12230: VC opID HB-Sp[email protected] maps to vmkernel opID 2410731a
410731a)Config: 706: "VMOverheadGrowthLimit" = 0, Old Value: 4294967295, (Status: 0x0)
2301dcf)World: 12230: VC opID HB-Sp[email protected] maps to vmkernel opID 42301dcf
2301dcf)Config: 706: "HostLocalSwapDirEnabled" = 0, Old Value: 0, (Status: 0x0)
ef3e3b4)World: 12230: VC opID AddVmfsDatastoreFlow-add-768-ngc-74-bd-24ab maps to vmkernel opID 5ef3e3b4
ef3e3b4)LVM: 12525: LVMProbeDevice failed with status "Device does not contain a logical volume".
viceIO: 2962: Cmd(0x4395009e8100) 0x16, CmdSN 0xa668 from world 0 to dev "naa.6000c2910e5baeebfff76aa088cbaa85" failed H:0x0 D:0x2 P:0x0 Valid sense data: 0x5 0x24 0x0.
ef3e3b4)LVM: 10306: LVMProbeDevice failed on (3658013760, naa.6000c2910e5baeebfff76aa088cbaa85:1): Device does not contain a logical volume
ef3e3b4)FSS: 2295: Failed to create FS on dev [naa.6000c2910e5baeebfff76aa088cbaa85:1] fs [Datastore] type [vmfs3] fbSize 1048576 => Not supported
viceIO: 2948: Cmd(0x4395009e8100) 0x1a, CmdSN 0xa66e from world 0 to dev "naa.6000c2978e1edd0770c667133be8280d" failed H:0x0 D:0x2 P:0x0 Valid sense data: 0x5 0x24 0x0.
viceIO: 2948: Cmd(0x4395009e8100) 0x1a, CmdSN 0xa672 from world 0 to dev "naa.6000c2910e5baeebfff76aa088cbaa85" failed H:0x0 D:0x2 P:0x0 Valid sense data: 0x5 0x24 0x0
Same error for me. The only way i could make a datastore was to connect a drive to the emulated IDE-controller. Also tried the LSI SAS-controller but no go.
Ive the same issue
I think that same issue. During creating datastore: "Failed to create VMFS datastore "DATASTORENAME" - An error occured during host configuration.That same for device 4 , 8 GB or if i add additional disk... Some solution ? or workaround ? Added disk to "Virtual Device node" IDE(1:1) Hard Disk 1 not resolve my problem..because after this i dont have active button to create datastore...
Great work William. Any tips on how to get this masterpiece to run single node VSAN? I tried to hack the .vmx to change the 8GB SSD to a HDD, but I just ended up hacking it to pieces :O
Disregard. Found a way...
Any tips?
I Try to make a nested iscsi target connection from my nested esxi server to my vsan hardware cluster but it was not working. I do the same with a windows server virtual machine to the vsan hardware cluster and it works.
I may have made a mistake but i use far more often vmware iscsi software initiatator than the microsoft one.
Do you have already test this configuration ?
thanks for your help.
Hi Francois, did you get this to work with the esxi server to vsan?
I'm testing this deployment in Ravello's cloud, have Nested virtualizatino enabled on the ESXi 6.5 host and the VM's go great. The VCSA stops with this message:
The vCSA 'JGvCenter6' requires hardware virtualization support from the ESXi
host 'JGESXi6-Mgmt2.localdomain'. You are recommended to check the BIOS settings
of the ESXi host for hardware virtualization support before proceeding.
helpful news for ESXi 6.5 Virtual Appliance
Thanks for awesome work. I would like to build my own ESXi 6.5 templates. Can you please guide me how we can import ESXi profile into VMware Studio and start?
Great ova file, once started , what was the user and password should be use please ?
Regards
default user is root
password is empty ( nothing ). just press ENTER would let you in.
During the OVA import, it doesn't look like it take any of the variable fields I filled in. Such as the hostname, static IP, and root password.
What is default root login?
Hi William,
I have some troubles creating datastore on top of vSAN iSCSI target. The vSAN iSCSI target being a new feature, there is little to no information online. Hopefully you have a suggestion how to solve this problem.
My setup is composed of
3 physical boxes:
* vSAN 6.2 in ESX 6.5
* vSAN cluster over all 3 nodes, all flash, dedup & compress activated
2 initiators:
* nested ESX 6.0u2 in HA & LB cluster
* these VMs reside in a DS other than vSAN
* the software initiators are not using port binding
* all 3 vSAN nodes are added as paths to a iSCSI LUN
* the nested ESX share a VMDK that works fine as shared DS but I need to get rig of this as is disables the VM snapshot feature
The target and initiators reside in distinct subnets.
In the nested ESX, the iSCSI disk looks very strange:
vml.010000000056495453455249414c3830343839613538633836393332373063393031306363343761363636566972747561 -> iqn.1998_01.com.vmware:f8f97556_0547_abcc_ed9e_fba7a47a31be,L,0x0000000000000000
When trying to create the DS, the error is: "Failed to retrieve data from the server".
Marking this disk SSD/HDD did not help. Marking it local fails with "Cannot change the host configuration"
Thanks,
Sergiu.
Hi, Can you please explain why you are adding disk.enableUUID parameter to VMX file!?
I am not able to login with: root and blank password
I found it: the default password is: VMware1!
On these nested instances, why are the interface settings greyed out? I had hoped to pass vLAN tags to my switch via the Physical host interfaces.
Hi William,
Thanks for your great blog and helping enable us to lab quickly and efficiently with the minimum hardware.
I was just creating an NSX lab on my MBPRO with Fusion to do some presentation to a customer and took advantage of your Nested_ESXi6.5d_Appliance_Template_v1.ova download which was a great time saver. However, I had the impression from the top of this article that VHV was pre-enabled "VHV added (more info here)" but when I tried to deploy an NSX controller it wouldn't power on and deleted itself "The target host does not support the virtual machine's current hardware requirements". Scratching my head I tried increasing RAM and CPU on the hypervisor to no avail. Finally it twigged that perhaps it's something to do with VHV so I added the line vhv.enable = "TRUE" to the .vmx, restarted Fusion an voila all is good.
May I humbly suggest you add it to your template?
Cheers,
Steve
Hi William,
I just updated to the newly GA'd Fusion 10 and now running into a problem deploying the Nested_ESXi6.5d_Appliance_Template_v1.ova. It first complains with the message "The import failed because I Nested_ESXi6.5d_Appliance_Template_v1.0.ova did not pass OVF specification conformance or virtual hardware compliance checks." so I click retry as indicated to relax the checks but it subsequently fails during the importing phase with the message "Invalid target disk adapter type: pvscsi" and doesn't deploy the VM.
Can you please tell me why this is failing and suggest a fix?
Thanks,
Steve
I am also experiencing this issue, however I am using VMWare Workstation Pro 14 on a Windows Host. I would be most grateful for help in getting it working!
James and Steve,
I ran into this as well and have found that apparently its and issue with the version of OVFTool that comes with the latest version of Fusion and Workstation. Apparently it doesn't recognize a PVSCSI as a valid HBA to import from an OVF. I am still looking into how to get around that. If you install the prior version of Workstation Pro you can import the VM with the default settings (no OVF customization) and it will work but then fail in Fusion 10 or Workstation Pro 14 unless you change the OS type to ESX 6.5. If you find a work around to allow the deployment to work with the latest and greatest or if William does please post.
Thanks,
Paul
Thanks Paul. Sorry for the tardy response but I haven't got back to looking at this in ages. I did want to say though I do appreciate you taking the time to help out. I've decided to stump up for a beefy home lab server for ESXi as I've just burned so much time attempting to get useable labs running on the desktop with only 16GB RAM and the memory demands only get higher as time goes on. Sadly my Mac mini toys or at 16GB or I would have gone to at least 32 or 64Gb long ago. There's only so many hours in the day so I need something that gives me a stable fountain to learn on and not have to constantly tweak and troubleshoot. I certainly don't intend to downplay William Lams legendary efforts as I'm most appreciative of the insights he provides. All the best, Steve
Sorry I have a question why is it that the Nested ESXi 6.5 ova primary's disk so small, I have had problems installing NSX-T vibs no space. I moved the logs and swap but still not room to install the vib's is there some way to make this work?
Could you tell me how to build the nested_esxi appliance like you do? I try to build one VM with vApp but it fails to deploy it with ovftool. Thank you for your support.
VMware1! is not working for default pass
I am trying to deploy this lab and I am getting the following error.
Get-VirtualNetwork : The term 'Get-VirtualNetwork' is not recognized as the name of a cmdlet, function, script file,
or operable program. Check the spelling of the name, or if a path was included, verify that the path is correct and
try again.