WilliamLam.com

  • About
    • About
    • Privacy
  • VMware Cloud
  • Tanzu
    • Application Modernization
    • Tanzu services
    • Tanzu Community Edition
    • Tanzu Kubernetes Grid
    • vSphere with Tanzu
  • Home Lab
  • Nested Virtualization
  • Apple
You are here: Home / Automation / Automated vSphere 7 and vSphere with Kubernetes Lab Deployment Script

Automated vSphere 7 and vSphere with Kubernetes Lab Deployment Script

04.13.2020 by William Lam // 112 Comments

I know many of you have been asking me about my vSphere with Kubernetes automation script which I had been sharing snippets of on Twitter. For the past couple of weeks, I have been hard at work making the required changes between the vSphere 7 Beta and GA workflows, some additional testing and of course documentation. Hopefully the wait was worth it (I think it is) and if you enjoy the script or have benefited, please consider adding 🌟to the Github repo to show your support! Thanks and enjoy

Had to make some updates to one of my vGhetto Automated Lab Deployment Scripts

💥44min to automate all required #vSphere7 infrastructure! 🤛🎤🥳

1 x VCSA 7.0
3 x ESXi + vSAN 7.0
1 x NSX-T 3.0 UA
1 x NSX-T Edge

Need to clean up #ProjectPacific wording but its working great! pic.twitter.com/ZInPgVgbGS

— William Lam (@lamw) April 4, 2020

The Github repository:

  • https://github.com/lamw/vghetto-vsphere-with-kubernetes-external-nsxt-automated-lab-deployment

Before getting started, please carefully read through the requirements section along with the complete sample end-to-end execution if you are new to vSphere with Kubernetes. You will need to have a VMware Cloud Foundation (VCF) 4.0 license before you can get started and specifically an NSX-T Advance license which is one of the required parameters within the script. If you do not have access to a VCF 4 license, I strongly recommend taking part in the recent VMUG Advantage Homelab Group Buy effort which I had started to easily get access to the latest VMware releases along with a nice 15% discount!

The script supports deploying both a standard vSphere 7 environment with just VCSA, ESXi and vSAN as well as the complete solution which includes NSX-T to support vSphere with Kubernetes. For more details, please refer to the FAQ.

More from my site

  • Setup custom login banner when logging into a vSphere with Kubernetes Cluster
  • Guest Customization support for Instant Clone in vSphere 7
  • Troubleshooting tips for configuring vSphere with Kubernetes
  • Packer reference for VMware Harbor Virtual Appliance
  • Is vSphere with Kubernetes available for evaluation? 

Categories // Automation, Kubernetes, Nested Virtualization, NSX, VMware Tanzu, VSAN, vSphere, vSphere 7.0 Tags // Kubernetes, NSX-T, Project Pacific, VMware Cloud Foundation, vSphere 7.0, vSphere with Kubernetes

Comments

  1. jperformer says

    04/13/2020 at 11:08 am

    Perfect job Wiliam!

    Reply
  2. Ray says

    04/16/2020 at 7:55 am

    Thanks William!

    I'm getting this error during deployment, any ideas?

    https://imgur.com/a/2UbuIZk

    Reply
    • VirtualizeStuff says

      04/17/2020 at 11:49 am

      @Ray I received a similar message regarding the storage policy part, today.

      For lines 1035 & 1036 add " -Server $vc " to the New-TagAssignment command and to the New-SpbmStoragePolicy.

      This should resolve the issue as the policy is being applied to the main VCSA instance instead of the nested one. I am in the process of redeploying and will confirm.

      Hope that helps.

      Reply
    • VirtualizeStuff says

      04/17/2020 at 11:53 am

      Regarding the NSX-T Manager error make sure you can resolve the FQDN for the NSX-T manager from where your executing the script from.

      Reply
    • VirtualizeStuff says

      04/17/2020 at 12:21 pm

      https://imgur.com/vecSEe3

      Reply
      • VirtualizeStuff says

        04/17/2020 at 12:28 pm

        The deployment was successful adding the -Server $vc mentioned above.
        https://imgur.com/vtsryUx

        Reply
        • William Lam says

          04/18/2020 at 4:33 am

          Thank you VirtualizeStuff. I've just pushed the VM Storage Policy scope fix and glad to hear everything deployed successfully for you

          Reply
          • Ray says

            04/20/2020 at 12:57 pm

            Thank you VirtualizeStuff!

            I realized the issue with NSX-T Manager and was able to get that one resolved. I tried deploying again with your fix and it appears to work, but I'm still getting an unauthenticated error for the Get-Tag command:

            https://imgur.com/a/IPSIK43

            Not sure if this is really a problem. The rest of the script seems to complete successfully.

  3. Heng Yan says

    04/17/2020 at 1:26 pm

    I am getting below error. Everything else is working perfectly.

    1036 | New-SpbmStoragePolicy -Name $StoragePolicyName -AnyOfRuleSets …
    | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    | 4/17/2020 4:12:39 PM New-SpbmStoragePolicy Index was outside the bounds of the array.

    Reply
    • William Lam says

      04/17/2020 at 2:17 pm

      Thanks Heng. I believe this was also reported on the Github repo and I think I know why. Not had time to look at it yet, but that should go away in a future update. Glad to hear everything else is working for you

      Reply
      • Heng Yan says

        04/17/2020 at 2:20 pm

        Thanks William for your quick reply! I am new to this. Do I need to manually apply any change for this failure?

        Reply
        • William Lam says

          04/17/2020 at 2:30 pm

          No, it's just a warning. You should see a new Pacific Gold VM Storage Policy in the deployed vCenter Server Appliance. If you see that, then you're good

          Reply
  4. andrewjmoser says

    04/17/2020 at 4:35 pm

    Thanks for creating this! I'm following your scripts and readme on github, but I am getting tripped up on the Ingress/Egress both nested, and in my real lab that was built manually. I can successfully ping the IP that gets assigned to the control plane but whenever I try to launch the webpage, it times out either from VCSA or just typing in the URL. From reading the main docs on VMware site, your edge vlan and Ingress/Egress must be on the same VLAN, which I see is there, and can't overlap IP's, which they aren't. I'm not getting why I can ping it but can't connect, this is after multiple deployments.

    Reply
  5. jmgilman says

    04/19/2020 at 10:30 pm

    Thanks for the script! Do you have plans or suggestions for a (non-automated) guide for deployment in a non-nested environment? I'd love to get this working on my NUC vSAN cluster 🙂

    Reply
  6. Dennis says

    04/22/2020 at 1:16 am

    Great work William,

    deployment works like a charm.
    Unfortunately I run into a weird situation when configuring the workload management as its stuck after deploying the supervisorcontrolplanes. The first one starts, but then it doens't continue (I waited about 3 hours) showing the following status:
    Configure operation for the Master node VM with identifier vm-1012 failed.

    The log shows that it loops: master is not fully configured

    Anybody running into the same issue?

    Reply
    • Ray says

      04/22/2020 at 10:46 am

      Yep, I'm seeing the same problem as well.

      Reply
      • Thomas says

        04/27/2020 at 9:51 pm

        Yes, I am experiencing the exact same issue. 1 of the 3 initially deployed Master nodes starts being configured and it freezes there.

        Reply
        • skalugin2014 says

          05/03/2020 at 2:31 pm

          I have exact same issue there. The problem is that the 1st SupervisorControlPlaneVM is not reachable via network. I'm struggling with network topology there. My management network at 300th VLAN. Should I specify VLAN in my nested network portgroups as well?
          https://imgur.com/a/v7f3aYz

          Reply
          • skalugin2014 says

            05/04/2020 at 5:51 am

            Ok, my issue was with configuration of "underlay" ESX network with promiscuous and forget transmit off. After switching it on the Control VM become available via network. But the error remained: vmware vsphere Configure operation for the Master node VM with identifier "identifier" failed.

    • Dennis Zimmer says

      05/05/2020 at 2:47 am

      found and fixed the issues - unfortunately, I don't know what exactly caused the issues, but I changed the following
      1) allow promiscuous and forget transmit on the dvswitch the supervisor-VMs (thanks for the hint skalugin2014!)
      2) changed the NSX T0 connected vSwitch to MTU 9000 (was 1600 before)
      Then the workload deployment completed successfully (it can take an hour and you need to ignore the errors)

      But I couldn't add the tanzu content library and image pull for new pods failed as well - the fetcher wasn't able to download the images
      3) changed the routeros (not sure what the issue was) to a pfsense as my default gateway
      4) enabled mtu 9000 for the port towards the NSX T0
      5) enabled sNAT for the Kubernetes ip ranges

      Now it runs fine and so far all seems to work.

      Reply
      • Dennis says

        05/06/2020 at 10:31 am

        I shared my findings here after deploying the nested setup using William's script: https://itnext.io/nested-vsphere-7-and-kubernetes-lab-deployment-explained-f9bfca0112f5

        Reply
        • mikelbar says

          07/29/2020 at 2:59 pm

          Dennis, thanks for the write up! Can you configure the Pfsense with MTU 9000 and not have to touch the TOR switches? Trying to get this to work without having to change the TOR switches.

          TIA, Mike

          Reply
          • opvizordz says

            07/29/2020 at 10:57 pm

            Mike, I always changed the physical switches as well. I don't think that it will work without changing the switch ports pointing towards your LAN.

      • kastro says

        11/12/2020 at 9:23 am

        I have the same error: stuck at Master VM deployment.
        Checked MTUs on both "host" esxi and nested, forged and promiscues, nothing helped.
        Strange is that i had enabled Workload once on this cluster, then I removed everything and now i have this error. Nothing changed (as fas as I know).
        Using distributed switch (not NSX-T)

        Reply
        • nick liu says

          12/23/2020 at 8:04 am

          I hit the same. only one SupervisorControlPlaneVM is up, other 2 even complaint "There is no network assigned to this virtual machine". MTU, forged and promiscuous helped nothing.

          Reply
          • kastro says

            12/24/2020 at 3:19 am

            I think that I solved this with "DNS suffix" input during enablement of Workload. It's optional input but still.
            Now I have similar problems/error at deploying Guest clusters.....

  7. Mark Wolfe says

    04/22/2020 at 4:01 pm

    Awesome work, looking forward to getting this going!

    Anybody had luck getting this running on AMD non-Epyc? The NSX-T Edge won't come up. It reports lack of AES-NI but seems like it may actually be lack of DPDK support per Mike and Ben's discussion here:

    https://vswitchzero.com/2019/02/21/nsx-t-pcpu-requirements-for-edges/

    Hoping I can find out where config.py lives in the Edge OVA so that I can try the "hack" and get the script to complete. Not sure if not having DPDK support will cause further issues later though.

    Any other suggestions?

    Just built a shiny new Ryzen lab to try this out. Hopefully Ryzen isn't kicked out of the vSphere with Kubernetes homelab club! 🙂

    Reply
    • William Lam says

      04/23/2020 at 4:39 am

      Always carefully read through new product release notes, you'll never know when you find some goodies in there 🙂

      In NSX-T 3.0, AMD is indeed supported but for specific models (snippet from https://docs.vmware.com/en/VMware-NSX-T-Data-Center/3.0/rn/VMware-NSX-T-Data-Center-30-Release-Notes.html)

      AMD EPYC support: Edge Nodes, VM and Bare Metal can now be deployed on AMD EPYC series CPU:
      AMD EPYC 7xx1 Series (Naples)
      AMD EPYC 3000 Embedded Family and newer
      AMD EPYC 7xx2 Series (Rome)

      Reply
  8. virt.ninja says

    05/05/2020 at 8:54 am

    It's worth noting that I was getting failures on the Import-VApp commands e.g. when importing the NSX-T manager appliance - due to the vCenter Server certificate not being trusted, I assume, as the md5 sum checked out.

    I found that adding "-Force" to all of the Import-VApp statements fixed it for me.

    Reply
    • SimonS says

      05/06/2020 at 2:59 am

      Thanks, I had the same issue and it really helped me!

      Reply
  9. Michel Kamp says

    05/14/2020 at 11:09 am

    Hi,
    after spending a complete day on getting this deploy to work using "VMware Workstation 15" on Windows 2019 I mananged to get the scripted components deployed. BUT at enabling the workload management candys i get a incompatible cluster when i press the (i) i get message "Compatible clusters must have a minimum of two ESXi hosts configured with a validate license for Workload Management, Fully Automated DRS, vSphere HA, a vSphere Distributed Switch 7.0 along with enough capacity to store the requisite infrastructure componenters you'll be configuring."

    But i don't have a glue what i have misconfigured. I used these settings here : https://www.williamlam.com/2020/04/deploying-a-minimal-vsphere-with-kubernetes-environment.html

    Is there any log file were i can see what did not passed the test ??

    Thanks,
    Michel

    Reply
    • Michel Kamp says

      05/14/2020 at 11:16 am

      i found some issues in the log file. looks like this could be the issue ???
      [05-14-2020_07:26:18] Powering On pacific-nsx-edge-3a ...
      [05-14-2020_07:26:18] Creating vApp vGhetto-Nested-Project-Pacific-NSX-T-External-Lab-pSZrDVlI ...
      New-VApp: D:\powershell\vghetto-vsphere-with-kubernetes-external-nsxt-lab-deployment.ps1:766
      Line |
      766 | $VApp = New-VApp -Name $VAppName -Server $viConnection -Location …
      | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
      | 5/14/2020 7:26:20 PM New-VApp The operation is not supported on the object.

      [05-14-2020_07:26:20] Creating VM Folder Project-Pacific ...
      [05-14-2020_07:26:20] Moving Nested ESXi VMs into vGhetto-Nested-Project-Pacific-NSX-T-External-Lab-pSZrDVlI vApp ...
      Move-VM: D:\powershell\vghetto-vsphere-with-kubernetes-external-nsxt-lab-deployment.ps1:777
      Line |
      777 | … Move-VM -VM $vm -Server $viConnection -Destination $VApp -Confir …
      | ~~~~~
      | Cannot validate argument on parameter 'Destination'. The argument is null or empty. Provide an
      | argument that is not null or empty, and then try the command again.

      Move-VM: D:\powershell\vghetto-vsphere-with-kubernetes-external-nsxt-lab-deployment.ps1:777
      Line |
      777 | … Move-VM -VM $vm -Server $viConnection -Destination $VApp -Confir …
      | ~~~~~
      | Cannot validate argument on parameter 'Destination'. The argument is null or empty. Provide an
      | argument that is not null or empty, and then try the command again.

      [05-14-2020_07:26:20] Moving pacific-vcsa-3 into vGhetto-Nested-Project-Pacific-NSX-T-External-Lab-pSZrDVlI vApp ...
      Move-VM: D:\powershell\vghetto-vsphere-with-kubernetes-external-nsxt-lab-deployment.ps1:784
      Line |
      784 | … Move-VM -VM $vcsaVM -Server $viConnection -Destination $VApp -Confir …
      | ~~~~~
      | Cannot validate argument on parameter 'Destination'. The argument is null or empty. Provide an
      | argument that is not null or empty, and then try the command again.

      [05-14-2020_07:26:21] Moving pacific-nsx-3 into vGhetto-Nested-Project-Pacific-NSX-T-External-Lab-pSZrDVlI vApp ...
      Move-VM: D:\powershell\vghetto-vsphere-with-kubernetes-external-nsxt-lab-deployment.ps1:790
      Line |
      790 | … ove-VM -VM $nsxMgrVM -Server $viConnection -Destination $VApp -Confir …
      | ~~~~~
      | Cannot validate argument on parameter 'Destination'. The argument is null or empty. Provide an
      | argument that is not null or empty, and then try the command again.

      [05-14-2020_07:26:21] Moving NSX Edge VMs into vGhetto-Nested-Project-Pacific-NSX-T-External-Lab-pSZrDVlI vApp ...
      Move-VM: D:\powershell\vghetto-vsphere-with-kubernetes-external-nsxt-lab-deployment.ps1:797
      Line |
      797 | … ve-VM -VM $nsxEdgeVM -Server $viConnection -Destination $VApp -Confir …
      | ~~~~~
      | Cannot validate argument on parameter 'Destination'. The argument is null or empty. Provide an
      | argument that is not null or empty, and then try the command again.

      [05-14-2020_07:26:21] Moving vGhetto-Nested-Project-Pacific-NSX-T-External-Lab-pSZrDVlI to VM Folder Project-Pacific ...
      Move-VApp: D:\powershell\vghetto-vsphere-with-kubernetes-external-nsxt-lab-deployment.ps1:802
      Line |
      802 | Move-VApp -Server $viConnection $VAppName -Destination (Get-Folde …
      | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
      | 5/14/2020 7:26:21 PM Move-VApp Could not find VApp with name
      | 'vGhetto-Nested-Project-Pacific-NSX-T-External-Lab-pSZrDVlI'.

      Move-VApp: D:\powershell\vghetto-vsphere-with-kubernetes-external-nsxt-lab-deployment.ps1:802
      Line |
      802 | Move-VApp -Server $viConnection $VAppName -Destination (Get-Folde …
      | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
      | 5/14/2020 7:26:21 PM Move-VApp Value cannot be found for the mandatory parameter VApp

      [05-14-2020_07:26:21] Disconnecting from 192.168.236.140 ...

      Reply
      • William Lam says

        05/14/2020 at 12:55 pm

        Do you have DRS enabled on your physical vSphere Cluster? This is required to create the vApp construct ... but even then, it shouldn't impact your enablement of Workload Management. As suggested in the troubleshooting blog post, take a look at the various compat checks and tail the wcp.log to see whats happening, this is probably environmental

        Reply
        • Michel Kamp says

          05/15/2020 at 7:58 am

          hi william,

          thanks for replying

          I fixed the Vapp and moved all the hosts into this.

          but still is get a incompatible cluster

          i got this message below from the wcp log.

          Any idea how to fix?

          2020-05-15T12:47:52.0052 debug ucp [opID=uapi] Sending response with output {"output”:{"STRUCTURE”:{ "com.umuare.cis.data.prouider.resource_mode1.resu1t_set”:{"items":[{"STRUCTURE":{"com.umuare.cis.dat a.prouider.resource_mode1.resource_itern":{"property_ualues":[{"OPTIONAL":"domain-cl006:90e22647-9a24 -4e34-949e-04d3fdf50cl0">,{"OPTIONAL":"ClusterComputeResource">,{"OPTIONAL":[{"STRUCTURE":{"com.umwa re.uapi.std.localizable_message":{"args":["90e22647-9a24-4e34-949e-04d3fdf50cl0"I,"default.message": "Failed to list all distributed switches in uCenter 90e22647-9a24-4e34-949e-04d3fdf50cl0.","id":"uce nter. wcp. nsx. 1 ist. dus. error" /'localized": {"OPTIONAL" : nu 11 >, "params" : {"OPT IONAL" : nu 11 »», {"STRUCTURE ":{"com.umware.uapi.std.loca1izable.message":{"args":["domain-cl006"],"defau 1t.message":"Cluster dom ain-cl006 is missing compatible NSX-T UDS.","id":"ucenter.ucp.ncp.cluster.incompatible","localized": {"OPTIONAL":null>,"params":{"OPTIONAL":nul1>>>>]>]>>>],"properties":["OmodelKey’V’Otype", "errors"]," tota l.count" : {"OPT IONAL" : nu 11 »»>
          2020-05-15T12:47:52.2262 debug ucp [opID=5ebe77e81 No notifications. seqNum: 1, Current seqNum: 0

          Reply
          • William Lam says

            05/15/2020 at 10:09 am

            As you can see from the error, it stating "Cluster domain-cl006 is missing compatible NSX-T UDS"

            Not sure if this is a localization issue but UDS == VDS. Since you're using the Automation, which defaults creates a vSphere 7 VDS (not NSX-T N-VDS), its highly possible VC can't talk to NSX-T due to timeskew which is something I've seen customers run into. You can verify this by looking at the NSX-T Manager API logs which I reference https://www.williamlam.com/2020/05/troubleshooting-tips-for-configuring-vsphere-with-kubernetes.html You should see some token issues if it is what I suspect

          • Michel Kamp says

            05/17/2020 at 1:08 pm

            Thanks William, During the copy and paste some chars went corrupted , was forgotten to correct it. So yes UDV is VDS. Thanks for the blog link i will check it.

            I also did a complete new deploy but at the VCSA deployment i got a IO network copy error , i changed the script to use import-vapp but thats giving me a trust error. Didn't try out the repack tool from vmware to fix this trust error. So i went to deploy it manually. Currently running.

            Let's hope it is going to deploy correctly ....

            michel

  10. Pedro Calixto says

    05/18/2020 at 10:20 am

    Great job, William! Is it possible to use this same script to deploy into a standalone ESXi host instead of deploying into a vCenter?

    Reply
    • William Lam says

      05/18/2020 at 8:40 pm

      No

      Reply
    • Michel Kamp says

      05/19/2020 at 2:32 am

      i have a setup with VMware Workstation on a 16 core , 128 GB mem 1 TB storage server . It is almost working ... Don't think the almost is vmware workstation related . I am sure William could get this to work 😉

      Reply
  11. torreslm says

    05/20/2020 at 9:55 am

    What to say thanks for the script but when I start to deploy I get this error "Unable to locate Pacific-VTEP portgroup, please create this network before continuing ..." Thanks for any help you can give

    Reply
    • Michel says

      05/20/2020 at 11:36 am

      Indeed it would be better to also add this port group creation in a pre instal setup script. Did not wrote the script but maybe my experiences help: You have to create this port group on the VC you are using to run the script against . Also be sure to change all IP numbers in the script to a range you can access from the server where you run the PS on. (Only the once’s listed in the git page) The script creates a couple of VMs on the VC you connect it to. It also creates a new VC . This new VC you can use at the end to enable the kubernetes on. Also be sure to have ALL VM dns names resolvable !! This rollout depends really on DNS. I used CoreDNS for this.
      Especially the creation of the edge pre setup was a challenge due to failing REST calls to the NSX , at the end I created it manually.

      Nevertheless never got it completely to finish , at enabling the kubernetes cluster I get a vm master setup error.(could be due to my manual adds pointed above )

      I will continue to get it working and will also PUSH some changes to the script to make it Re-runable in case of a stopped deploy .

      Oh one more point , I run it on VM workstation as platform .

      Michel

      Reply
      • torreslm says

        05/20/2020 at 12:41 pm

        Thanks for the info

        Reply
      • Leef Torres says

        05/21/2020 at 7:42 am

        Ok I got it to start deploying and everything seem to be going good but then i get this error

        OperationStopped: E:\Utility\ALPHAIT_Home_Lab\vghetto-vsphere-with-kubernetes-external-nsxt-lab-deployment.ps1:1034
        Line |
        1034 | $ipPool = $ipPoolService.create($ipPoolSpec)
        | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
        | A server error occurred: 'com.vmware.vapi.std.errors.invalid_request': . Check
        | $Error[0].Exception.ServerError for more details.

        [05-21-2020_01:42:39] Creating Overlay & VLAN Transport Zones ...
        [05-21-2020_01:42:40] Creating ESXi Uplink Profile ...
        [05-21-2020_01:42:40] Creating Edge Uplink Profile ...
        [05-21-2020_01:42:43] Creating Transport Node Profile Pacific-Host-Transport-Node-Profile ...
        Write-Error: Error in creating NSX-T Transport Node Profile
        Write-Error:
        ({
        "details" : "Field level validation errors: {required property host_switch_spec.host_switches[0].ip_assignment_spec.ip_pool_id is missing}",
        "httpStatus" : "BAD_REQUEST",
        "error_code" : 255,
        "module_name" : "common-services",
        "error_message" : "Field level validation errors: {required property host_switch_spec.host_switches[0].ip_assignment_spec.ip_pool_id is missing}"
        }.Exception.Message)

        Reply
        • Michel says

          05/21/2020 at 10:20 am

          I had exact same issue . So I created it manually in the NSX portal.

          Reply
  12. henry says

    05/22/2020 at 1:22 pm

    Trying this deployment - got a strange error (below).
    It looks as if it is failing when trying to create the disk group on each host.
    I'm only doing a vsphere and vsan deployment here - followed instructions in FAQ (set other options to zero)

    And everything looks to be there ... I just have to log in after and through the UI cretae disk groups ...
    The code looks fine though ... I checked all of the variables (jn-line, after the script completes) ..

    .....................................................................
    Querying ESXi host disks to create VSAN Diskgroups ...
    [05-22-2020_04:06:19] Creating VSAN DiskGroup for tdclab-esx5.tdclab.deployed.af.mil ...
    DriveLetter FriendlyName FileSystemType DriveType HealthStatus OperationalStatus SizeRemaining Size
    ----------- ------------ -------------- --------- ------------ ----------------- ------------- ----
    D VMware VCSA 0 5 0 {2}
    New-VsanDiskGroup: C:\Users-Source\Scripts\dev\vtdclab7.ps1:879:13
    Line |
    879 | New-VsanDiskGroup -Server $vc -VMHost $vmhost -SsdCanonic …
    | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    | 05/22/20 4:06:20 PM New-VsanDiskGroup VSAN runtime fault on server
    | '/VIServer=vsphere.local\[email protected]:443/': Unknown server error: '
    | Required property deviceName is missing from data object of type HostScsiDisk
    while
    | parsing serialized DataObject of type vim.host.ScsiDisk
    at line 1, column 360
    while
    | parsing property "cacheDisks" of static type ArrayOfHostScsiDisk
    while parsing
    | serialized DataObject of type vim.vsan.host.DiskMappingCreationSpec
    at line 1, column
    | 298
    while parsing call information for method InitializeDiskMappings
    at line 1, column
    | 171
    while parsing SOAP body
    at line 1, column 64
    while parsing SOAP envelope
    at line
    | 1, column 0
    while parsing HTTP request for method initializeDiskMappings
    on object of
    | type vim.cluster.VsanVcDiskManagementSystem
    at line 1, column 0'. See the event log for
    | details..

    Reply
  13. Tronics says

    06/03/2020 at 2:39 am

    Any specific configuration for "Pacific-VTEP" port group?

    Reply
    • lamw says

      06/03/2020 at 5:52 am

      Nope, its just a dummy portgroup and you can use the defaults when creating it.

      Reply
      • Tronics says

        06/03/2020 at 6:30 am

        Thanks William, I had another question regarding VCF 4 license, based on my understanding it is a suite of products and each product has its own license isn't it? and its managed via SDDC manager.

        Any reference document where VCF4 licensing is explained?

        Reply
  14. David P. says

    06/23/2020 at 3:30 am

    Hello

    I don't understand why, but when I try to enable "workload management", I configure all options, process is beginning, but during configuration, I don't know why but I have nested ESXi host which reboot without, but no warning or error during process
    Anyway I have sufficient ressource 🙁
    Do someone have the same issue ?
    Thank you
    ps : sorry for my bad english

    Reply
    • David P. says

      06/23/2020 at 6:44 am

      After some another test, I know exactly when nested esxi host shutdown
      The deploy of "SupervisorControlPlaneVM" is OK, and just after, it try to power ON.
      And just at this time, ESXi host shutdown immediatly. VM's doesn't have time to boot.
      So "SupervisorControlPlaneVM" VM go to another ESXi
      Try to power ON
      and ... shutdown ESXi host again 🙁
      Someone do you know why ?
      My hardware lab is 3 chinese barebone with 10170U proc and 64Gb each with vSAN

      Reply
      • David P. says

        06/23/2020 at 7:13 am

        so... I think problem is on nested ESXi, because regardless of any VMs I make, when I start the VMs, ESXi stop immediatly 🙁
        I use ova "Nested_ESXi7.0_Appliance_Template_v1.ova" anyway ...
        snifff

        Reply
        • William Lam says

          06/23/2020 at 7:20 am

          David,

          The symptoms you're describing is definitely not related to vSphere w/K8s but sounds like its due to https://www.williamlam.com/2020/04/heads-up-nested-esxi-crashes-in-esxi-7-0-running-on-older-cpus.html

          The fix should be out later today with the release of ESXi 7.0 Patch 01 which will resolve Nested ESXi crash

          Reply
          • David P. says

            06/23/2020 at 7:52 am

            by the way, thank you for your answer 🙂
            Il will avoid me to search for long time unnecessarily.

  15. David P. says

    06/23/2020 at 7:47 am

    yes, you're right...
    so, I will waiting up to the patch 01 for ESXi 7.0... snifff

    Reply
  16. dennisjohnson104 says

    06/28/2020 at 6:15 am

    Hi, i'm running into an issue when running the script. I've updated each editable variable a few times to verify everything was correct but cant get past this error.

    553 | … n $ovfconfig -Name $VMName -Location $cluster -VMHost $vmhost -Datast …
    | ~~~~~~~
    | Cannot validate argument on parameter 'VMHost'. The argument is null or empty. Provide an argument that is not
    | null or empty, and then try the command again.

    Reply
    • dennisjohnson104 says

      06/28/2020 at 11:02 am

      Added my physical host to the cluster and it resolved the issue.

      Reply
  17. David P. says

    06/29/2020 at 5:52 am

    Hello

    Deployment is OK
    Workload management is OK
    I create namespace

    but ...
    I have a issue to access at my namespace.

    when I click to 'open'
    http://lufia.konyxia.com/NSXT/0.click_OK.png

    first access is ok
    http://lufia.konyxia.com/NSXT/first-access.png

    but if I try to refresh or download CLI tools, I have a "timed out"
    http://lufia.konyxia.com/NSXT/second-access(with_F5).png
    I always ping anyway

    I have to close and re-open, but same issue.

    promiscuous and forged transmits are ok, I put 1600 mtu everywhere,
    but always same issue, I think for a mtu error.

    Dennis Zimmer has write
    2) changed the NSX T0 connected vSwitch to MTU 9000 (was 1600 before)
    4) enabled mtu 9000 for the port towards the NSX T0

    can you tell me what vswitch please ?
    thanks

    Reply
    • David P. says

      07/06/2020 at 8:46 am

      Hello
      I tried lot of solutions, but none was successful.
      best I can do after change lots of parameters (mtu, promiscuous, change vlan for a full level 2 transports) is to be able to have each time the home page, but after a big time, but it's all, I can't have more 🙁
      Nobody would have an idea ?
      Thanks

      Reply
      • kastro says

        11/24/2020 at 1:32 am

        Hi
        I have similar problem. Did you find a solution ?
        Thanx

        Reply
  18. Michael Otu says

    07/05/2020 at 9:09 pm

    William,
    I want to commend you for all your hardwork in putting this series of scripts together. I can only imagine the amount of effort you put in to get this to a working state. Please keep up the good job!

    Regards,

    Reply
    • William Lam says

      07/06/2020 at 6:40 am

      Thanks for the note Michael. Yes, this has definitely been a ton of work which most folks don't see other than the final results. Its also challenging to keep things up to date with so many other scripts I've written, so finding spare time is always hard as well.

      Reply
  19. David P. says

    07/10/2020 at 12:04 pm

    Hello everybody,

    I think I know why I can't access to workload page
    http://lufia.konyxia.com/NSXT/0.click_OK.png
    All is done up to the namespace, but unable to continue.

    I use usb network with flings drivers, and I noticed that mtu is limited to 4000
    https://flings.vmware.com/usb-network-native-driver-for-esxi/bugs/816

    I use vsphere 7 with only usb network because realtek is not more supported with vsphere 7 (only vusbX, not vmnicX).

    I would like to know if somebody has successfully used this lab with only fings usb network ?
    or if someone have an idea for have it run? (specific conf nsx or nested esxi ?)
    Thanks.

    Reply
  20. David P. says

    07/14/2020 at 8:18 am

    Hello,
    I have resolved my issue, it was a MTU error beetwen 2 switch, it's ok now

    However I have a last issue, even if ostensibly all seem good.
    the output of "kubectl get machine" give me :
    error: the server doesn't have a resource type "machine"

    show picture : http://lufia.konyxia.com/NSXT/error_get_machine.png

    anyway, no error up to Step 6 - Verify the TKG Cluster is ready before use by running the following command:
    Did I forget something ?
    Thanks

    Reply
  21. Igor M. says

    08/07/2020 at 7:45 am

    First of all - thanks a lot!

    Just wondering - how long should take creating of TKG cluster?
    I mean:
    ./kubectl apply -f tkg-cluster.yaml

    I did it 30 min ago, but I got nothing (status is still "creating" for "tkg get cluster"):
    https://i.gyazo.com/a8dfdff25d3ebd65f32fcd93d37d8cb9.png

    Reply
    • Igor M. says

      08/07/2020 at 8:29 am

      Found out that the problem is in:

      NSX IP pool exhausted
      FailedRealizeNSXResource
      Aug 7, 2020, 5:26:45 PM

      vm is not yet created: vmware-system-capw-controller-manager/WCPMachine/infrastructure.cluster.vmware.com/v1alpha2/im-tkg-nmsps-1/im-tkg-cluster-1/im-tkg-cluster-1-control-plane-bm97l
      ReconcileFailure
      Aug 7, 2020, 5:26:14 PM

      Hmm, how could it be? I deployed only Supervisor cluster and started to deploy TKG cluster (no demo applications etc.).

      Reply
  22. Johnnyb says

    08/20/2020 at 12:25 pm

    William,

    Thanks for the work, script, attention to detail, and follow up on these questions. I really appreciate it.

    I have a quick licensing questions, though. I have VMUG, Paid, and I have VCF 4.0. Reading the setup part of the script, it only shows where to plug in an NSX-T license. I happen to have an NSX-T key, so put it in, and ran the script. The script completed with no errors. Now, though, when I try to add a workload, I get an error saying I'm not licensed for Kubernetes and that I need an Esxi Enterprise with Kubernetes add on license key. I don't have one, nor do I see where to get the license from VMUG.

    Where did I go wrong?

    Reply
    • William Lam says

      08/20/2020 at 3:21 pm

      For vSphere w/K8s to be enabled from vCenter Server standpoint, you need to the Enterprise w/K8s key which is NOT included from VMUG afaik, BUT you can simply use an eval deployment (which is what the script assumes) and hence you'll get 60 days to satisfy that requirement. The VCF 4.0 license includes all of these details but for that to be successful, you need to follow the VCF deployment which uses SDDC Manager, the underlying products are then license on its behalf. Hope that makes sense

      Reply
      • Johnnyb says

        08/20/2020 at 6:37 pm

        It kinda makes sense, but I must be missing something somewhere...When I try to add a workload, it fails claiming I don't have the needed license, even within the 60 days. So what kind of trial is it? I'm sorry if I'm being block-headed.

        Reply
        • William Lam says

          08/20/2020 at 8:12 pm

          Just make sure your VCSA/ESXi hosts are eval and you’ve got proper NSX-T License

          Reply
  23. Max says

    08/21/2020 at 5:28 pm

    So there really is no way to properly license so that it runs for more than 60 days using the VMUG membership at this time. Just about no-one has the hardware in a home lab to run VCF, and until they make the ESXi with Kubernetes licenses available, you'll only have 60 days to try this out. Unless I'm missing something. Don't really have the will to re-create the entire environment every 60 days, even if it's scripted.

    I find VMware's crazy requirements exceptionally limiting. Rancher is really the way to go if you want to use Kubernetes and get K8S up and running without jumping through insane hoops.

    Reply
    • William Lam says

      08/21/2020 at 6:55 pm

      Take a look at the TKG Demo Appliance if you just want basic K8s https://www.williamlam.com/2020/08/tanzu-kubernetes-grid-tkg-demo-appliance-1-1-3.html

      Reply
  24. t4james says

    09/13/2020 at 4:27 am

    I tried deploying the standard deployment for 6.7 but Im having an issues adding the ESXi hosts to VCSA 6.7.. The script seems to hang during the adding ESXi host to Vcenter Cluster phase until the session times out. No hosts are added . I tried manually adding the hosts but "Im having this error "Authenticity of the host's SSL certificate is not verified ". Anyone had this issue before ? Im using ESXi 6.7 and VCSA U3.

    Reply
  25. Mbriet says

    09/17/2020 at 5:20 am

    hello,
    can we use a existing vcenter and deploy only esx vsan nsx-t.....

    Reply
  26. Ana says

    10/07/2020 at 1:29 am

    Just downloaded VCF from VMUG Advantage, and it is a 20 GB OVA file (VMware-Cloud-Builder-4.0.0.0-16008466_OVF10.ova), and not four ISOs, as it seems needed to be able to follow the script. Any ideas or comments?
    Thanks in advance.

    Reply
  27. Brad Bentley says

    10/12/2020 at 1:00 pm

    Keep running into this error

    Reply
    • Brad Bentley says

      10/12/2020 at 1:00 pm

      Line |
      685 | if(!(Connect-NsxtServer -Server $NSXTMgrHostname -Username $NSXAd …
      | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
      | 10/12/2020 12:35:10 PM Connect-NsxtServer The SSL connection could not be established, see inner exception..

      Unable to connect to NSX-T Manager, please check the deployment

      Reply
      • Brad Bentley says

        10/14/2020 at 8:55 am

        The fix for the error above was to set Powershell to ignore the SSL certificate.

        Be sure to "Set-PowerCLIConfiguration -InvalidCertificateAction:Ignore" before you run the script. perhaps add it to the script and once finished set back to default.

        Reply
  28. OvidiuM says

    11/09/2020 at 1:32 am

    Hi William, thanks again for your great work! Could you please tell me if it is a requirement that the VCA connects to the Internet in order to deploy the supervisor cluster? It seems that first it tries to download some stuff, then is failing and then it deploys some control VMs OVAs but not able to configure them. I have enabled a proxy in the VCSA admin, but still not working. Any ideas will be appreciated. Thank you!

    Reply
  29. kastro says

    12/04/2020 at 12:12 am

    Hi William, thank you for all scripts provided, great work.

    Question about DNS: all hostnames used need to be in our DNS server, everything resolvable as FQDN or are you using IPs for setting up appliances etc?

    Reply
    • William Lam says

      12/04/2020 at 4:49 am

      Either can be supported but for ease of use, recommend setting up DNS

      Reply
  30. jays says

    01/13/2021 at 6:50 am

    Folks - any ideas on these errors.

    [01-12-2021_11:12:51] Creating External T0 Gateway Interface ...
    OperationStopped: C:\Users\win10\Desktop\vm\vghetto-vsphere-with-kubernetes-external-nsxt-lab-deployment.ps1:1396
    Line |
    1396 | $t0GatewayInterface = $t0GatewayInterfacePolicyService.update …
    | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    | A server error occurred: 'com.vmware.vapi.std.errors.invalid_request': . Check
    | $Error[0].Exception.ServerError for more details.

    [01-12-2021_11:12:53] Adding Static Route on T0 Gateway Interface from 0.0.0.0/0 to 192.168.2.1 ...
    OperationStopped: C:\Users\win10\Desktop\vm\vghetto-vsphere-with-kubernetes-external-nsxt-lab-deployment.ps1:1417
    Line |
    1417 | $staticRoute = $staticRoutePolicyService.patch($T0GatewayName …
    | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    | A server error occurred: 'com.vmware.vapi.std.errors.invalid_request': . Check
    | $Error[0].Exception.ServerError for more details.

    Reply
    • jays says

      01/14/2021 at 9:32 pm

      PS C:\Users\win10\Desktop\vm> $Error[0].Exception.ServerError.data

      httpStatus error_code module_name error_message
      ---------- ---------- ----------- -------------
      BAD_REQUEST 500012 Policy The path=[] is invalid

      Reply
    • ThomasD says

      01/15/2021 at 1:22 pm

      I also had this issue. I tried to mannually do these steps in the UI, but don't know if I did them right because I still have problems configuring the workload management feature. What NSX T version are you using? Mine is 3.1 limited export (maybe the limited export part has something to do with it?)

      Reply
      • jays says

        01/15/2021 at 5:13 pm

        Interesting...im on same - nsx-unified-appliance-3.1.0.0.0.17107212-le
        I also have nsx 3.0.1 i could try as well.

        Reply
        • jays says

          01/15/2021 at 7:10 pm

          I just tried with nsx-unified-appliance-3.0.1.0.0.16404476.ova and i get a bunch of errors now:

          [01-15-2021_07:07:02] Creating External T0 Gateway Interface ...
          InvalidOperation: C:\Users\win10\Desktop\vm\vghetto-vsphere-with-kubernetes-external-nsxt-lab-deployment.ps1:1390
          Line |
          1390 | $t0GatewayInterfaceSpec.segment_path = "/infra/segments/$Netw …
          | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
          | The property 'segment_path' cannot be found on this object. Verify that the property exists and can be
          | set.

          InvalidOperation: C:\Users\win10\Desktop\vm\vghetto-vsphere-with-kubernetes-external-nsxt-lab-deployment.ps1:1391
          Line |
          1391 | $t0GatewayInterfaceAddResult = $t0GatewayInterfaceSpec.subnet …
          | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
          | You cannot call a method on a null-valued expression.
          InvalidOperation: C:\Users\win10\Desktop\vm\vghetto-vsphere-with-kubernetes-external-nsxt-lab-deployment.ps1:1392 Line | 1392 | $t0GatewayInterfaceSpec.type = "EXTERNAL" | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | The property 'type' cannot be found on this object. Verify that the property exists and can be set.
          InvalidOperation: C:\Users\win10\Desktop\vm\vghetto-vsphere-with-kubernetes-external-nsxt-lab-deployment.ps1:1393
          Line |
          1393 | $t0GatewayInterfaceSpec.edge_path = $edgeClusterNodePath
          | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
          | The property 'edge_path' cannot be found on this object. Verify that the property exists and can be
          | set.

          InvalidOperation: C:\Users\win10\Desktop\vm\vghetto-vsphere-with-kubernetes-external-nsxt-lab-deployment.ps1:1394
          Line |
          1394 | $t0GatewayInterfaceSpec.resource_type = "Tier0Interface"
          | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
          | The property 'resource_type' cannot be found on this object. Verify that the property exists and can
          | be set.

          OperationStopped: C:\Users\win10\Desktop\vm\vghetto-vsphere-with-kubernetes-external-nsxt-lab-deployment.ps1:1395
          Line |
          1395 | $t0GatewayInterface = $t0GatewayInterfacePolicyService.update …
          | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
          | Unable to get field 'tier0_interface', no field of that name found.

          [01-15-2021_07:07:05] Adding Static Route on T0 Gateway Interface from 0.0.0.0/0 to 192.168.2.1 ...
          OperationStopped: C:\Users\win10\Desktop\vm\vghetto-vsphere-with-kubernetes-external-nsxt-lab-deployment.ps1:1416
          Line |
          1416 | $staticRoute = $staticRoutePolicyService.patch($T0GatewayName …
          | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
          | A server error occurred: 'com.vmware.vapi.std.errors.invalid_request': . Check
          | $Error[0].Exception.ServerError for more details.

          Reply
          • ThomasD says

            01/15/2021 at 11:44 pm

            Did you use that version of both nsx t manager AND nsx t edge? Maybe they don't like being on different versions...

          • ThomasD says

            01/16/2021 at 12:54 am

            Actually I think I just got it working using the 3.1.0 le version. It says the workload management cluster is running but can't test it right now, will come back to that later...
            What I did was set MTU to 1600 on everything I could (all VDS's, T0interface...) like described in this writeup: https://itnext.io/nested-vsphere-7-and-kubernetes-lab-deployment-explained-f9bfca0112f5

            And also did the accept promiscuous mode and forged transmits on everything I could find in both the nested and native vsphere instance.

            To manually configure the part of the t0 gateway that the script failed on (not the ones you just posted but the ones before that) I did this in the NSX T manager gui:
            click on networking -> T0 gateway (there should already be one)
            Edit the gateway
            Go to interfaces
            add
            name: default
            type: external
            ipaddress/mask: what you set in the variables (192.168.0.14/24 in my case)
            connected to segment: pacific-segment
            edge node: the edge node
            MTU: 1600

            Then save and close the interface editing part
            Go to routing -> static routes
            name default network 0.0.0.0/0 next hop 192.168.0.1(your internet gateway)

            Will update on if it actually functions tonight, but this got me further in the deployment than before 🙂

  31. jays says

    01/16/2021 at 9:25 am

    Thanks ThomasD
    I deployed with full version of NSX 3.0.2 and I got same errors as the last ones i posted.

    When you say you got it working, i take it you did the manual fixes from your last post. The script still displayed the same error messages, correct?

    I will follow the manual steps you outlined and hopefully will get it going as well.

    Reply
    • ThomasD says

      01/16/2021 at 12:52 pm

      Yep, so I ignored the 2 errors (the one from creating T0 gateway & adding static route) and then continued with the steps I said above. I can now confirm it works, got workload management enabled by following the rest of the guide on the github repo and got the demo app running.

      Found on vmware docs that these are the limitations of the limited export version of nsx t
      no IPSec VPN
      no HTTPS-based Load Balancer

      But HTTP load balancer does seem to work, so that's enough for lab environments I guess

      Reply
      • jays says

        01/17/2021 at 10:42 am

        Great news with no errors ...retried with NSX-T - nsx-unified-appliance-3.0.2.0.0.16887203.ova & nsx-edge-3.0.2.0.0.16887208.ova

        Issue as you pointed out is likely with LE version.

        Reply
  32. jones says

    01/20/2021 at 9:58 pm

    Anyone knows what this error means?
    Everything else seems to run successfully

    Creating Project Pacific Storage Policies and attaching to vsanDatastore ...
    New-SpbmRule: C:\Users\win10\Desktop\vm\new-updated -vghetto-deployment.ps1:923
    Line |
    923 | … AllOfRules (New-SpbmRule -AnyOfTags (Get-Tag $StoragePolicyTagName))) …
    | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | All the tags should be from same connection.

    Reply
  33. CSavage says

    02/19/2021 at 7:32 am

    I must be doing something completely wrong. I changed the script for my network specifics and some of the names etc and ensured I met all of the requirements, but the scripts fails almost immediately with various errors like:

    New-NetworkAdapter: C:\ESX\vghetto-crs-vsphere-lab-deployment.ps1:554
    Line |
    554 | New-NetworkAdapter -VM $vm -Type Vmxnet3 -NetworkName $NSXVTE …
    | ~~~
    | Cannot process argument transformation on parameter 'VM'. Object reference not set to an instance of
    | an object.

    New-NetworkAdapter: C:\ESX\vghetto-crs-vsphere-lab-deployment.ps1:555
    Line |

    and then finally fails and stops with:

    [02-19-2021_10:19:24] Creating VCSA JSON Configuration file for deployment ...
    WARNING: Resulting JSON is truncated as serialization has exceeded the set depth of 2.
    [02-19-2021_10:19:24] Deploying the VCSA ...
    Template structure validation failed for template
    C:\Users\crsav\AppData\Local\Temp\jsontemplate.json.
    The key 'datacenter' in section 'new_vcsa' subsection 'vc' is required. Its
    value cannot be null or empty.
    The key 'datastore' in section 'new_vcsa' subsection 'vc' is required. Its value
    cannot be null or empty.
    Error message: Template structure validation failed for template
    C:\Users\crsav\AppData\Local\Temp\jsontemplate.json.
    [02-19-2021_10:19:26] Setting up NSX-T Edge to join NSX-T Management Plane ...
    Connect-NsxtServer: C:\ESX\vghetto-crs-vsphere-lab-deployment.ps1:683
    Line |
    683 | if(!(Connect-NsxtServer -Server $NSXTMgrHostname -Username $NSXAd …
    | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    | 2/19/2021 10:19:26 AM Connect-NsxtServer Unable to connect to the remote server

    Unable to connect to NSX-T Manager, please check the deployment

    This is my first time trying one of the scripts, I have the kubernetes portion marked out (per github FAQ) as I just want the environment with NSX-T. Any help? This is a C240 running 6.7

    Reply
    • murat says

      05/09/2021 at 7:16 am

      You can try the Brad's solution.

      "The fix for the error above was to set Powershell to ignore the SSL certificate.

      Be sure to "Set-PowerCLIConfiguration -InvalidCertificateAction:Ignore" before you run the script. perhaps add it to the script and once finished set back to default."

      Reply
  34. CSavage says

    02/19/2021 at 11:57 am

    So reading through some of the other comments,I downloaded a newer version of the NSX-T Manager and Edge to ensure the LE was not tripping it up. I also cannot seem to change the JSON depth error that keeps coming up although not sure that is actually stopping it. I continue to get the error:

    [02-19-2021_02:55:49] Creating VCSA JSON Configuration file for deployment ...
    WARNING: Resulting JSON is truncated as serialization has exceeded the set depth of 2.
    [02-19-2021_02:55:49] Deploying the VCSA ...
    Template structure validation failed for template
    C:\Users\crsav\AppData\Local\Temp\jsontemplate.json.
    The key 'datacenter' in section 'new_vcsa' subsection 'vc' is required. Its
    value cannot be null or empty.
    The key 'datastore' in section 'new_vcsa' subsection 'vc' is required. Its value
    cannot be null or empty.
    Error message: Template structure validation failed for template
    C:\Users\crsav\AppData\Local\Temp\jsontemplate.json.
    [02-19-2021_02:55:52] Setting up NSX-T Edge to join NSX-T Management Plane ...
    Connect-NsxtServer: C:\ESX\crstest.ps1:684
    Line |
    684 | if(!(Connect-NsxtServer -Server $NSXTMgrHostname -Username $NSXAd …
    | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    | 2/19/2021 2:55:52 PM Connect-NsxtServer Unable to connect to the remote server

    Unable to connect to NSX-T Manager, please check the deployment
    PS C:\ESX>

    And it kicks back out. I have confirmed the execution policy, updated PowerShell etc etc. I am POSITIVE I am doing something stupid, but for the life of me cannot see where I am going wrong here. Any help appreciated.

    Reply
  35. ninefivepm says

    03/04/2021 at 8:41 am

    With 7.0u1 vCLS vm fails to start due to insufficient resources. Are you aware of any workaorunds for vsan datastore. I guess I can workaround using a different iscsi datastore.

    Reply
    • ninefivepm says

      03/05/2021 at 7:19 am

      I added a dedicated vmk interface(same vlan as management interface) enabled with vsan and the issue seems to have dissappeared.

      Reply
  36. David P. says

    03/19/2021 at 5:11 am

    Hello,

    I just try the last version (Mar 13, 2021), I get a little error about "TKGContentLibraryName " variable, It seems to be not filled in.
    I guess It's related with the library tanzu, surely "https://wp-content.vmware.com/v2/latest/lib.json"

    not really a issue because we do it after, but I just notice it.

    1007 | … New-ContentLibrary -Server $vc -Name $TKGContentLibraryName -Descri …
    | ~~~~~~~~~~~~~~~~~~~~~~
    | Cannot bind argument to parameter 'Name' because it is null.

    Reply
    • David P. says

      03/19/2021 at 6:10 am

      I answer myself, just add the following variable :

      $TKGContentLibraryName = "TanzuLib"
      $TKGContentLibraryURL = "https://wp-content.vmware.com/v2/latest/lib.json"

      Reply
  37. Jeroen says

    03/20/2021 at 10:10 am

    I had several runs but whatever I do I keep getting an error when adding vCenter as a compute manager: A server error occurred: 'com.vmware.vapi.std.errors.invalid_request'. When I try it afterwards in the GUI I get a similar error: cannot register compute manager server vcsa.cpbu.corp, credentials are incorrect or invalid compute manager. (Error code: 7061).

    I tried it with several versions of vCenter (7.0.0, 7.0.1 and 7.0.2) and two versions of NSX: nsx-unified-appliance-3.1.1.0.0.17483186.ova and nsx-unified-appliance-3.1.0.0.0.17107212-le.ova.

    What am I missing? Could the issue be in the underlying ESXi host? Or vSwitch?

    Reply
    • Jeroen says

      03/22/2021 at 2:13 am

      I think my issues were related to time... After changing the time server the vCenter can be added as Compute Manager.

      Reply
  38. Johannes says

    03/24/2021 at 10:04 am

    Hi there!

    I am using Workstation 16 for my Homelab (with 1 nested ESX, 1 vCenter, 1 NSX-T Manager and 1 NSX-T Edge). Everything works (NSX Overlay, Edge Uplink, etc) and is prepared for Tanzu (Content-Library, Storage policies, ...)

    When I try to enable Workload Management, the Supervisor Control Plane VMs get deployed (I use 2 master VMs), they get IPs and are pingable. But then the setup runs forever (wcp log: Supervisor node bootstrap configure operation is still in progress...)

    Has anybody experienced this issue?

    BR Johannes

    Reply
    • kastro says

      03/24/2021 at 12:48 pm

      Hi

      I had similar issue but with HAproxy and VDS. Anyway, after I input DNS suffix (it should be optional) while enabling Workload it finished OK.

      Reply
      • N says

        03/31/2021 at 8:59 pm

        When you refer to "DNS Suffix", do you mean the "DNS Search Domain" when enabling Workload Management?

        Reply
        • kastro says

          03/31/2021 at 11:42 pm

          Yes, I meant “DNS Search Domain”.

          Reply
    • N says

      03/31/2021 at 8:28 pm

      Did you ever find a fix for this? Running into same problem when enabling Workload Management. I do have a DNS entry for the 3 Supervisor Control Plane VMs and I did specify the search domain when setting up Workload Management. Thank you!

      Reply
      • Johannes says

        03/31/2021 at 11:16 pm

        Yes, the solution is to use a DNS server on your PC for name resolution. The integrated DNS server in Workstation does not work correctly.

        Reply
        • Jeremy says

          09/27/2021 at 11:14 am

          @Johannes - I know this is old, but I am running into the same issue and not really sure what you mean by this statement - I have the DNS search domain filled out, but it is still hanging on this. What exactly do you mean by this comment that you said fixes it?

          Reply
          • Johannes says

            09/27/2021 at 11:48 pm

            My issue was related to VMware Workstation and DNS resolution. I solved this problem by using a dedicated DNS server on my PC for proper name resolution. VMware Workstation provides an internal DNS server, but which is not working correctly - this was the reason why the supervisor vms were not deploying correctly.

  39. N says

    04/01/2021 at 10:23 am

    After enabling Workload Management successfully and creating a namespace (and assigning it storage), I try to log in via kubectl (through powershell) but getting this error. Also should be noted that I have the kubectl in my environment variables already.

    KUBECTL_VSPHERE_PASSWORD environment variable is not set. Please enter the password below.

    Has anyone gotten this?

    Reply
  40. Jay Scheponik says

    05/28/2021 at 5:24 am

    This tool is awesome. One thing I was wondering, can you provide a sample of how to edit the Default Layer3 Rule from ALLOW to DROP? Looking to deploy this as a ZTN.

    Reply
  41. craigiej says

    09/28/2021 at 4:20 am

    william please help i am coming up with a lot of errors for example

    Import-VApp: C:\Users\Administrator\Downloads\TANZU\william.ps1:632
    Line |
    632 | … $vm = Import-VApp -Source $NestedESXiApplianceOVA -OvfConfigura …
    | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    | 28/09/2021 12:10:57 Import-VApp Network 'SJC-CORP-MGMT' is not accessible from host with id
    | 'HostSystem-host-14'

    can you give me some pointers of what is going wrong please?

    Reply
  42. abbed says

    01/21/2023 at 1:45 am

    Hi William,
    Thank you so much for all the good work,
    i made issue #30 github just for that

    additional code that is required to make use of the 2nd edge
    at end of line 1450 after member_index
    add [0]
    member_index[0]

    that's it

    Cheers

    Abbed strivevirtually.net

    Reply

Thanks for the comment! Cancel reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Search

Author

William Lam is a Senior Staff Solution Architect working in the VMware Cloud team within the Cloud Infrastructure Business Group (CIBG) at VMware. He focuses on Cloud Native technologies, Automation, Integration and Operation for the VMware Cloud based Software Defined Datacenters (SDDC)

Connect

  • Email
  • GitHub
  • LinkedIn
  • RSS
  • Twitter
  • Vimeo

Recent

  • How to disable the Efficiency Cores (E-cores) on an Intel NUC? 03/24/2023
  • Changing the default HTTP(s) Reverse Proxy Ports on ESXi 8.0 03/22/2023
  • NFS Multi-Connections in vSphere 8.0 Update 1 03/20/2023
  • Quick Tip - How to download ESXi ISO image for all releases including patch updates? 03/15/2023
  • SSD with multiple NVMe namespaces for VMware Homelab 03/14/2023

Advertisment

Privacy & Cookies: This site uses cookies. By continuing to use this website, you agree to their use.
To find out more, including how to control cookies, see here: Cookie Policy

Copyright WilliamLam.com © 2023

 

Loading Comments...