WilliamLam.com

  • About
    • About
    • Privacy
  • VMware Cloud Foundation
  • VKS
  • Homelab
    • Hardware Options
    • Hardware Reviews
    • Lab Deployment Scripts
    • Nested Virtualization
    • Homelab Podcasts
  • VMware Nostalgia
  • Apple

Custom script bundle is now possible with Auto Deploy in vSphere 6.5

11.23.2016 by William Lam // 10 Comments

It has been some time since I had looked at the Auto Deploy and Host Profile feature in vSphere. As a former customer, I still remember one of the challenges I had in evaluating Auto Deploy and specifically Host Profiles was the fact that it did not cover all the possible ESXi configurations. This made it very difficult to operationalize as you still needed to handle post-configurations through other mechanisms. Trying to keep both solutions did not make sense for me and I ended up opting for the traditional scripted installation method via Kickstart which I had hooks into automate the full ESXi configuration.

In vSphere 6.5, there was a huge effort to significantly improve both Auto Deploy and Host Profile to what customers had expected of this feature, especially around bringing parity between the configurations that could be done using the vSphere Clients and/or vSphere APIs into Host Profiles. In addition, there was also several UI enhancements that now makes it possible to use both Auto Deploy and Image Builder directly from the vSphere Web Client which was never possible before. For more details, be sure to check out the What's New vSphere 6.5 white paper here.

One new feature that I think is worth calling out is the new Script Bundle capability in Auto Deploy. Previously, if a particular configuration was not available via Host Profiles, there was nothing you could really do and you had to write your own custom post-deployment script to apply to the ESXi host. As I mentioned earlier, in vSphere 6.5, we have closed the gap on the ESXi configurations that were not possible using Host Profile and will ensure that will be in sync going forward. Having said that, there are still certain configurations that are not possible today such as creating a custom ESXi firewall rule for example. For these cases, you either had to either hack it up using a method like this or to create a custom ESXi VIB which would then force customers to lower their ESXi's software acceptance level which was not ideal nor acceptable, especially for customers that are security conscious.

[Read more...]

Categories // Automation, ESXi, PowerCLI, vSphere 6.5 Tags // Add-ScriptBundle, auto deploy, ESXi 6.5, PowerCLI, vSphere 6.5

Automated vSphere Lab Deployment for vSphere 6.x

11.21.2016 by William Lam // 95 Comments

For those of you who follow me on Twitter, you may have seen a few tweets from me hinting at a vSphere deployment script that I have been working on. This was something I had initially built for my own lab use but I figured it probably could benefit the larger VMware community, especially around testing and evaluational purposes. Today, I am please to announce the release of my vGhetto vSphere Lab Deployment (VVLD) scripts which leverages the new PowerCLI 6.5 release which is partly why I needed to wait until it was available before publishing.

There are literally hundreds if not more ways to build and configure a vSphere lab environment. Over the years, I have noticed that some of these methods can be quite complex simply due to their requirements, or incomplete as they only handle specific portion of the deployment or add additional constraints and complexity because they are composed of several other tools and scripts which can make it hard to manage. One of my primary goals for the project was to be able to stand up a fully functional vSphere environment, not just deploying a vCenter Server Appliance (VCSA) or a couple of Nested ESXi VMs, but rather the entire vSphere stack and have it fully configured and ready for use. I also wanted to develop the scripts using a single scripting language that was not only easy to use, so that others could enhance or extend it further but also with the broadest support into the various vSphere APIs. Lastly, as a stretch goal, I would love to be able to run this script across the different OS platforms.

With these goals in mind, I decided to build these scripts using the latest PowerCLI 6.5 release. Not only is PowerCLI super easy to use, but I was able to immediately benefit from some of the new functionality that was added in the latest PowerCLI release such as the native VSAN cmdlets which I could use a sub-set of the cmdlets against prior releases of vSphere like 6.0 Update 2 for example. Although, not all functionality in PowerCLI has been ported over to PowerCLICore, you can see where VMware is going with it and my hope is that in the very near future, what I have created can one day be executed across all OS platforms whether that is Windows, Linux or Mac OS X and potentially even ARM-based platforms 🙂

Changelog:

  • 11/22/16
    • Automatically handle Nested ESXi on vSAN
  • 01/20/17
    • Resolved "Another task in progress" thanks to Jason M
  • 02/12/17
    • Support for deploying to VC Target
    • Support for enabling SSH on VCSA
    • Added option to auto-create vApp Container for VMs
    • Added pre-check for required files
  • 02/17/17
    • Added missing dvFilter param to eth1 (missing in Nested ESXi OVA)
  • 02/21/17 (All new features added only to the vSphere 6.5 Std deployment)
    • Support for deploying NSX 6.3 & registering with vCenter Server
    • Support for updating Nested ESXi VM to ESXi 6.5a (required for NSX 6.3)
    • Support for VDS + VXLAN VMkernel configuration (required for NSX 6.3)
    • Support for "Private" Portgroup on eth1 for Nested ESXi VM used for VXLAN traffic (required for NSX 6.3)
    • Support for both Virtual & Distributed Portgroup on $VMNetwork
    • Support for adding ESXi hosts into VC using DNS name (disabled by default)
    • Added CPU/MEM/Storage resource requirements in confirmation screen
  • 04/18/18
    • New version of the script vsphere-6.7-vghetto-standard-lab-deployment.ps1 to support vSphere 6.7
    • Added support for vCenter Server 6.7, some of the JSON params have changed for consistency purposes which needed to be updated
    • Added support for new Nested ESXi 6.7 Virtual Appliance (will need to download that first)
    • vMotion is now enabled by default on vmk0 for all Nested ESXi hosts
    • Added new $enableVervoseLoggingToNewShell option which spawns new PowerShell session to provide more console output during VCSA deploy. FR by Christian Mohn
    • Removed dvFilter code, since thats now part of the Nested ESXi VA

Requirements:

  • 1 x Physical ESXi host OR vCenter Server running at least ESXi 6.0 Update 2 or greater
  • PowerCLI 6.5 R1 installed on a Window system
  • Nested ESXi 6.0 or 6.5 Virtual Appliance OVA
  • vCenter Server Appliance (VCSA) 6.0 or 6.5 extracted ISO
  • NSX 6.3 OVA (optional)
    • ESXi 6.5a offline patch bundle

Supported Deployments:

The scripts support deploying both a vSphere 6.0 Update 2 as well as vSphere 6.5 environment and there are two types of deployments for each:

  • Standard - All VMs are deployed directly to the physical ESXi host
  • Self Managed - Only the Nested ESXi VMs are deployed to physical ESXi host. The VCSA is then bootstrapped onto the first Nested ESXi VM

Below is a quick diagram to help illustrate the two deployment scenarios. The pESXi in gray is what you already have deployed which must be running at least ESXi 6.0 Update 2. The rest of the boxes is what the scripts will deploy. In the "Standard" deployment, three Nested ESXi VMs will be deployed to the pESXi host and configured with vSAN. The VCSA will also be deployed directly to the pESXi host and the vCenter Server will be configured to add the three Nested ESXi VMs into its inventory. This is a pretty straight forward and basic deployment, it should not surprise anyone. The "Self Managed" deployment is simliar, however the biggest difference is that rather than the VCSA being deployed directly to the pESXi host like the "Standard" deployment, it will actually be running within the Nested ESXi VM. The way that this deployment scenario works is that we will still deploy three Nested ESXi VM onto the pESXi host, however, the first Nested ESXi VM will be selected as a "Bootstrap" node which we will then construct a single-node vSAN to then deploy the VCSA. Once the vCenter Server is setup, we will then add the remainder Nested ESXi VMs into its inventory.

vsphere-6-5-vghetto-lab-deployment-0
For most users, I expect the "Standard" deployment to be more commonly used but for other advanced workflows such as evaluating the new vCenter Server High Availability feature in vSphere 6.5, you may want to use the "Self Managed" deployment option. Obviously, if you select the latter deployment, the provisioning will take longer as you are now "double nested" and depending on your underlying physical resources, this can take quite a bit more time to deploy as well as consume more physical resources as your Nested ESXi VMs must now be larger to accommodate the VCSA. In both scenarios, there is no reliance on additional shared storage, they will both create a three node vSAN Cluster which of course you can expand by simply editing the script.

Deployment Time:

Here is a table breaking down the deployment time for each scenario and vSphere version:

Deployment Type Duration
vSphere 6.5 Standard 36 min
vSphere 6.0 Standard 26 min
vSphere 6.5 Self Managed 47 min
vSphere 6.0 Self Managed 34 min

Obviously, your miles will vary based on your hardware configuration and the size of your deployment.

Scripts:

There are four different scripts which covers the scenarios we discussed above:

  • vsphere-6.0-vghetto-self-manage-lab-deployment.ps1
  • vsphere-6.0-vghetto-standard-lab-deployment.ps1
  • vsphere-6.5-vghetto-self-manage-lab-deployment.ps1
  • vsphere-6.5-vghetto-standard-lab-deployment.ps1
  • vsphere-6.7-vghetto-standard-lab-deployment.ps1

Instructions:

Please refer to the Github project here for detailed instructions.

Verification:

Once you have saved all your changes, you can then run the script. You will be provided with a summary of what will be deployed and you can verify that everything is correct before attempting the deployment. Below is a screenshot on what this would look like:

Sample Execution:

Here is an example of running a vSphere 6.5 "Standard" deployment:


Here is an example of running a vSphere 6.5 "Self Managed" deployment:

vsphere-6-5-vghetto-lab-deployment-2
If everything is succesful, you can now login to your new vCenter Server and you should either see the following for a "Standard" deployment:

vsphere-6-5-vghetto-lab-deployment-5
or the following for "Self Managed" deployment:

vsphere-6-5-vghetto-lab-deployment-6
I hope you find these scripts as useful as I do and feel free to enhance these scripts to perform additional functionality or extend them to cover other VMware product deployments such as NSX or vRealize products for example. Enjoy!

Categories // Automation, Home Lab, PowerCLI, VCSA, vSphere 6.0, vSphere 6.5 Tags // homelab, Nested ESXi, nested virtualization, PowerCLI, vSphere 6.5

5 ways to a run PowerCLI script using the PowerCLI Docker Container

10.25.2016 by William Lam // 5 Comments

In case you missed the exciting update last week, the PowerCLI Core Docker Container is now hosted on Docker Hub. With just two simple commands you can now quickly spin up a PowerCLI environment in under a minute! This is really useful if you need perform a couple of operations using the cmdlets interactively and then discarding the environment once you are done. If you want to do something more advanced like run an existing PowerCLI script as well as potentially persist its output (Docker Containers are stateless by default), then there are few options to consider.

To better describe the options, lets use the following scenario. Say you have a Docker Host, this can be a VMware's Photon OS or a Microsoft Windows, Linux or Mac OS X system which has the Docker Client running. The Docker Host is where you will run the PowerCLI Core Docker Container and it also has access to a collection of PowerCLI scripts that you have created or downloaded else where. Lets say the location of these PowerCLI scripts are located in /Users/lamw/scripts and you would like them to be available within the PowerCLI Core Docker Container when it is launched, say under /tmp/scripts.

Here is a quick diagram illustrating the scenario we had just discussed.

4-different-ways-to-use-powercli-core-docker-containerHere are 5 different ways in which you can run your PowerCLI scripts within the Docker Container. Each will have its pros/cons and I will be using real sample scripts to exercise each of the options. You can download all the sample scripts in my Github repository: powerclicore-docker-container-samples

Note: Before getting started, please familiarize yourself with launching the PowerCLI Core Docker Container which you can read more about here. In addition, you will need access to either a vCenter Server or ESXi host environment and also please create a tiny "Dummy" VM called DummyVM which we will be using to update its Notes field with the current time.

UPDATE (04/11/18) - Microsoft has GA'ed PowerShell Core, one of the changes is the name of the PS binary from powershell to pwsh. For entrypoint parameter, you will need to specify /usr/bin/pwsh rather than /usr/bin/powershell

Option 1:

This is the most basic and easiest method. You literally run a PowerCLI script that already contains all of the necessary information hardcoded within the script itself. This means things like credentials as well as user input that is required can be found within the script. This is obviously simple but makes it very inflexible as you would need to edit the script before launching the container. Another downside is that you now have your vSphere credentials hardcoded inside of the script which is also not ideal from a security standpoint.

To exercise example 1, please edit the pcli_core_docker_sample1.ps1 script and update it with your environment credentials and then run the following command:

docker run --rm -it \
-v /Users/lamw/scripts:/tmp/scripts vmware/powerclicore /tmp/scripts/pcli_core_docker_sample1.ps1

If executed correctly, the Docker container should launch, connect to your vSphere environment, update the notes field of DummyVM with the current time and then exit. Pretty straight forward and below is a screenshot of this example.

run-powercli-scripts-using-powercli-core-docker-container-0

Option 2:

Nobody likes hardcoding values, especially when it comes to endpoints and credentials. This next method will allow us to pass in variables from the Docker command-line and make them available to the PowerCLI scripts inside of the container as OS environmental variables. This allows for greater flexibility then the previous option but the downside is that you may potentially be exposing credentials in plaintext which can be inspected by others who can perform docker run/inspect commands. You also need to update your existing PowerCLI scripts to handle the environmental variable translation which may not be ideal if you have a lot of pre-existing scripts.

To exercise example 2, run the following command and specify your environmental credentials in the command-line instead:

docker run --rm -it \
-e VI_SERVER=192.168.1.150 \
-e VI_USERNAME=*protected email* \
-e VI_PASSWORD=VMware1! \
-e VI_VM=DummyVM \
-v /Users/lamw/scripts:/tmp/scripts vmware/powerclicore /tmp/scripts/pcli_core_docker_sample2.ps1

If executed correctly, you will see that the variables that we have defined are passed into the container and we are now able to make use of them within the PowerCLI script by simply accessing the respective environmental variable names as shown in the screenshot below.

run-powercli-scripts-using-powercli-core-docker-container-1

Option 3:

If you have created some PowerCLI scripts which already prompt for user input which can include also include credentials, then another way to run those script is to do so interactively. If the parameters are required for a given script, then it should prompt for input. The benefit here is that you can reuse your existing PowerCLI scripts without needing to make any modifications even when executing it within a Docker container. You are also not exposing any credentials in plaintext. To take this step further, you could also implement the secure string feature in PowerShell but that would still require you to include a small snippet in your PowerCLI script to do the appropriate decoding when connecting.

To exercise example 3, run the following command and specify your environmental credentials in the command-line instead:

docker run --rm -it \
-v /Users/lamw/scripts:/tmp/scripts vmware/powerclicore /tmp/scripts/pcli_core_docker_sample3.ps1

If executed correctly, you will be prompted for the expected user inputs to the script and then it will perform the operation as shown in the screenshot below.

run-powercli-scripts-using-powercli-core-docker-container-2

Option 4:

Similiar to Option 3, if you have defined parameters to your PowerCLI script, you can also just specify them directly in the Docker command-line just like you would if you were to manually run the PowerCLI script in a Windows environment. Again, the benefit here is that you can reuse your existing PowerCLI scripts without any modifications. You do risk exposing any credentials if you are passing it through the command-line, but the risk was known as you are already doing that with your existing scripts. A downside to this option is if your PowerCLI script accepts quite a few parameters, your Docker run command can get quite long. You may just consider prompting for endpoint/credentials and the rest of the user input can then be passed in dynamically if you were to go with this option.

To exercise example 4, run the following command and specify your environmental credentials in the command-line instead:

docker run --rm -it \
-v /Users/lamw/scripts:/tmp/scripts vmware/powerclicore /tmp/scripts/pcli_core_docker_sample3.ps1 -VI_SERVER 192.168.1.150 -VI_USERNAME *protected email* -VI_PASSWORD VMware1! -VI_VM DummyVM

run-powercli-scripts-using-powercli-core-docker-container-3

Option 5:

The last option is a nice compromise of the above in which you can continue leveraging your existing scripts but providing a better way of sending in things like credentials. As I mentioned before, Docker Volumes allows us to make directories and files available from our Docker Host to the Docker Container. This not only allows us to make our PowerCLI scripts available from within the container but it can also be used to provide access to other things like simply sourcing a credentials file. This method works on a per-individual basis running the container without any major modification to your existing scripts, you simply just need to source the credential file at the top of each script. Best of all, you are not exposing any sensitive information

Note: Some of you might be thinking about PowerCLI's credential store and seeing how that might be a better solution but currently today that has not been implemented yet in PowerCLI Core which is really leveraging Microsoft's credential store feature. Once that has been implemented in .NET Core, I am sure the PowerCLI team can then add that capability which is probably the recommended and preferred option both from a security perspective as well as Automation standpoint.

To exercise example 5, edit the credential.ps1 file and update it with your environmental credentials and run the following command:

docker run --rm -it \
-v /Users/lamw/scripts:/tmp/scripts vmware/powerclicore /tmp/scripts/pcli_core_docker_sample4.ps1 -VI_VM DummyVM

If executed correctly, the same variables in the credentials file will then be loaded into the PowerCLI script context and run the associated operations and exit.

run-powercli-scripts-using-powercli-core-docker-container-4
As you can see, there are many different ways in which you can run your existing PowerCLI scripts using the new PowerCLI Core Docker Container. Hopefully this article gives you a good summary along with some real world examples to consider. Given this is still an active area of development by the PowerCLI team, if you have any feedback or suggestions, please do leave a comment. I know the Alan (PM) as well as the engineers are very interested in hearing your feedback and seeing how else we could better improve the user experience of both PowerCLI Core as well as consuming PowerCLI Core through these various interfaces.

UPDATE (10/25/16) - It looks like PowerCLI Core Docker Container has been updated with my suggestion below, so you no longer need to specify the --entrypoint parameter 🙂

One finale note, right now the PowerCLI Core Docker Container does not automatically startup the Powershell process when it is launched. This is why we have the --entrypoint='/usr/bin/powershell' command appended to the Docker command-line. If you prefer to have Powershell start up which will automatically load the PowerCLI module, you can check out my updated PowerCLI Core Docker Container: lamw/powerclicore which uses the original as a base with one tiny modification. Perhaps this is something Alan and the team would consider making as a default in the future? 🙂

Categories // Automation, Docker, PowerCLI, vSphere Tags // Docker, PowerCLI, powershell

  • « Previous Page
  • 1
  • …
  • 47
  • 48
  • 49
  • 50
  • 51
  • …
  • 56
  • Next Page »

Search

Thank Author

Author

William is Distinguished Platform Engineering Architect in the VMware Cloud Foundation (VCF) Division at Broadcom. His primary focus is helping customers and partners build, run and operate a modern Private Cloud using the VMware Cloud Foundation (VCF) platform.

Connect

  • Bluesky
  • Email
  • GitHub
  • LinkedIn
  • Mastodon
  • Reddit
  • RSS
  • Twitter
  • Vimeo

Recent

  • Automating the vSAN Data Migration Pre-check using vSAN API 06/04/2025
  • VCF 9.0 Hardware Considerations 05/30/2025
  • VMware Flings is now available in Free Downloads of Broadcom Support Portal (BSP) 05/19/2025
  • VMUG Connect 2025 - Minimal VMware Cloud Foundation (VCF) 5.x in a Box  05/15/2025
  • Programmatically accessing the Broadcom Compatibility Guide (BCG) 05/06/2025

Advertisment

Privacy & Cookies: This site uses cookies. By continuing to use this website, you agree to their use.
To find out more, including how to control cookies, see here: Cookie Policy

Copyright WilliamLam.com © 2025

 

Loading Comments...