A BOSH stemcell is a versioned operating system image.
You must create a BOSH stemcell for Windows before you can deploy these products on vSphere:
To create a Windows stemcell for vSphere, you create a base Windows VM from a volume licensed ISO and subsequently maintain that base template with all Windows-recommended security updates, but without the BOSH dependencies.
The Windows VM with security updates serves as the base for all future stemcells produced from clones of that base VM. This enables you to build new stemcells without having to run Windows updates from the beginning each time. You can also use a “snapshot” feature to maintain an updated Windows image that does not contain the BOSH dependencies.
VMware recommends installing any available updates and then rebuilding the stemcell from a clone of the original VM.
The BOSH stemcell that you create in this topic is based on Windows Server 2019. If you already have a BOSH stemcell for Windows on vSphere, see Monthly stemcell upgrades.
For more information, see Best Practices for Stembuild for Tanzu Application Service & Tanzu Kubernetes Grid Integrated Edition in VMware Tanzu Tech Tutorials.
Stembuild is a binary that you use to build BOSH stemcells for Windows Server 2019.
Stembuild creates a BOSH stemcell from a base Windows image. The stembuild CLI has two commands, construct and package, which you run against a Windows Server 2019 VM. Step 4: Construct the BOSH stemcell and Step 5: Package the BOSH stemcell as follows explain how to run these commands.
Before using stembuild to create a stemcell, you must create a Windows Server 2019 VM and update the VM with the latest Windows updates. Step 1: Create a base VM for the BOSH stemcell, Step 2: Configure the base VM, and Step 3: Clone the base VM explain how to prepare the Windows Server 2019 VM.
Before you create a BOSH stemcell for Windows on vSphere, you must have:
A vSphere environment that makes sure the VM hardware used by the stemcell is compatible with your deployment environment’s ESXi/ESX host, and vCenter Server versions. For more information about the vSphere environment, see ESXi/ESX hosts and compatible virtual machine hardware versions list (2007240) in the Broadcom Support Knowledge Base.
An ISO for a Windows Server 2019 Server Core installation, build number: 17763, from Microsoft Developer Network (MSDN) or Microsoft Volume Licensing Service Center (VLSC). The Windows Server 2019 ISO must be a clean, base ISO file. You can use an evaluation copy for testing, but VMware discourages an evaluation copy for production, because the licensing expires. For more information, see the Windows Server documentation or the Microsoft Volume Licensing Service Center website.
A clean ISO file has no custom scripts or tooling. For example, the ISO must have no logging or antivirus tools installed.
A download of the stembuild
command line interface (CLI) from a 2019.x release in Stemcells (Windows) on Broadcom Support.
stembuild
CLI for Windows corresponding to both the operating system of your local host and the stemcell version that you want to build.stembuild
CLI version listed as compatible in the Product Snapshot table in the release notes for your TKGI version: Release NotesMicrosoft Local Group Policy Object Utility (LGPO) downloaded to the same folder as your stembuild
CLI.
The minimum vCenter user permissions required to use stembuild
for vSphere stemcells, specifically:
VirtualMachine.GuestOperations.Modify
VirtualMachine.GuestOperations.Execute
VirtualMachine.GuestOperations.Query
VirtualMachine.Config.AddRemoveDevice
VirtualMachine.Interact.SetCDMedia
VApp.Export
System.Anonymous
*System.Read
*System.View
*Permissions designated with an *
are generated when you create a new user in vCenter and cannot be set within the vCenter UI.
Learn how to create, configure, and verify a base VM for Windows from a volume licensed ISO.
Note The instructions in this topic are based on vSphere v6.0.
To upload the Windows Server 2019 ISO to vSphere:
Log in to the vSphere Web Client.
Click Storage and select a datastore.
Select or create a folder in which to upload the Windows Server 2019 ISO.
Click Upload a File and select the Windows Server 2019 ISO.
You can use the scp
utility instead of the vSphere Web Client to copy the file directly to the datastore server.
To create and customize a base VM:
In the vSphere Web Client, click the VMs and Templates view to display the inventory objects.
Right-click an object and click New Virtual Machine, then New Virtual Machine.
On the Select a creation type page, click Create a new virtual machine and click Next.
Click Next.
On the Select a compute resource page, select a compute resource to run the VM and click Next.
On the Select storage page:
Click Next.
On the Select compatibility page, for the Compatible with configuration setting, select ESXi 6.0 and later and click Next.
On the Select a guest OS page:
Click Next.
On the Customize hardware page, configure the VM hardware using the information as follows and click Next.
For New CD\DVD Drive:
Review the configuration settings on the Ready to complete page, and click Finish.
To install Windows Server on the base VM:
Click Custom installation.
Complete the installation and enter a password for the administrator user.
To verify that you are using the correct OS version, run this PowerShell command on the base VM:
[System.Environment]::OSVersion.Version
The output displays:
[System.Environment]::OSVersion.Version Major Minor Build Revision ---- ---- ----- -------- 10 0 17763 0
To install VMware Tools on the base VM:
In the vSphere Web Client, right-click the base VM and select Guest OS, then Install VMware Tools.
Click Mount.
Log in to the VM.
Go to the D:
drive.
To install the VMware Tools, run:
setup64.exe
Wait for a pop-up window.
Note The VMware Tools install window might appear behind the CLI window. To avoid this, shrink or minimize the CLI window while you are waiting.
When prompted, follow the instructions to finish the installation.
To complete the installation, start the VM again.
Install all the Windows Server updates so that you have the latest, most secure, version of the Windows Server operating system.
To configure the base VM network settings and install Windows updates:
From the vSphere Web Client, right-click the base VM and select Open Remote Console.
On the command line, enter sconfig
to run the SConfig utility.
On the Server Configuration page, in Network Settings, enter 8
.
On the Network Settings pane:
1
to set the network adapter address.S
to set a static IP address.2
to set a DNS server.4
to return to the main menu.On the Server Configuration page, in Download and Install Updates, enter 6
.
Enter A
to search for all updates.
For Select an option, enter A
to install all updates. You might need to restart the base VM while installing the updates, if so run Download and Install Updates again after reboot until no more updates are found.
On the vSphere Web Client, click Actions, then click Edit Settings on the VM.
In the CD/DVD drive 1 row deselect the Connected check box. Do not remove the CD/DVD drive.
Restart the VM.
After the VM restarts verify that you can ping the IP address you assigned to the base VM.
To clone the base VM:
From the vSphere Web Client, power off the base VM. This is important because your base VM and the clone VM you create share the same IP address.
Right-click the base VM.
Click Clone, then click Clone to Virtual Machine. This clone is your target VM.
Save the base VM. You run Windows updates on this VM for future stemcells.
Take snapshots of both your base and target VMs. If there is an issue when you run stembuild, use these snapshots to revert to a clean state. For more information, see Managing snapshots in vSphere Web Client in the Broadcom Support Knowledge Base.
To start the target VM, click Power > Power On in the Actions tab for your VM.
To create your BOSH stemcell, follow these steps:
If you are running govc, a CLI for vSphere operations, unset any GOVC_
environment variables before you run stembuild construct
.
For each of the following environment variables, run echo $GOVC_VAR
to record the current setting, and then run export GOVC_VAR=
to temporarily unset it:
GOVC_USERNAME
GOVC_INSECURE
GOVC_PASSWORD
GOVC_URL
Explanation: Local GOVC
parameters that are set to configure govc
take precedence over command-line parameters that stembuild construct
passes in to govc
, which can cause stembuild construct
to fail.
Collect the following information from the vCenter Web Client Inventory > VMs and Templates tab:
/YOUR-DATA-CENTER/vm/YOUR-FOLDER/YOUR-VM
format, where:
YOUR-DATA-CENTER
is the name of the data center.vm
is a static string.YOUR-FOLDER
is the name of the folder that contains the VM. If the target VM is not in a folder, use the /YOUR-DATA-CENTER/vm/YOUR-VM
format instead.YOUR-VM
is the name of the target VM.The target VM must be routable from your local host. Before you run the construct
command, you must be logged out of the target VM.
To construct the BOSH stemcell, run this command from the folder where you downloaded the CLI:
./STEMBUILD-BINARY construct ^
-vm-ip 'TARGET-VM-IP' ^
-vm-username TARGET-USERNAME ^
-vm-password 'TARGET-VM-PASSWORD' ^
-vcenter-url VCENTER-URL ^
-vcenter-username VCENTER-USERNAME ^
-vcenter-password 'VCENTER-PASSWORD' ^
-vm-inventory-path 'INVENTORY-PATH' ^
-vcenter-ca-certs 'CUSTOM-CERTS-PATH'
Where:
STEMBUILD-BINARY
is the stembuild
file for the version of your local host operating system and the version of the stemcell that you want to build. For example, stembuild-windows-2019-2
.TARGET-VM-IP
is the IP address of your target VM.TARGET-USERNAME
is the user name of an account with administrator privileges.TARGET-VM-PASSWORD
is the password for the administrator account. The password must be enclosed in single quotes.VCENTER-URL
is the URL of your vCenter.VCENTER-USERNAME
is the user name of your account in vCenter.VCENTER-PASSWORD
is your password. The password must be enclosed in single quotes.INVENTORY-PATH
is the vCenter inventory path to the target VM.CUSTOM-CERTS-PATH
is the file path to custom CA certificates for the destination vCenter. The -vcenter-ca-certs
flag is optional.For more information, see Authenticate into a destination vCenter using CA certificates as follows.
For example:
./STEMBUILD-BINARY construct -vm-ip '192.0.2.254' -vm-username user001 -vm-password 'P1a2s3Sword5' -vcenter-url example.com -vcenter-username user002 -vcenter-password 'P1a2s3Sword5' -vm-inventory-path '/datacenter/vm/folder/test-vm' 2020-01-29T08:52:26.4523812-08:00 Successfully created stemcell version file. Finished executing setup script. WinRM has been disconnected so the VM can reboot. Preparing the VM to be shutdown. 2020-01-29T16:53:27.505532+00:00 Still preparing VM... 2020-01-29T16:54:27.94085+00:00 Still preparing VM... 2020-01-29T16:55:28.374568+00:00 Still preparing VM... Stembuild construct has finished running and the VM has now been shutdown. Run 'stembuild package' to finish building the stemcell.
Caution Your stembuild construct
operation can take up to an hour to complete. Although the WinRM connection stops during construct
processing, construct
is still running. Do not attempt to run the construct
command again.
This example shows the messages that appear as stembuild construct
runs:
2020-01-01T00:00:00 Successfully created stemcell version file. Finished executing setup script. WinRM has been disconnected so the VM can reboot. Preparing the VM to be shutdown. 2020-01-01T00:01:00 Still preparing VM... 2020-01-01T00:02:00 Still preparing VM... 2020-01-01T00:03:00 Still preparing VM... Stembuild construct has finished running and the VM has now been shutdown. Run 'stembuild package' to finish building the stemcell.
(Optional) To monitor the status of your construct
job:
Log in to the target VM.
Start PowerShell.
Run:
Get-Content -Path "C:\provision\log.log" -Wait
Restore the values of any govc
variables you unset previously.
For more information about stembuild construct
, see stembuild construct in the Cloud Foundry stembuild GitHub repository.
To package the BOSH stemcell:
Gather the vCenter Web Client VMs and Templates information that you recorded earlier in the procedure:
/YOUR-DATA-CENTER/vm/YOUR-FOLDER/YOUR-VM
format, where:YOUR-DATA-CENTER
is the name of the data center.vm
is a static string.YOUR-FOLDER
is the name of the folder that contains the VM. If the target VM is not in a folder, use the /YOUR-DATA-CENTER/vm/MY-VM
format instead.YOUR-VM
is the name of the target VM.Stop the VM. If you do not stop the VM before you continue to the next step, the package command fails, and states that the storage location for the VM cannot be read.
To package the BOSH stemcell, run this PowerShell command from your local host:
./STEMBUILD-BINARY package ^
-vcenter-url VCENTER-URL ^
-vcenter-username VCENTER-USERNAME ^
-vcenter-password VCENTER-PASSWORD ^
-patch-version PATCH-VERSION ^
-vm-inventory-path 'INVENTORY-PATH' ^
-vcenter-ca-certs 'CUSTOM-CERTS-PATH'
Where:
STEMBUILD-BINARY
is the stembuild
file for the version of your local host operating system and the version of the stemcell that you want to build. For example, stembuild-windows-2019-2
.VCENTER-URL
is the URL of your vCenter.VCENTER-USERNAME
is the user name of your account in vCenter.VCENTER-PASSWORD
is your password. The password must be enclosed in single quotes.PATCH-VERSION
is the patch version for the stemcell being built.PATCH-VERSION
is an unquoted version name, for example, 2019.12.3
, or a quoted patch number, for example, “3”
.INVENTORY-PATH
is the vCenter inventory path to the target VM.CUSTOM-CERTS-PATH
is the file path to custom CA certificates for the destination vCenter. The -vcenter-ca-certs
flag is optional.For more information, see Authenticate into a destination vCenter using CA certificates.This command creates a stemcell on your local host in the folder where you ran the command and can take up to 30-minutes to complete.
For more information about stembuild package
, see stembuild package in the Cloud Foundry stembuild GitHub repository.
To upload the BOSH stemcell to Tanzu Operations Manager:
In Tanzu Operations Manager, go to Stemcell Library.
Upload your BOSH stemcell.
Deploy the TAS for VMs [Windows] or TKGI tile.
For more information about stembuild
, see Stembuild in the Cloud Foundry stembuild GitHub repository.
After Microsoft Corporation releases operating system updates, you can upgrade your BOSH stemcell.
To upgrade your BOSH stemcell:
Install Windows updates on the base VM.
Replace the existing stemcell in the Tanzu Operations Manager stemcell library with this new stemcell.
Deploy the TAS for VMs [Windows] or TKGI tile.
Symptom
You authenticate with vCenter and see this error:
./out/stembuild: ServerFaultCode: Cannot complete login due to an incorrect user name or password. vcenter_client - unable to validate url: vcenter.example.com
Explanation
stembuild
uses govc libraries. These libraries cannot parse the special characters /
, #
, and :
. This causes errors when authenticating with vCenter.
You might also experience this issue on Windows if your password includes a single quote character, '
. This also affects the Inventory path if it contains a single quote or a space.
Workaround
If your vCenter user name or password contains /
, #
, or :
, or '
on Windows, set these environment variables:
For Linux:
export GOVC_USERNAME=VCENTER-USERNAME
export GOVC_PASSWORD=VCENTER-PASSWORD
For Windows:
set GOVC_USERNAME=VCENTER-USERNAME
set GOVC_PASSWORD=VCENTER-PASSWORD
set GOVC_PATH=VCENTER-INVENTORY-PATH
Where:
VCENTER-USERNAME
is your vCenter account user name. For example, johndoe
.VCENTER-PASSWORD
is your vCenter account password. For example, pass#word
.VCENTER-INVENTORY-PATH
is the location of your VM in the cluster inventory.If you use other special characters, add single quotes around the input parameters, or set them in an environment variable.
For example:
For Linux:
./STEMBUILD-BINARY package \
-vcenter-url VCENTER-URL \
-vcenter-username 'admin@' \
-vcenter-password VCENTER-PASSWORD \
-patch-version PATCH-VERSION \
-vm-inventory-path 'INVENTORY-PATH' \
-vcenter-ca-certs 'CUSTOM-CERTS-PATH'
Where:
STEMBUILD-BINARY
is the stembuild
file for the version of your local host operating system and the version of the stemcell that you want to build. For example, stembuild-windows-2019-2
.VCENTER-URL
is the URL of your vCenter.VCENTER-PASSWORD
is your password. The password must be enclosed in single quotes.PATCH-VERSION
is the patch version for the stemcell being built.PATCH-VERSION
is an unquoted version name, for example, 2019.12.3
, or a quoted patch number, for example, “3”
.INVENTORY-PATH
is the vCenter inventory path to the target VM.CUSTOM-CERTS-PATH
is the file path to custom CA certificates for the destination vCenter. The -vcenter-ca-certs
flag is optional. For more information, see Authenticate into a destination vCenter using CA certificates.For Windows:
set GOVC_PASSWORD=VCENTER-PASSWORD
./STEMBUILD-BINARY package ^
-vcenter-url VCENTER-URL ^
-vcenter-username %GOVC_USERNAME% ^
-vcenter-password %GOVC_PASSWORD% ^
-patch-version PATCH-VERSION ^
-vm-inventory-path %GOVC_PATH% ^
-vcenter-ca-certs 'CUSTOM-CERTS-PATH'
Where:
VCENTER-PASSWORD
is your vCenter account password. For example, A_Strange!PAssword@Here#1
.STEMBUILD-BINARY
is the stembuild
file for the version of your local host operating system and the version of the stemcell that you want to build. For example, stembuild-windows-2019-2
.VCENTER-URL
is the URL of your vCenter.PATCH-VERSION
is the patch version for the stemcell being built. PATCH-VERSION
is specified as an unquoted version name, for example, 2019.12.3
, or as a quoted patch number, for example, “3”
.CUSTOM-CERTS-PATH
is the file path to custom CA certificates for the destination vCenter. The -vcenter-ca-certs
flag is optional. For more information, see Authenticate into a destination vCenter using CA certificates.Windows environment variables do not automatically override vCenter command line parameters, so you must specify the environment variables in the vCenter command.
Symptom
You are running stembuild
from a location outside of your destination vCenter and stembuild
is unable to authenticate.
Explanation
You are running stembuild from a location with different vCenter permissions than your target VM.
Solution
You must provide stembuild
with the CA certificates needed to authenticate into your destination vCenter.
To stage CA certificates for stembuild
to use to access a target VM in the destination vCenter:
stembuild
.To use the staged CA certificates with stembuild
, include the optional -vcenter-ca-certs
flag in your stembuild
command as shown in the example construct
and package
command lines.