vCenter 8 Machine SSL Certificate Management

CaptainvOPs

vSphere 8
Windows Server 2019 Certificate Authority
Blog Date: December 16, 2022

Replacing the machine SSL certificate is a breeze in vSphere 7 and 8. Many organizations have security requirements and need for the vSphere web interface to have that secure padlock icon. Most organizations I have come across have a Microsoft Certificate Authority in house, but there are exceptions. In my lab, I have a Microsoft CA configured, and that is what I’ll be using in this example.

VMware has a pretty good KB2097936 article that discusses how to use vSphere Certificate manager.

Installing and managing custom signed certificates is not difficult. Can this be done during normal business hours? I would argue, yes. Some organizations will choose to do it after hours, however. When the certificate is installed, services are restarted on the vCenter to reflect the change. During that minute or two, you won’t be able to manage your vSphere infrastructure from the vSphere client, but all of your ESXi hosts, and all of your workloads will continue to run untouched. If the certificate is bad, you can always revert to snapshot and have everything as it was prior to replacing the SSL certificate in a matter of minutes. Again, the ESXi and VMs will be unaffected. However, what you do need to keep in mind, is that when you replace the certificate on the vCenter, other services that direct connect to the vCenter like Aria (vRealize) Operations, log insight, automation, network insight, etc., would all need to have their connection to the vCenter refreshed. Any third party solutions out there would also need to have their connection to the vCenter refreshed.

  1. I recommend that you take a snapshot prior to messing with certificates.
  2. After you’ve logged into the vSphere client with an admin level account, click the three hash marks next ‘vSphere Client’ in the upper left (some call it a hamburger), and select “Administration” from the menu.
  3. On the Administration page, select “Certificate Management” under ‘Certificates’.
  4. On the Certificate Management screen, you will see Trusted Root Certificate at the bottom and Machine SSL Certificate at the top. In this example, we are only worried about the Machine SSL Certificate. Just below it, you will see an “Actions” drop menu, and from the menu we need to select Generate Certificate Signing Request (CSR).

5. Fill out the specific details for your certificate. Every box should be filled out. Boxes marked optional are just that. I personally fill out the Subject Alternative Name with the FQDN and IP address on the vCenter comma separated. Click NEXT.

6. You can COPY or DOWNLOAD the CSR. In this example, click COPY and then click finish.

7. Now we need to connect to the Microsoft CA web portal. Click “Request a Certificate”.

8. Click advanced certificate request

9. Paste the vCenter CSR created earlier into the saved request box. Next, chose the Certificate Template. I created a VMware template using the VMware knowledge base article here KB2112009.

10. Select “Base 64 encoded”, and then click “Download certificate”.

11. Now that we have downloaded the base 64 encoded certificate for the vCenter, we also need the CA certificate. We can download this from the Microsoft CA Web Portal homepage. Click “Download a CA certificate”

12. For the Encoding Method, select Base 64, then click “Download CA certificate”.

13. In the vSphere client Certificate Management screen, click the Actions drop menu and select “Import and Replace Certificate”.

14. On the Replace vCenter Server Certificate screen, select the “Replace with external CA certificate where CSR is generated from the vCenter Server….” option and click NEXT.

15. Upload the vCenter Machine SSL Certificate in the top box, and the CA certificate in the bottom box. In some configurations, your organization might have more than one CA, so you may need to upload a chain CA root certificate(s). In this example, I only have one CA certificate, so there’s only one to upload. Once you click REPLACE, know that the vCenter will become unavailable while services are restarted.

16. You will either need to dump your existing cookies for the vSphere environment that you just replaced the certificate for, or use private/incognito browser mode and reconnect to the vCenter. Services should restart in a few moments. As long as your certificate was correct, you should now see the secure padlock.

17. Once logged into the vCenter, you can go back to the Certificate Management screen, and you should see the new Machine SSL Certificate. You will also see a new Trusted root certificate has been installed. From this point forward, you will need to manage both the Machine SSL certificate and the CA Trusted Root Certificate that has been installed on the vCenter. Depending on organizational policies, the validity length of these certificates will differ. In my home lab, I have my Microsoft CA configured for 10 years because I know I will rebuild it prior to the certificates expiring, and I don’t want to manage certificates more than I have to.

When everything checks out, don’t forget to clean your room and delete the snapshot.

How To Manually Set The IP Address On Photon OS From Command Line

CaptainvOPs

Blog Date: December 8th 2022.

Out in the field, I have had a Photon OS appliance like Site Recovery Manager, and vRealize Lifecycle Manager deploy without the proper networking configuration. For me, these appliances have been deployed to a network where the Gateway and DNS were reachable, but the appliance will sometimes complete its first boot without successfully configuring the network. Sometimes you can get out of this situation by simply redeploying the appliance. There are those other times where the appliance just refuses to configure the networking, so a manual approach is required.

DISCLAIMER: I have only done this process on VMware Photon OS based appliances that did not successfully complete their first boot. Your mileage may vary attempting this on an appliance that has successfully deployed.

As always – Take snapshots before proceeding.

Commands To Update Photon OS Network Configuration

Using the VMware console, we first need to check and see what interface is being used with this command:

# /opt/vmware/share/vami/vami_interfaces

Next we need to set the interface IP, NETMASK, and GATEWAY with this command:

# /opt/vmware/share/vami/vami_set_network <INTERFACE> STATICV4 <IP_ADDRESS> <NETMASK> <GATEWAY>

To set the domain suffix and DNS, use this command:

# /opt/vmware/share/vami/vami_set_dns -d <domain suffix> <DNS1> <DNS2>

Assuming no errors, reboot the appliance.

In the event those commands do not work because not all Photon Operating Systems are 100% alike, you can also try this method below:


Update Photon OS Network Configuration File:


Let’s first list the network configuration files. To do this, run the following command:

# ls /etc/systemd/network

In this example, we see one configuration file “10-eth0.network”. We will use the next command to edit the file using vi and make the appropriate changes:

# vi /etc/systemd/network/10-eth0.network

Once you are done making the necessary changes, do a “wq!” to save the changes and quit the editor.

Reboot the appliance for the changes to take affect.

vSphere with Tanzu: Deploying and configuring your first Namespace.

CaptainvOPs

Blog Date: November 08, 2022
NSX-ALB Controller version: 22.1.1
vSphere version 7.0.3 Build 20150588

In the previous blog series I covered deploying vSphere with Tanzu using the NSX Advanced Load Balancer. You can find those blog posts below:

1 – vSphere with Tanzu: NSX-ALB Controller Requirements and Deployment Prep
2 – vSphere with Tanzu: Deployment of NSX-ALB Controller
3 – vSphere with Tanzu: Replacing NSX-ALB Controller Certificates
4 – vSphere with Tanzu: Configuring the NSX-ALB Controller
5 – vSphere with Tanzu: Storage Policy and Subscribed Content Library Creation
6 – vSphere with Tanzu: Deploying Tanzu with NSX-ALB

Whether you have a vSphere with Tanzu deployment with NSX-T, a vSphere with Tanzu deployment on VMware Cloud Foundation, or vSphere with Tanzu deployment with NSX-ALB, you will eventually arrive here. In this blog post, I will walk through the process of Deploying and configuring your first Namespace.

On the workload management screen, you can create your first Namespace on the Namespaces tab by clicking CREATE NAMESPACE.

Be sure to create a Namespace with a DNS compliant name. Click CREATE when ready.

You will also want to set permissions on the Namespace to control access.

  1. Click Add Permissions
    1. Identity source: <make selection>
    2. User/Group Search: <customer specific>. In this example, I have created a vsphere.local account. You can easily use an active directory account or group here.
    3. Role: <customer specific>. In this example, I have chosen “can edit” that way I can create and destroy things inside the namespace.
    4. Click Ok
    5. (Rinse-wash-repeat as necessary)

Click Add Storage and add the storage policy. 

The namespace is configured with a storage policy and user permissions.  The assigned storage policy is translated to a Kubernetes storage class.

Under VM Service, click Add VM Class. Here we need to associate a VM class with the namespace, that will allow developers to self-service VMs in the namespace. This gives vSphere administrators flexibility with resources available in the cluster. In this example, best-effort-xsmall was chosen because this is a nested lab environment. You should work with your developers to determine the best sizing strategy for the containerized workloads.

Now that the Namespace, Storage, and VM Class policies have all been defined, your window should look something like:

You are now ready to start deploying Kubernetes workloads to Tanzu.

vSphere with Tanzu: Deploying Tanzu with NSX-ALB

CaptainvOPs

Blog Date: September 08, 2022
NSX-ALB Controller version: 22.1.1
vSphere version 7.0.3 Build 20150588

In this vSphere with Tanzu NSX-ABL series, I have covered several prerequisites and deployments leading up to this final blog in the series, the actual deployment of Tanzu on the NSX-ALB controller. You can find those blogs here:

1 – vSphere with Tanzu: NSX-ALB Controller Requirements and Deployment Prep

2 – vSphere with Tanzu: Deployment of NSX-ALB Controller

3 – vSphere with Tanzu: Replacing NSX-ALB Controller Certificates

4 – vSphere with Tanzu: Configuring the NSX-ALB Controller

5 – vSphere with Tanzu: Storage Policy and Subscribed Content Library Creation

If you have been following along in this blog series of Deploying Tanzu with NSX-ALB controller, I have been using a spreadsheet filled out ahead of time for this deployment. We will be referring to it here to it here as well to deploy Tanzu. Note: In this style of deploying the NSX-ALB controller without NSXT networks, we do not get access to the embedded harbor registry.

Prior to deploying Tanzu, you need to create some storage tags and policies (at least one of each), and create a subscribed content library. I go through that setup here.

In this deployment example, I am only using the NSX-ALB controller with no NSX(NSXT) networks. vSphere Distributed Switch is selected by default. Click NEXT in the lower left.

VMware recommends a minimum of 3 hosts in a compute cluster when enabling Tanzu for production deployments. However, this is a lab and I only have 2 hosts available. Select the desired compute cluster with HA and DRS enabled. If no cluster is compatible,

Select the desired storage policy and click NEXT.

Refer to the spreadsheet, and fill in the information for the NSX-ALB controller. You will also need to log into NSX-ALB and get the certificate for the controller. This can be found on Templates > Security > SSL/TLS Certificates. Click on the down arrow to the right of the controller to export, and then on the new screen that opens, under Certificate, click COPY TO CLIPBOARD.

Click NEXT

Refer to the spreadsheet, and fill in the details for the management network. Remember the control plane needs a block of 5 consecutive IP addresses.

Refer to the spreadsheet, and fill in the details for the workload network. The “Internal Network for Kubernetes Services” default CIDR can be used. You can also specify your own if the default CIDR conflicts with other networks. This is strictly for internal communication.

Select the content library we configured earlier.

Choose the Control Plane Size that fits your needs and add the API server DNS Name. A word of Caution: Even though the “API Server DNS Name(s)” section says ‘Optional’, I would still fill this in. Currently, there is no easy way to add it after the initial deployment.

Click FINISH

The deployment process can 15 to 20 minutes. Sometimes longer depending on the size of the cluster. Good time to grab a drink…

In the vCenter recent tasks window, you should start seeing some deployment activities.

After the deployment completes successfully, there will be a notification bar across the top “TKG plugin has been added”, asking to refresh the browser. After the screen refresh, you should see a config status of Running.

Browsing the vCenter inventory, you should now see a new object called Namespaces, with at least two Avi SE (Service Engine) VMs.

Don’t forget to license the Compute Cluster for Tanzu.

If you haven’t done so already, the next step would be to setup a dev VM to have a common jump server to interact with Tanzu. I have detail that process on the following blog: vSphere with Tanzu: Install Kubernetes CLI Tools from the Tanzu Control Plane on Ubuntu Dev VM

Stay tuned for more content.

vSphere with Tanzu: Storage Policy and Subscribed Content Library Creation

CaptainvOPs

Blog Date: August 16, 2022
vSphere version 7.0.3 Build 20150588

Storage Policies for vSphere with Tanzu

Part of the prep work prior to Tanzu Kubernetes deployment in your environment, is to configure a storage policy or policies for Tanzu workloads. for more information, consult VMware’s Storage Policies for vSphere with Tanzu and vSphere with Tanzu Storage documentation. Here I will walk through the basic configuration we setup for customers to get them started.

Create the Storage Tag

1 – in the vSphere inventory, select the desired storage to house the Tanzu Kubernetes workloads.
2 – Under tags, click assign.

3 – Click Add Tag

4 – Name the tag.  In this example we are using tanzu.

5 – Click the Create New Category, and give it a name.  In this example we used: k8s
Leave all other defaults, and click CREATE.

6 – Click CREATE again to complete the tag setup.

7 – Assign the newly created tanzu tag to the datastore.

Create and Assign a Storage Policy

1 – On the vSphere menu, select “Policies and Profiles”.
2 – Under VM Storage Policies, click CREATE.

3 – Name the storage profile. Make sure to use a DNS compliant name (lowercase, no spaces).  In this example, we used : tanzu-storage. Click NEXT

4 – On Policy structure, Enable tag based placement rules.  Click NEXT

5 – On the Tag based placement screen:
5a – Choose the Tag category: k8s 
5b – Usage option: Use storage tagged with Tags: tanzu
5c – Click NEXT

6 – On the Storage compatibility screen, the tagged compatible datastores will be listed.  Click NEXT

7 – Review and click FINISH.

At this point, we just completed the configuration and assignment of storage tags and policies. Let’s create the subscribed content library for Tanzu.

Create Subscribed Content Library for Tanzu Kubernetes

For more information on creating a subscribed content library for Tanzu Kubernetes, see VMware documentation here.

1 – vSphere Menu, Select Content Libraries, and click CREATE.

2 – Name the new content library (example: tanzu-content-library), select the desired vCenter and click NEXT. 

3 – Configure the content library.
3a – Choose “Subscribed content library”
3b – Enter the Subscription URL: https://wp-content.vmware.com/v2/latest/lib.json
3c – Download content frequency depends on customer needs and bandwidth requirement. There’s roughly 27 OVAs available to download.

4 – Click NEXT

5 – Respond yes to the “tanzu – unable to verify authenticity” message.  This is expected

6 – Select a security policy if needed, otherwise click NEXT.

7 – Select the storage for the content library.

8 – Review the configuration, and click FINISH

Depending on the chosen sync frequency, you may start to see that a sync has started in the vSphere recent tasks window. If you click on the content library, you can see the available OVAs.

This completes the vSphere environment prep for the Tanzu deployment.

vSphere with Tanzu: Configuring the NSX-ALB Controller

CaptainvOPs

Blog Date: August 12, 2022
NSX-ALB Controller version: 22.1.1
vSphere version 7.0.3 Build 20150588

In my previous blog: vSphere with Tanzu: Deployment of NSX-ALB Controller, I went over the basic NSX-ALB controller deployment and activation. In this blog, I will go over the configuration of the controller in preparation of deploying Tanzu to the target workload compute cluster.

Picking up where I left off previously, I just assigned the Essentials license to the controller.

Next, we need to configure our default cloud. 


Configure the Default Cloud

Click on the Infrastructure tab, then select Clouds in the left menu. We see the controller comes with a default cloud already configured, and we can edit this for our needs. To the right of the green dot, you’ll see several icons. Click the gear to the right of the pencil.

On the Convert Cloud Type window, select “VMware vCenter/vSphere ESX” in the cloud type drop menu. Click YES, CONTINUE.

For the Default-Cloud, select “Prefer Static Routes vs Directly Connected Network” under Default Network IP Address Management.

Then under vCenter/vSphere section, click add credentials.

Here you will need to add the FQDN of the vCenter, along with the service account the controller will use to access the vCenter. We can use the example spreadsheet we filled out earlier. Click CONNECT.

This will kick you back to the setup wizard. However we now see a little blue information bar “VMware vCenter/vSphere ESX cloud needs to be created before proceeding. Please ‘Save & Relaunch’ the modal to complete setup.” However, the SAVE & RELAUNCH button in the lower right corner is grayed out. We first need to deselect the “Use Content Library” . Now we can click save & relaunch.

Make sure Data Center drop menu has the desired data center, else select it.

Now we can configure the management network information. Select the management network, add its CIDR and gateway. Under the Static IP Address Pool, we need to click the ADD button. This will need 5 consecutive IP addresses.

Click the SAVE button in the lower left. We will come back and edit this section later on.


________________________________________________________________________________

Configure the NSX-ALB Controller to use a certificate

Now we need to update the NSX-ALB SSL certificate. We can either use a self signed certificate, or we can create a CSR and sign the certificate with a CA. In my lab, I have applied the signed CA certificate.

Check out my blog where I go over both options and how to create them here: vSphere with Tanzu: Replacing NSX-ALB Controller Certificates Applying a certificate to the controller has to be done before proceeding to the next step!
________________________________________________________________________________


Configure the Service Engines

For that, we click the Infrastructure tab, and then on the left we expand Cloud Resources, and select Service Engine Group.

1 – Click the pencil on the default-group.
2 – The default configuration, the Legacy HA is already configured to be Active/Standby.  This is the only mode available to the essentials license.
3 – The number of Virtual Services per Service Engine, is 10 at default. This is the number of load balancing services the SE will support.  Each Tanzu cluster you create, will consume one of these load balancing services, and every service you expose from the tanzu cluster will consume a service.  This can be turned up to 1000 depending on your needs.
4 – The maximum number of service engines is limited by the essentials license used.

Click on the Advanced tab.

Under cluster we need to select the workload cluster that will run Tanzu, and we need to specify all hosts.

Click Save in the lower right.


Configure The NSX-ALB Controller Networks

Now that we have configured the Service Engine in the previous section, we now need to configure our networks.  On the left menu, select Networks under Cloud Resources.  We can see that it has detected all of our available networks in the vCenter. 

You’ll notice that it does not detect the network settings because we are using static IPs instead of DHCP, so first we will edit the Data network. Click the pencil on the right.

Click the ‘+ Add Subnet’ button. Refer to the spreadsheet again, copy the ‘Data Network CIDR address’, and paste it into the ‘IP subnet field’. Also click the ‘+ Add Static IP Address Pool’ button, and copy the pool for the Data Network off the spreadsheet. End result should look similar to this.

Click Save in the lower right. Click SAVE again on the next screen. Now the Data network is configured. Next we need to configure the routing.

On the left hand side, select VRF Context to configure the routing. To the right of ‘global’ select the edit button.

We need add the default gateway route and set to 0.0.0.0/0. In the Next hop, we can add the gateway for the data network from the spreadsheet.

Click Save in the lower right.

Now the Data network has been setup.


Configure the IPAM profile

Next, we need to make sure that the NSX ALB knows which IPs it should use, so we need to setup an IPAM as well.

1 – Click on the Templates tab, and then under Profiles, select IPAM/DNS Profiles.
2 – Click the CREATE button, and select ‘IPAM Profile’ from the drop menu.  With the essentials license, we can only create an IPAM Profile.

1 – Name the profile. In this example we use: tanzu-ip.
2 – Under Cloud, select Default-Cloud in the drop menu.
3 – Under Usable Networks, click the ADD button, and in the lower menu, select the data network.

Click SAVE in the lower right.

Now the IPAM profile is configured.


Assign the IPAM profile to the Default-Cloud

Next we need to assign the IPAM to the default cloud. Click the Infrastructure tab, select Clouds, and then to the right of the default-cloud, click the edit button.

Now we can update the default-cloud IPAM Profile, with the IPAM profile just created.

Click SAVE in the lower right. Next, wait for the status to turn green if it hasn’t already.

Congrats! We have finished the setup for the NSX-ALB Controller, and are now ready to deploy Tanzu. I’ll cover that in my next blog. Stay tuned.

vSphere with Tanzu: Replacing NSX-ALB Controller Certificates

CaptainvOPs

Blog Date: August 12, 2022
NSX-ALB Controller version: 22.1.1
vSphere version 7.0.3 Build 20150588

In this post, I’ll be going over the steps to replace the SSL/TLS controller certificate. You can either replace the controller certificate with one from a Certificate Authority, or you can create a self signed certificate that includes the FQDN and IP in the SAN. This information is missing in the default certificate, and it will cause the deployment of Tanzu to barf if not done. In this blog I will cover both methods.

(OPTION A) Replacing the NSX-ALB Controller SSL/TLS Certificate using a CA.

The following process is what I used in my lab to replace the NSX-ALB controller certificate using my Microsoft CA. Your mileage may vary.

Remember to take a snapshot of the NSX-ALB controller before proceeding.

1- First we will need to generate a CSR. In the NSX-ALB interface, go to: Templates -> Security -> SSL/TLS Certificates.
2 – Click the blue “Create” button in the upper right, and select Controller Certificate from the drop-down menu.

Enter the name of the certificate, and then under Type, select CSR.

Add the certificate information, and then add the FQDN and IP address to the subject Alternate Name (SAN). I am doing this example in my home lab, so I only have a single controller. Large deployments might have clustered controllers, so your configuration will very if you have multiple SANs to add.

Click the green “Save” button in the lower left.

Now you will see the certificate in a grey status in the list. Click the pencil icon on its right side to edit it.

Now we need to copy the Certificate signing request to take over to our CA. Click the “Copy to Clipboard” button.

Now we need to connect to the Microsoft CA web portal. Click “Request a Certificate”.

Click on “advanced certificate request”.
Past the certificate request into the box. *Caution* -> Make sure there is no empty line after
—–END CERTIFICATE REQUEST—–.

For this certificate, you will need to use the “Web Server” template on the Microsoft CA. I have created a VMware version of it following VMware’s KB article 2112009 for Creating a Microsoft Certificate Authority Template for SSL certificate creation in vSphere 6.x/7.x. I called my web server template VMware, so that is what I have selected here.

Click “Submit”.

Select “Base 64 encoded”, and then click the “Download certificate” link.

We also need to obtain a copy of the root ca certificate from the Microsoft CA to complete the chain.

In order to complete the cert chain, in a text editor like Notepad++, paste the NSX-ALB SSL certificate you created first, and then paste the root ca certificate from the Microsoft CA. It should look like this:

Now you have the full chain, so copy it. Go back to the pending controller certificate in the NSX-ALB interface, and paste the full chain into the window.

Click the “Save” button in the lower left corner.

Now back to the main SSL/TLS Certificates window, we should see that the controller SSL certificate has a green status. You might need to refresh the page a couple of times. If the controller SSL certificate is invalid, a little exclamation symbol will appear next to the certificate, and you will need to troubleshoot further.

Assuming the SSL certificate is proper, we now need to apply it to the NSX-ALB controller. Go to Administration tab, In the left menu expand Settings, and then select Access Settings.

Click the pencil in the upper right, to edit the system access settings.

On the Wizard that opens, on the left hand side you’ll see ‘SSL/TLS Certificate’ and it will have the default certificates in it.  Click the ‘X’ on those to remove them.

Select the new NSX-ALB controller certificate you imported. Click the ‘Allow Basic Authentication’ check box as well and add the check mark.

Click the “Save” button in the lower right. After 5 to 10 seconds, the certificate will be updated on the controller. Close and reopen your browser to see the newly installed certificate.

That’s it for replacing the NSX-ALB controller certificate. Not terrible once you understand the process. Don’t forget to clean your room, and delete the snapshot you took prior.


(OPTION B) Replacing the NSX-ALB Controller SSL/TLS Certificate using self signed method.

1 – In the NSX-ALB interface, go to: Templates -> Security -> SSL/TLS Certificates.
2 – Click the blue “Create” button in the upper right, and select Controller Certificate from the drop-down menu.

Type: Self Signed will be selected by default.
Add the certificate information, and then add the FQDN and IP address to the subject Alternate Name (SAN). I am doing this example in my home lab, so I only have a single controller. Large deployments might have clustered controllers, so your configuration will very if you have multiple SANs to add.
Fill in the details below to complete the self signed certificate. Click Save.

We now need to apply it to the NSX-ALB controller. Go to Administration tab, In the left menu expand Settings, and then select Access Settings.

Click the pencil in the upper right, to edit the system access settings.

On the Wizard that opens, on the left hand side you’ll see ‘SSL/TLS Certificate’ and it will have the default certificates in it.  Click the ‘X’ on those to remove them.

Select the new NSX-ALB controller self-signed certificate in the SSL/TLS Certificate drop menu. Click the ‘Allow Basic Authentication’ check box as well and add the check mark.

Click the “Save” button in the lower right. After 5 to 10 seconds, the certificate will be updated on the controller. Refresh the page a couple of times, and you might get the “Potential Security Risk Ahead” warning message in the browser. Else you might need to close and reopen your browser to see the newly installed certificate.

That’s it. Using a self-signed certificate is a straight forward process. Most production deployments will use a CA signed certificate. Don’t forget to clean your room, and delete the snapshot you took prior.

In my next blog: vSphere with Tanzu: Configuring the NSX-ALB Controller, I continue on and configure the NSX-ALB controller to use with Tanzu.

vSphere with Tanzu: Deployment of NSX-ALB Controller

CaptainvOPs

Blog Date: August 11, 2022
NSX-ALB Controller version: 22.1.1
vSphere version 7.0.3 Build 20150588

In my previous blog: vSphere with Tanzu: NSX-ALB Controller Requirements and Deployment Prep. I went over the basic requirements and prep work for the NSX-ALB controller to use with Tanzu. In this blog, I’ll demonstrate deploying the NSX-ALB into my home lab. In this blog, I will be doing the basic NSX-ALB controller in my lab with no NSX-T.

NOTE: Production deployments should use NSX-ALB controller cluster. In my lab however, I am only deploying a single controller for this example.

NSX-ALB Controller Deployment

Obtain the NSX-ALB controller by following the VMware KB Article 82049. In this example I am using version 22.1.1

Import the Controller OVA into the vCenter.

OVA Import Step 3: The controller would need to be deployed on a Compute Cluster that has access to the networks configured for this deployment. Click NEXT.

OVA Import Step 5: Select storage that the desired Compute Cluster has access to.

OVA Import Step 6: Select the management network distributed port group in the excel we filled in earlier, and click NEXT.

OVA Import Step 7: fill out the networking information for the NSX-ALB controller. Here we just need to add the controller IP, subnet, and gateway. Click NEXT.

OVA Import Step 8: Check for accuracy, and then click FINISH to start the deployment.

Once the NSX ALB OVA is deployed, start the VM. Wait for the VM to fully boot, and then access the web interface.

On the first login, you will need to create the admin password. Click CREATE ACCOUNT

After the admin account is created, you’ll need to add the DNS resolver(s) and DNS Search Domain. Click NEXT.

Add the “from” email address of your choosing. For this example, I am just using Local Host. Click NEXT.

For the multi-Tenancy, the defaults can be used unless otherwise specified. Toggle the check mark in the “Setup Cloud After” in the lower right because we want to configure the cloud component later, and click SAVE.

Now we are logged into the admin interface, and are immediately receive a controller faults error message that it doesn’t have license/subscription information, so we need to add it.

Click the Administration tab, and then on the left, expand Settings, and click Licensing.  Click the gear.

Select the Essentials Tier for Tanzu license. Click SAVE.

You can see the interface has changed, and it wants us to add a license key, however we are in the essentials mode, and can only use essentials features.  We do not need to change anything.

That covers the basic deployment for the NSX-ALB controller. In my next blog, I will walk through the process of assigning a signed certificate from my Microsoft CA to the controller. I will also show how to create and assign the self-signed certificate to the controller. Stay tuned.

vSphere with Tanzu: NSX-ALB Controller Requirements and Deployment Prep.

CaptainvOPs

Blog Date: August 11, 2022
NSX-ALB Controller version: 22.1.1
vSphere version 7.0.3 Build 20150588

VMware Customers can find additional details on system design options and preparation tasks in the vSphere 7 with Tanzu Prerequisites and Preparations Guide. This blog is a summary focused on requirements when using vSphere Networking and NSX-ALB Load Balancing Controller, and the deployment of the controller. This example is for the the deployment of the controller without NSX (NSXT).

Hardware Requirements  

vSphere Cluster No. of Hosts CPU Cores Per Host Memory Per Host NICs Per Host Shared Datastore 
Minimum Recommended 3 16 (Intel CPU Only) 128GB 2x 10GbE 2.5 TB 

Note: Increasing the number of hosts eases the per-host resource requirements and expand the resource pool for deploying additional or larger Kubernetes clusters, applications, other integrations, etc. 

Software Requirements 

Note: For the most current information, visit the VMware Product Interoperability Matrices and vSphere 7 Release Notes 

Product Supported Version Required /Optional Download Location 
VMware ESXi hosts 7.x Required VMware Product Downloads 
VMware vCenter** vSphere 7.0U2 and later Enterprise Plus w/ Add-on for Kubernetes (or Higher) Required 
NSX ALB* NSX ALB Required 

Network Requirements – Layer 2/3 Physical Network Configuration 

vSphere with Kubernetes requires the following L2/L3 networks to be available in the physical infrastructure and extended to all vSphere cluster(s)’. This is the official reference page with the networking information and options for configuring vSphere with Tanzu networking: https://docs.vmware.com/en/VMware-vSphere/7.0/vmware-vsphere-with-tanzu/GUID-8908FAD7-9343-491B-9F6B-45FA8893C8CC.html 

To mitigate deployment delays from troubleshooting infrastructure-related problems, customers need to preconfigure both NICs with the appropriate network access, as detailed in the table below, and test for connectivity in advance of any on-site work. 

VLAN Description Host vmnic(s) Virtual Switch MTU IPv4 CIDR Prefix  Routable 
Management Network* NIC 1 & 2 vDS ≥≥1500 ≥≥ /27 Yes 
vMotion Network** NIC 1 & 2 vDS ≥≥1500 ≥≥ /29 No 
Storage / vSAN Network NIC 1 & 2 vDS ≥≥1500 ≥≥ /29 No 
Workload Network*** NIC 1 & 2 vDS ≥≥1500 ≥≥ /24 Yes 
Data Network NIC 1 & 2 vDS ≥≥1500 ≥≥ /24 Yes 

* If the ESXi hosts’ mgmt vmkNIC and other core components such as vCenter operate on separate networks, the two networks must be routable. 

** As opposed to a separate network, vMotion can operate on a shared network with ESXi hosts’ mgmt vmkNIC  

*** The workload network hosts K8s control plane and worker nodes 

When choosing the vSphere Networking model, all network segments and routed connectivity must be provided by the underlying network infrastructure. The Management network can be the same network used for your standard vCenter and ESXi VMKernel port functions, or a separate network with full routed connectivity.  Five consecutive IP addresses on the Management network are required to accommodate the Supervisor VMs, and one additional IP is required for the ASX ALB controller. The Workload CIDRs in the table above account for the typical number of IP addresses required to interface with the physical infrastructure and provide IP addresses to Kubernetes clusters for ingress and egress communications. If the CIDR ranges for Workload and Frontend functions are consolidated onto a single segment, they must be different ranges and non-overlapping. 

Additionally, the Workload Management enablement will default the IP address range for Kubernetes pods and internal services to 10.96.0.0/23. This range is used inside the cluster and will be masked behind the load balancer from the system administrators, developers, and app users. This range can be overridden if needed but should remain a minimum of a  /24. 

Tanzu Mission Control (if available): 

TKG cluster components use TCP exclusively (gRPC over HTTP to be specific) to communicate back to Tanzu Mission Control with no specific MTU outbound requirements (TCP supports packet segmentation and reassembly). 

Firewall Requirements 

VMware HIGHLY RECOMMENDS unfiltered traffic between networks for the system. Reference the VMware vSphere 7 with Kubernetes Prerequisites and Preparations Guide for the summary firewall requirements. 

If Tanzu Mission Control (TMC) is available, the platform needs internet access connectivity. 

Storage 

You will need to use vSphere supported shared storage solution. Typically, this is vSAN, NFS, iSCSI or Fibre Channel. Shared storage is required. Presenting storage volumes directly is not. 

Enterprise Service Requirements 

  • DNS: System components require unique resource records and access to domain name servers for forward and reverse resolution  
  • NTP: System management components require access to a stable, common network time source; time skew < 10 seconds  
  • AD/LDAP (Optional): Service bind account, User/Group DNs, Server(s) FQDN, and port required for authentication with external LDAP identity providers 
  • DRS and HA need to be enabled in the vSphere cluster. 

With the above mentioned requirements in mind, after the environment prep has been completed, I like to fill out the following excel with all of the information needed for the deployment and configuration of the NSX-ALB controller.

That covers the basic requirements and prep work for the NSX-ALB controller deployment. In my next blog: vSphere with Tanzu: Deployment of NSX-ALB Controller, I will walk through the basic deployment of the controller.

vSphere with Tanzu: Install Kubernetes CLI Tools from the Tanzu Control Plane on Ubuntu Dev VM.

CaptainvOPs

Blog Date: August 8, 2022

Ubuntu 22.04
vSphere version 7.0.3 Build 20150588

On engagements with customers, I’ll have them deploy a developer VM where we can work and I can get them started on their Tanzu and Kubernetes journey. This one VM will have docker, docker credential helper, and the Tanzu Kubernetes CLI installed. For the purpose of this blog series, I’ll do the same. For this blog, I’ll walk through the process using Ubuntu 22.04. In my lab, I have already configured my Ubuntu VM with a static IP and host name.

Getting Started with Ubuntu and Installing Docker

Docker is available for download and installation on Ubuntu 22.04. To get started, let’s update the package repo first, and then we can install docker.

$ sudo apt update
$ sudo apt install docker.io

After the Docker installation has completed successfully, let’s start the Docker service, and enable it to run on boot.

$ sudo systemctl start docker
$ sudo systemctl enable docker.service

(Optional) Once that completes, run the following commands to allow docker to run as non-root:

$ sudo groupadd docker
$ sudo usermod -aG docker $USER
$ sudo newgrp docker

Let’s run the following command to see the Docker path. We should get a return /usr/bin/docker

$ which docker

Let’s make sure Docker is running with the following command. 

$ sudo systemctl status docker

We should see an active (running) status.

You can verify the current Docker version installed, and see additional details with the following command. 

$ sudo docker version

At this point, my customer would have completed to per-requisite to have an Ubuntu VM deployed for me. In this blog however, I will continue on assuming vSphere with Tanzu has already been enabled on a workload cluster in the environment.

Downloading The Kubernetes CLI vsphere-plugin.zip

The official VMware guide to Download and Install the Kubernetes CLI Tools for vSphere can be found here. In the vSphere Client, click the 3 hash marks in the upper left and select Workload Management. Select the Supervisor Clusters tab. Make note of the Control Plane Node Address (IP address). It will be needed for the next command to download the vsphere-plugin.zip.

In the example command, I am using the optional –no-check-certificate. Replace the IP address with your Control Plane Node IP Address

$ wget https://192.168.6.13/wcp/plugin/linux-amd64/vsphere-plugin.zip --no-check-certificate

Next, let’s unzip the vsphere-plugin.zip to /usr/bin/vsphere-plugin

$ sudo unzip vsphere-plugin.zip -d /usr/bin/vsphere-plugin

VMware does provide instructions on the control plane node landing page, if you open a web browser and go to https://<insert_control_plane_node_ip_address_here >

We are up to step 2, and we need to put the contents of the unzipped vsphere-plugin into your executable search path. Let’s execute the following commands.

$ echo 'export PATH=$PATH:/usr/bin/vsphere-plugin/bin' >> ~/.bash_profile
$ echo 'source <(kubectl completion bash)' >> ~/.bash_profile

Next, we’ll exit our current SSH session on the ubuntu dev vm.

$ exit

Now re-establish a SSH session to the ubuntu dev vm using the same account as before. Once logged in, let’s test that kubectl completion bash is working. We do this by entering ‘kubectl’ followed by a space and then hit tab twice.

$ kubectl

Test Connection to Tanzu Control plane (optional)

From the Ubuntu dev VM, use the kubectl CLI to connect to the vSphere with Tanzu control plane as the authenticated user. Here I am using the –insecure-skip-tls-verify optional parameter 

$ kubectl vsphere login --server <cluster_ip> --insecure-skip-tls-verify

It will prompt for a username and password. Once successfully logged in and if the account you chose has permissions to access the Tanzu control plane node, you should see a message stating you have access to the following contexts.

Now you are ready to begin working with Tanzu. More content to come. Stay tuned.