Upgrading To vSphere 6.7 Update 1, and Using The vCenter Converge Tool: Part 1

I recently wrapped up a vSphere 6.7 U1 upgrade project, while on a VMware Professional Services (PSO) engagement, with a customer in Denver Colorado. On this project, I had to upgrade their three VMware environments from 6.5, to 6.7 Update 1. This customer also had three external Platform Services Controllers (PSC), a configuration that is now depreciated in VMware architecture.

Check the VMware Interoperability, and Compatibility Matrices

The first thing I needed to do, was to take inventory of the customer’s environment. I needed to know how many vCenters, if they had external Platform Services Controllers, how many hosts, vSphere Distributed Switch (VDS), and what the versions were.

  • From my investigation, this customer had three vCenters, and three external platform services controllers (PSC), all apart of the same SSO domain.
  • I also made note of which vCenter was paired with what external PSO. This information is critical not only for the vSphere 6.7 U1 upgrade, but also the convergence process that I will be doing in part two of this blog series.
  • Looking at the customer’s ESXi hosts, the majority were running the same ESXi 6.5 build, but I did find a few Nutanix clusters, and six ESXi hosts still on version 5.5.
  • The customer had multiple vSphere Distributed Switch (VDS) that needed to be upgraded to 6.5 before the 6.7 upgrade.

The second thing that I needed to do was to look at the model of each ESXi host and determine if it is supported for the vSphere 6.7 U1 upgrade. I also need to validate the firmware and BIOS each host is using, to see if I need to have the customer upgrade the firmware and BIOS of the hosts. We’ll plug this information into the VMware Compatibility Guide .

  • From my investigation, the three ESXi hosts running ESXi 5.5 were not compatible with 6.7U1, however they were compatible with the current build of ESXi 6.5 the customer was running on their other hosts. I would need to upgrade these hosts to ESXi 6.5 before starting the vSphere 6.7 U1 upgrade.
  • This customer had a mix of Dell and Cisco UCS hosts, and almost all needed to have their firmware and BIOS upgraded to be compatible with ESXi 6.7 U1.

The third thing I needed to check was to see what other platforms, owned by VMware, and/or bolt on third parties, that I needed to worry about for this upgrade.

  • The customer is using a later version of VMware’s Horizon solution, and luckily for me, it is compatible with vSphere 6.7 U1, so no worries there.
  • The customer has Zerto 6.0 deployed, and unfortunately it needed to be upgraded to a compatible version.
  • The customer has Actifio backup solution, but that is also running a compatible version, so again no need to update it.

Lets Get those ESXi 5.5 hosts Upgraded to 6.5

I needed to schedule an outage with the customer, as they had three offsite locations, with two ESXi 5.5 hosts each. These hosts were using local storage to house and run the VMs, so even though they were in a host cluster, HA was not an option, and the VMs would need to be powered off.

Once I had the outage secured, I was able to move forward with upgrading these six hosts to ESXi 6.5.

Time to Upgrade the vSphere Distributed Switch (VDS)

For this portion of the upgrade, I only needed to upgrade the customers VDS’s to 6.5. This portion of the upgrade was fast, and I was able to do it mid day without the customer experiencing an outage. We did submit a formal maintenance request for visibility, and CYA. Total upgrade time to do all of their VDS’s was less than 15 minutes. Each switch took roughly a minute.

Upgrade the External Platform Services Controllers Before the vCenter Appliances

Now that I had all hosts to a compatible ESXi 6.5 version, I can move forward with the upgrade. I was able to do this upgrade during the day, as the customer would only lose the ability to manage their VMs using the vCenters. I made backups of the PSC and vCSA databases, and created snapshots of the VMs just in case.

I first needed to upgrade the external PSCs (3) to 6.7 U1, so I simply attached the vCSA.iso to my jump VM, and launched the .exe. I did this process one PSC at a time until they were all upgraded to 6.7 U1.

Upgrade the vCenter Appliances to 6.7 Update 1

Now that the external platform services controllers are on 6.7 U1, it is time to upgrade the vCenters. The process is the same with the exe, so I just did one vCenter at a time. Both the external PSC’s and the vCSA’s upgraded without issue, and within a couple of hours both the external PSC’s and vCSA’s had finished the vSphere 6.7 Update 1.

Upgrade Compatible ESXi Hosts to 6.7 Update 1

I really wanted to use the now embedded VMware Update Manager (VUM), but I either faced users who re-attached ISO’s to their Horizon VMs, or had administrators who were upgrading/installing VMware Tools. In one cluster I even happened to find a host that had improper networking configured compared to its peers in the cluster. Once I got all of that out of the way, I was able to schedule VUM to work its way down through each cluster, and upgrade the ESXi hosts to 6.7 Update 1. There were still some fringe cases where VUM wouldn’t work as intended, and I needed to do one host at a time.

Conclusion for the Upgrade

In the end, upgrading the customer’s three environments, vCSA, PSC and ESXi to 6.7 Update 1 took me about a couple of weeks to do alone. Not too shabby considering I finished ahead of schedule, even with all of the issues I faced. After the upgrade, the customer started having their Cisco UCS blades purple screen at random. After opening a case with GSS, that week Cisco came out with an emergency patch for the fnic driver, on the customer’s UCS blades, for the very issue they were facing. The customer was able to quickly patch the blades. Talk about good timing.

Part 2 Incoming

Part 2 of this series will focus on using the vCenter Converge Tool. Stay tuned.

Blog Date: 4/15/2019

2019 VMware vExpert Announcement

It’s that time of year again. I’m honored and humbled to continue to be apart of the VMware vExpert program. This program challenges me every day to continue to learn, and contribute to the #vCommunity. For me, this isn’t just some title. This is a family of community warriors where we learn from and help each other grow. Everyone in some way gives back to the community. This year, I am also excited to try my hand at public speaking, and give back to the VMUG community as a community session speaker. I don’t think that I would have had the courage to apply to be a speaker, if it wasn’t for my fellow vExperts encouraging me to do so.

Congrats to all the new and returning vExperts! https://blogs.vmware.com/vexpert/2019/03/07/vexpert-2019-award-announcement/

vRealize Operations Manager Dashboard: vSphere DRS Cluster Health. Part 1

A few weeks ago, I had a customer ask me about creating a custom vROPs dashboard for them, so that they could monitor the health of the clusters. For those of you who were unaware, VMware has packaged vROPs with a widget called “DRS Cluster Settings”, that does something similar, and look like this:

The idea behind this widget, is that it will list all clusters attached to the vCenter, giving you high level information such as the DRS setting, and the memory and CPU workload of the cluster. With a cluster selected, in the lower window you will see all of the ESXi hosts apart of that cluster, with their CPU and memory workloads as well. If you are interested in this widget, it can be added when creating a new custom dashboard, and you will find it at the bottom of the available widget list.

While this widget gave me some high level detail, it wasn’t exactly what I wanted, so I decided to create my own to give a deeper level of detail. I used the widget above as a template, and went from there.

This dashboard gives me the current memory and CPU workloads for each cluster in the upper left box, and once a cluster selected, it populates the right, and two middle boxes with data. The top right boxes gives me the memory and CPU workload for the past 24hrs, and the two middle boxes gives me the CPU demand and memory demand forecasts for the next 30 days.

Much like the widget mentioned above, by selecting a cluster in the upper left side, in the lower left side there is a box that will populate with all hosts attached to that cluster. Once a host is selected, in the lower right box, we also get a memory and CPU workload for the past 24hrs for the selected host. This dashboard is slightly larger than a page will allow, so unfortunately users would need to scroll down to see all of the data, but I believe it gives an outstanding birds-eye view of the clusters DRS capabilities.

In my next blog post, I’ll break down what’s involved in creating this dashboard.

The Home Lab Part 2

The very long over due followup post to my The Home Lab entry made earlier this year.  I did recently purchase another 64GB (2x 32GB) Diamond Black DDR4 memory to bring my server up to 128GB.  I had some old 1TB spinning disks I installed in the box for some extra storage as well, although I will phase them out with more SSDs in the future.  So as a recap, this is my setup now:

IMG_20171117_170133

Motherboard

motherboardSUPERMICRO MBD-X10SDV-TLN4F-O Mini ITX Server Motherboard Xeon processor D-1541 FCBGA 1667 

Newegg

 

Memory

memory

(x2) Black Diamond Memory 64GB (2 x 32GB) 288-Pin DDR4 SDRAM ECC Registered DDR4 2133 (PC4 17000) Server Memory Model BD32GX22133MQR26

                                   Newegg

M.2 SSD

m.2ssd

WD Blue M.2 250GB Internal SSD Solid State Drive – SATA 6Gb/s – WDS250G1B0B

Newegg

SSD

ssd

(x 2) SAMSUNG 850 PRO 2.5″ 512GB SATA III 3D NAND Internal Solid State Drive (SSD) MZ-7KE512BW

Newegg

 

Case

chassis

SUPERMICRO CSE-721TQ-250B Black Mini-Tower Server Case 250W Flex ATX Multi-output Bronze Power Supply

Newegg

 

Additional Storage

x2 1TB Western Digital Black spinning disks

 

Initially when I built the lab, I decided to use VMware workstation, but I recently just rebuilt it, installing ESXi 6.7 as the base.  Largely for better performance and reliability.  For the time being this will be a single host environment, but keeping with the versioning, vCSA and vROps are 6.7 as well.  Can an HTML 5 interface be sexy?  This has come a long way from the flash client days.

vcenter view

I decided against fully configuring this host as a single vSAN node, just so that I can have the extra disk.  However, when I do decide to purchase more hardware and build a second or third box, this setup will allow me to grow my environment, and reconfigure it for vSAN use.  Although I am tempted to ingest the SSDs into my NAS, carve out datastores from it and not use vSAN, at least for the base storage.

storageview

Networking is flat for now, so there’s nothing really to show here.  As I expand and add a second host, I will be looking at some networking hardware, and have my lab in it’s own isolated space.

Now that I am in the professional services space, working with VMware customers, I needed a lab that was more production. I’m still building out the lab so I’ll have more content to come.

VMware Education Services has updated the naming conventions of VMware’s professional certifications

FYI – VMware is making some major changes to their certification naming conventions. Changes take affect August 2018 for newly released certifications listed below, and are not retroactive.  This will not affect the naming of existing certifications however.

  • VMware Certified Professional – Desktop and Mobility 2018 (VCP-DTM 2018)
  • VMware Certified Advanced Professional – Data Center Virtualization Deployment (VCAP-DCV 2018 Deployment)
  • VMware Certified Advanced Professional – Cloud Management and Automation Deployment (VCAP-CMA 2018 Deployment)

Read more on their official blog post here:

https://blogs.vmware.com/education/2018/08/22/we-are-changing-the-way-we-name-vmware-certifications-the-year-makes-it-clear/

VMworld 2018 is right around the corner! Where will you be?

It’s almost that time a year again….some might even call it that special time of year where VMware geeks from across the globe converge on VMworld.  One might even consider this summer camp, and like any who have experienced this before, you meet new people in the vCommunity, make friends, and part ways after the week of technical sessions, social gatherings, and just the straight up shop talking, war story sharing, and the sharing of ideas.  Personally, this will be my third year attending, and I am super excited to be going.  This conference means enough to me that, due to other circumstances that happened early this year, I purchased my own pass so to ensure that I wouldn’t miss out.

Now is the perfect time to cash in on those early bird discounts on conference passes, good until June 15.  Why wouldn’t you want to save a couple hundred dollars on one of the best IT conferences of the year?  For an individual, it’s $1,795 vs $2,095.  That’s before other discounts that may be applied like vmug memberships, or the discount for VMware Certified Professionals who hold an active VCP.

So, why go to VMworld?

I think for many first timers, there’s a certain electricity, and excitement about going.  Let me be the first to tell you, that feeling…. never really goes away.  Like the past couple of years, VMworld in the US will be held once again in Las Vegas.

Image result for VMworld 2018

I personally love coming to VMworld and have looked forward to it every year.  There’s always good energy here; the minute you get off the plane, it is happy.  Every experience I’ve had here is fun, and people genuinely are in a good mood.  This conference gives attendees the chance to attend VMware lead, and partner lead sessions on platforms you may have thought about using or are currently using.  These sessions are meant to share best practices with the community, transfer knowledge in ways to use VMware platforms, and also give you a chance to ask the experts, many of whom work for VMware, and in some cases, are very involved with the development of the platforms you use.

VMworld is not just about attending sessions however.  This conference gives you the unique opportunity to network with other IT professionals from across the globe and establish relationships that you would otherwise never be able to do.  Like it did for me, this conference may also inspire you to join the vCommunity, a thriving community of professionals who not only share their knowledge with others, but who also need help themselves.  I think we can all agree that no two environments/businesses are alike, and we have all used VMware’s platforms in ways that were intended, and in ways that even VMware might not have ever considered.  Members of the vCommunity take it upon themselves to share their experiences with others, through blogs, social media, and support forums to help others.  This conference gives us a chance to get together, share war stories from our time in the trenches, and many times, you will find attendees getting together to engineer and develop something cool.

VMware {code} group has even put together a hackathon, where members from the vCommunity can get together while at VMworld, to develop some amazing things, and sometimes there are even prizes to be had for the coolest of the cool ideas.  But don’t let those words “code” or “hackathon” scare you.  These sessions are not just for developers!  Sure it will certainly help, but the power of the community, enables you to participate in these teams anyways.  You may not be able to contribute code, but you can still contribute ideas to the team, and you might even pick up a few coding skills in the fun.  Let’s face it; some pretty cool ideas are cooked up during hackathons.  VMware’s internal hackathon cooked up the idea to bring VR into the datacenter, and allow you to virtually move your workloads from On-Premises Data Centers, into the cloud.  It’s freakin VR man!  How cool is that?

Screenshot2

The VMworld conference also affords you the opportunity to attend instructor lead labs, along with VMware’s hands on labs that you can also experience from home.  While at the conference, there will be many vendors out on the floor where you can experience new products, ask questions about products that you already use, and lets not forget the vendor haul crawl where there will be free adult beverages, snacks, and cool swag vendors are giving out.  All can be found in the solutions exchange area.

Image result for VMworld 2017

I’m not going to lie, the parties at VMworld are pretty wild too.  Not saying that should be the only reason you go, but it is a good way to mingle with other conference attendees, jam out to some good music, and of course escape the Las Vegas heat.  VMworld of course wraps up with it’s own party, before the last day of the conference.

Screen Shot 2018-06-02 at 12.16.46 PM

So what are you waiting for?  I can’t think of any reason not to attend the US 2018 VMworld in Las Vegas, August 26th – 30th, or the UK 2018 VMworld in Barcelona, November 5th – 8th.  Follow this link here, and I will see you at the conference in Las Vegas!  Remember to take advantage of those early bird rates, good until June 15th!  REGISTER HERE FOR VMWORLD 2018

Screen Shot 2018-06-03 at 9.29.50 AM

 

Add Custom Recommendation to vROps alert definition for versions prior to 6.6

  • This is useful when a new SOP document is created, we will be able to link to it directly on the alert email that is sent.

Step-By-Step

  1. Log into the main vRealize Operations Manager page.
  2. Click Content and then Recommendations

  3. On this page you can create, edit and delete custom recommendations.  Click the green plus sign to create a new custom recommendation.
  4. Here you can enter the test for the custom recommendation.  Paste the link to the SOP, highlight it, and then click the hyperlink icon.  Now paste the link again and click OK.  The “actions” section will allow the use of automated functions if you were looking at the triggered alert in vROps.  For now, just click save.
  5. Now you can add the custom recommendation to an alert definition.  Click Content and then Alert Definitions.
  6. Search for the alert that you would like to add the SOP to, select it and click the edit button.
  7. Click on section 5: Add Recommendations and then click on the plus sign
  8. Now you will need to search for the new SOP recommendation you just created, so search for SOP, find it in the list on the left, click and drag to position under the Recommendations section.
  9. Finally click save.  Now when this alert is triggered, and an email is sent, there will be a clickable link in the email to the SOP document.

The Home Lab Hardware

IMG_20171117_170133

Setup

I decided to go with a Supermicro build as I wanted something power efficient, yet expandable, and this motherboard supports up to 128GB of ECC RDIMM DDR4 2133MHz server grade memory.  Now with this setup, when I feel the need to expand out my lab, I can build two more nodes, and I’ll have a rather nice VSAN cluster.  However I’m hoping the cost of DDR4 memory will have come down by then…

I did look at the Supermicro SYS-E300-8D and SYS-E200-8D style micro servers, but like most, I was concerned about the fan noise, and thus decided to go with a slightly larger chassis to get the larger fan.  Honestly the fan in the unit I bought makes no more noise then a regular desktop computer.

Here’s my hardware:

Motherboard

motherboard

SUPERMICRO MBD-X10SDV-TLN4F-O Mini ITX Server Motherboard Xeon processor D-1541 FCBGA 1667 

Newegg

Memory

memory

Black Diamond Memory 64GB (2 x 32GB) 288-Pin DDR4 SDRAM ECC Registered DDR4 2133 (PC4 17000) Server Memory Model BD32GX22133MQR26

                                   Newegg

M.2 SSD

m.2ssd

WD Blue M.2 250GB Internal SSD Solid State Drive – SATA 6Gb/s – WDS250G1B0B

Newegg

SSD

ssd

(x 2) SAMSUNG 850 PRO 2.5″ 512GB SATA III 3D NAND Internal Solid State Drive (SSD) MZ-7KE512BW

Newegg

Case

chassis

SUPERMICRO CSE-721TQ-250B Black Mini-Tower Server Case 250W Flex ATX Multi-output Bronze Power Supply

Newegg

Who doesn’t love some internal shots after the lab-box has been put together?  🙂

In the coming blog posts, I’ll be building out my lab.  Stay tuned….