You just need to generate the java heap dump from one of the cells. What you’ll need to succeed:
- IP tables disabled on the cell you are connecting to.
- Disk space available on the cell to accommodate the dump – I believe these can be between 8 and 10 GB in size
- Unless an emergency, do this operation outside of normal business hours as it will be CPU intensive for up to 3 minutes, can impact API call performance, and can potentially cause the VCD cell inventory service to hang.
Step #1: Disable iptables on the cell
ssh to the desired cell and run the following command:
# service iptables stop
Step #2: Connect with jconsole (java console)
- domain credentials should work here depending on your environment
- connect to port: 8999
- connect to desired cell
- If you get this message “Secure connection failed. Retry Insecurely?” just click the ‘insecure’ button to continue
Step #3: Generate the heap dump
- On the MBeans tab, in the com.sun.management/HotSpotDiagnostics object, select the Operation section.
- In dumpHeap parameters, enter the following information:
p1: true – do a garbage collection before dump heap
- Click the dumpHeap button.
- There will be no indication that the heapdump completes. I just watch the size of the file until the growth stops on the cell. This process typically takes less than two minutes.
Step #4: Cleanup and send-off
- Locate the heap dump in /opt/vmware/vcloud-director/ and move off to a location where you can compress and upload to VMware FTP site as you would for logs.
- Start the iptables on the cell: # service iptables start