I wanted to write a short paragraph on adding CPU's to VM guests. This is hardly a technical paper as you can see by the length, but I wanted to note down an explanation of how adding CPUs may actually slow things down.
When you configure a VMware guest to use a number of cores, then the hypervisor waits for a time-slot where that many cores become available. Imagine the following VMware host:
(Host has total of 8 Cores)
Guest #1 - 2 CPUs
Guest #2 - 4 CPU's
Guest #3 - 2 CPUs
vCPU/pCore = 8 / 8 = 1
i.e. vCPU/pCore <= 1
In this configuration there will always be cores available to the guests so you are in a happy place.
Now suppose Guest #1 was showing performance problems, you add 2 CPU's to the guest believing that this would help. However...
vCPU/pCore = 10/8 = 1.25
i.e. vCPU/pCore >= 1
This will actually slow Guest #1 down. It has to wait for a time slot to be free on 4 of the 8 cores, but since 6 of the cores are allocated to Guest #2 and Guest #3 it will have to wait for a period where both Guest #2 and Guest #3 have idle time.
Now if Guest #1 is truly CPU bound then you will have to move something off this guest or reduce the number of cores allocated to Guest#2 and Guest #3, but if Guest #1 is not CPU bound then adding the additional CPUs will actually slow things down. An analogy from my friend Andy - If you book 2 conference rooms for one meeting you are going to have a hard time scheduling your meeting.
This stuff is a science in itself and I am not doing the subject justice, but this is still a useful tip.