We're running into a strange issue in a deployed version of our product where the performance is just not very good. When we restore a backup of their DB on one of our machines, it performs as we would expect.
While there are plenty of obvious things to look at, I'm focused on one area for the moment: the number of CPU cycles is 5-7 times higher in profile traces on their machine than it is on ours - running the same queries against the same database.
How does the CPU column in SQL perfmon get calculated on a vm? Do cycles get clocked while the virtual CPU is waiting for a real one? Could the switching of the real CPU under the virtual one lead to cache loss and higher CPU numbers? Am I wasting my time here?
Details: - Both servers are on ESX hosts with SAN storage. - I believe their infrastructure may be overloaded. - VM has 4 CPUs, 16 (or more) GB ram minimal real load on either from within the VM - SAN host is running loads far less than max - ESX hosts are said to be not overloaded (although their infrastructure is expanding soon) - I will be digging into this more tomorrow.
I know there are a lot of other variables involved, but my curiosity is killing me about the huge difference in CPU numbers for the same queries on the same DB - even if it ends up not being the problem, I'd love to know more about what causes these differences!