Fastvue Reporter has some enhancements when it comes to processing multiple reports, with particular benefits to IT and Network Security teams. It will now schedule multiple report jobs sequentially based on the number of processor cores available to the server.
So what is the best way to plan, allocate and monitor CPU resources in your Hyper-V environment?
From Release Notes (TMG Reporter 2.0.1.5)
- Scheduled reports are now processed sequentially instead of concurrently.
For every three CPU cores available beyond an initial three cores, an extra report can be run concurrently. (e.g. 12 core system can run 4 reports concurrently).- These changes are to prevent a large number of scheduled reports from maxing out system resources.
Understanding Hyper-V CPU Usage
When running virtual servers in Hyper-V, there is often some confusion when determining how much CPU is actually being used. The typical first approach is to log into the Hyper-V host and open up task manager. Here most people are surprised to see that their CPU usage is minimal, this despite the indication that the VM’s CPU’s are 100% utilized.
Truly understanding Hyper-V virtual and physical CPU usage first requires us to dive into a little bit of theory.
Virtual CPU’s can be allocated to a virtual machine. The amount of virtual processors available are determined by the number of cores available on the hardware. So as an example, if you have a 4 socket server where each processor has 8 cores, this will present 32 logical processors. As a result you will be able to allocate a maximum of 32 virtual CPUs to a virtual server. Each virtual CPU maps down to a physical core.
Configuring Your Virtual Machine’s Processors
To understand how this all maps together, let’s have a look at the Hyper-V VM processor configuration. Before we start allocating CPU’s, let consider the available resource controls and how they may effect not just this VM, but also the host and other VMs.
Number of virtual processors
This is the amount of cores that the virtual machine can see and use. The maximum number that can be specified is the maximum number of cores on the hardware
Virtual machine reserve (percentage)
This is a reserve that the host will keep for this virtual machine. This is a percentage of the amount of allocated system CPU so 10% here would be 10% of all 16 cores. In short it is a lot.
Percentage of total system resources
This is a calculated value and it takes the number of allocated CPUs. Total system CPU / Allocated CPU * Reserve. Is this case 16 / 32 * 10 = 5%
Virtual machine Limit (percentage)
This indicates what percentage of every core the VM can consume. This limit or cap is across all the allocated CPUs.
Percentage of total system resources
This is a calculated value based on the amount of allocated CPUs and the maximum limit set. Total System CPU / Allocated CPU * Limit. In this case 16 / 32 * 90 % = 45%
Relative weight
This value is used by the processor scheduler to determine which virtual machine gets preference for CPU calls. A VM with a higher weight would get more CPU time than a similar spec VM with a lower weight. This is a deceiving number since it is not a percentage and the values have to be between 0 and 10 000.
Now that we have allocated the virtual CPU we need to see how they behave and how they impact the host.
Monitoring CPU Usage
The simplest way to monitor CPU usage to look at Hyper-V management console. The CPU usage column is representative of the host’s total CPU resources. Seeing 10% usage here would equate to 10% across 32 CPUs, or if we work it back to the amount of resources allocated to the VM it would be 20 % across the 16 VCPUs
They Hyper-V management console is great for getting a high level overview. But to dive a little deeper and see which cores are being used and by what process, we will have to break out Perfmon.
NOTE: When you start Perfmon, the default counter is %Processor Time. As mentioned earlier, this is deceptive as one would think it is for the host’s hardware, but it is in actual fact the usage for the host’s operating system. This is why there is often very low usage here but high total CPU usage.
Monitoring Physical CPUs
The counter you want to look for is Hyper-V Hypervisor Logical Processor. Specifically % Total Run Time. I have added all the instances of the CPU’s so you can see that there are 32 cores all using different percentages and an overall much lower _Total
Monitoring Virtual CPUs
A very useful counter is Hyper-V Hypervisor Virtual Processor. Specifically % Total Run Time. When you select this counter you will notice that it lists the virtual machines VPU’s. Each virtual processor here maps to a logical processor as seen above.
This allows us to see if there is a process running on the VM that might not be threading efficiently and spiking a single CPU core to 100% while the remainder stays low. In Fastvue Reporter’s case you will not see this bad behavior since it multi-threads rather nicely. The graph below shows a good spread across the CPUs.
Did you know: Fastvue Reporter produces clean, simple, web usage reports using data from your firewall that you can confidently send to department managers and HR team.
You may have noticed your total virtual CPU’s can exceed the number of physical CPU’s. This is where the weighting and reservation comes into play. A virtual CPU does map to a physical core, but you most likely will have multiple virtual CPUs tied to a physical core. This over subscription is one of the main benefits but also potential problem areas for virtualization.
Planning your Virtual Servers
If you look at the host summary from System Center Virtual Machine Manager 2012 SP1 (VMM) you can see that there are more virtual processors allocated than there are physical cores. You would imagine that this over subscription can cause really high CPU congestion. This may be true in acute cases where there are short periods where the CPU’s are all maxed, but this is not the only consideration.
The one major benefit from virtualization is that CPUs can be used when needed and shared when not needed. If you look at the CPU usage for the same server over the duration of an entire month, you will notice that the overall physical CPU usage is actually fairly low.
By using the various counters available in the Hyper-V Hypervisor Logical Processor, and Hyper-V Hypervisor Virtual Processor, categories you can get a very in-depth look into how your host and VM’s utilize the available CPU resources. If you combine this with a knowledge of the various virtual machines and when you expect peak CPU usage times, you can combine or host multiple virtual servers efficiently, enabling good CPU performance when needed.
Take the pain out of reporting on Web Usage and Network Traffic.
Fastvue Reporter produces clean, simple, web usage reports using data from your firewall that you can confidently send to department managers and HR team. Automate reports and get the job of reporting on web usage off your desk and into the hands of people that need it.
There are also a number of benefits to IT and network security teams with the Fastvue Reporter interface helping to reduce IT workload whilst getting a handle on your network and troubleshooting with ease. Download the 30 day free trial today!
Nice write up. Making me rethink some things with our Hyper-V. Thanks!
Nice article!
Quick question: “virtual machine reserve” – when the host has 50% VM reserve – does it mean a CPU starving VM cannot get more than 50% resources?
In general if no settings (reservations) are made for a VM – does it mean the hypervisor can allocate additional CPU cores in case VM peaks CPU util?
Hi James
There are two different concepts to understand here.
When it comes to host reservations it is a reservation based on the entire system CPU resource pool, as in all of the physical sockets and cores combined. If the host has 4 cores, A,B,C and D. A 10% reservation would be across all 4 cores.
Virtual machines are allocated cores, these cores map back to actual physical cores. For simplicity sake, if a virtual machine is allocated Core A and B it can only use those two. A reservation of 10% applies only to 10% of those two physical cores.
If a virtual machine has been allocated a certain amount of cores it can and will only ever be able to access that amount. The host will never allocate additional cores. I mentioned the virtual machine being allocated core A and B, this is simply the concept, they are are dynamically allocated and reassigned based on load. So if the cores A and B are congested the host will remove core A from the VM and give it core C instead.
So the reservation therefore is 10% of the number of cores, this is the minimum amount of the number of assigned physical cores. This would mean 5% of the host’s total CPU is reserved for that single VM.
So to simplify and answer your question (hopefully)
The VM can use 100% of the assigned cores to it,but no more, this would mean that the host is only 50% utilised and it will take it’s 50% reservation from the remaining cores.
Let me know if you still need more info, or if this answer just added confusion :)
Regards
I have a 2008 r2 server running BackupExec, and also hosting three virtual machines for some SQL Databases.
This server has a Quad 2.5 Xeon processor and 16 GB of RAM.
In one of the VM’s I placed this values on the VM Settings, do you think that is ok?
number of virutal processors 4 (the maximum value)
vm reserve 30
vm limit 90
relative weight 100
@Light
The setting you change per VM is really relevant in comparison to the rest of the virtual machines.
If you leave all of the VM with the default settings they all have the same “importance” so they will fight it out for who get to resources IF it becomes contained on the host.
You would only start adjusting the setting if you knowingly want to advantage one VM over the others.
The VM Limit is a good way to keep one VM from hogging all the resource, so in the same manner that you can use the setting to advantage a VM you can limit it.
It all depends on your environment. The thing to remember is that these setting are not fixed forever and always. Make a setting, run it for a while and see what the outcome is, tweak it a bit and start again. Keep going till your environment is tuned to your specific needs.
Regards
Etienne
This was a Microsoft exam question.
Your network contains an Active Directory domain named contoso.com. The domain contains a member server named Server1. Server1 runs Windows Server 2012 R2 and has the Hyper-V server role installed.
Server1 hosts 10 virtual machines. A virtual machine named VM1 runs Windows Server 2012 R2 and hosts a processor-intensive application named App1.
Users report that App1 responds more slowly than expected.
You need to monitor the processor usage on VM1 to identify whether changes must be made to the hardware settings of VM1.
Which performance object should you monitor on Server1?
A. Processor
B. Hyper-V Hypervisor Virtual Processor
C. Hyper-V Hypervisor Logical Processor
D. Hyper-V Hypervisor Root Virtual Processor
E. Process
Really need your help on this. Writing exams next week
Hi Rohan
The question is about whether or not the VM is running out of CPU.
The counter for this is “Hyper-V Hypervisor Virtual Processor” the answer is therefore B
Regards
Maybe i am missing something. I come from a VMWare environment. In VMWare i could give a VM One processor with 4 Cores. So is hyver processors equal to Vmware cores? Just trying to transition between the two.
Sorry that is HyperV not Hyver
Oops NVM. Read a little closer and got my question answered.
Thanks. This is a good Article.
Dear Ettiene,
Congrats for your post! It’s great.
Maybe you can help me with a issue, Is it anyway to assign a physical processor to a virtual machine?
I think this is an option in VMWare.
Kind Regards
Hi Alberto
There is no way to physically peg a virtual processor to particular CPU or core. There is a direct one to one relationship, but this is a dynamic allocation. Even if there was away of doing this I would not recommend it. One of the main advantage of using type 2 hyper visors like ESX and Hyper-V is that they can dynamically optimise the virtual machines based on the on the cumulative virtual machine load and the net hardware availability.
Regards
Etienne
A stupid question probably.
I have a virtual terminal server running.
If you look at cpu performance on the hyperv-core, there is 2-3% cpu usage. If i use Performance Monitor with counters on the Hyper-V Hypervisor Logical Processor(_Total)% Total Run Time system is loaded with average 70-80% and peeks at 100%. Users are having issues with delayed typing.
Is that systems cpu is fully loaded or what to do so the virtual machine uses full capacity.
We have 8 virtual cpu’s. Virtual Macine reserve (percentage) = 0. Virtual Machine limit = 100. Relative weight 10000
Thanks in advance
Hi Pascal
It sounds like you are maxing out the physical CPU(s)
Looking at the “Hyper-v core” only shows you the CPU used by the hypervisor itself, it should always be very low.
The counters you are looking at that show high usage relate the the physical CPU.
If you have more cores available allocate those to the VM.
There are a number of terminal server specific configuration options you could look into to try help things along.
Regards
Etienne
Number of logical processors assigned to the virtual machine = 8
I’m a little unclear on one point. The writer states:
“Percentage of total system resources
This is a calculated value and it takes the number of allocated CPUs. Total system CPU / Allocated CPU * Reserve. Is this case 16 / 32 * 10 = 5%”
It seems to me it should be Allocated CPU / Total system CPU * Reserve. A similar issue is found on this section:
“Percentage of total system resources
This is a calculated value based on the amount of allocated CPUs and the maximum limit set. Total System CPU / Allocated CPU * Limit. In this case 16 / 32 * 90 % = 45%”
Trying to learn this, but wouldn’t the 16 cores you allocate to the VM be called Allocated CPU in these examples? Or am I missing something?
Josh you are correct!
The wording is indeed the wrong way around! Thanks for spotting and pointing that out.
The correct wording is
Allocated CPU/ Total system CPU * Reserve. In this case 16 / 32 * 10 = 5%
and
Allocated CPU / Total system CPU * Limit. In this case 16 / 32 * 90 % = 45%
The article has now been updated so the wording is correct now and matches the math.
Thanks Josh
Hi Experts,
I am facing a special problem Here it is:-
I have two physical server with 24 core, 132 GB Mem and 1.2 TB HDD.
It is Citrix XenServer wherein around 50-70 users keep working on various application.
Many time user starts complaining slowness in accessing the applications when we see the CPU utilization on SCOM, it shows around 20%. For testing purpose we deployed WhatsUpGold monitoring tool which monitor CPU utilization on Core basis. We saw there is one core is being utilized around 99% while the others are under 15-20% utilized.
-Why?
-Secondly, does it impact the performance of my server if only one CPU is highly utilized and other remains under control?
Looking for your expert advice. Please help to understand this behavior and how to resolve it?
Regards
TRS
I want a clarification
I hired a VM with 16 processors and I saw only two processors have utlised by 90 % and another 80% and remaining 14 processors have utilised only 10-20 %.
Now we have increased the 32 processor after that I also seen my only two processors utlised 70 and another 60% and all other processors utlised only 20-30 %
Hi Help here,
i have one physical core running, one virtual machine. Since this setup evrything has terribly slowed down. I have one Virtual processor. what do you propose i set for the processor parameters. Currently virtual machine reserve is at 0,virtual macjine limit is at 100, relative weight at 100 as well.
Hi,
i have a server, which have 2 sockets, and 16 cores, but it shows logical processors are 16, it should be 32.
i’m unable to give more than 16 vCPU to each VM.
i have 2 VMs rights now.
Most of the modern servers have CPU multithreading, which this article didn’t cover. Since a core thread is being presented to the host OS as a logical CPU, in Hyper-V a virtual processor equals to a CPU thread and not actual core. So if I have a 4 core / 8 thread CPU on my host server, I can allocated up to 8 virtual processors which equals to 100% of CPU resources.