Wednesday, July 20, 2011

Hyper-V appears to run out of RAM when there is plenty

Here is one issue that I have been tipping folks off to in the forum for some time.

The scenario is:  I have an environment, it is running great, totally stable.  At some point try to do something and I am told there are not enough resources.

If I start to look at memory counters it appears that the management OS of the Hyper-V Server is running out of RAM.

The other thing – if you do all the math, there is adequate RAM in the system for the management OS and all of the VMs.

What I have described is the behavior that is seen.  And the messages that folks see on the screen makes them believe that the server does not have enough RAM to do what it needs to do, such as starting a VM.  But it really does.

The other symptom that might be seen is that the system appears to be sluggish.

The resolution that I tell folks is to be sure to logout of the Hyper-V Server when they are done administering.

This is where an unmentioned common thread appears:  In almost all of these cases the symptom is on a Server 2008x with Hyper-V Full installation. 

And the most important part as that the server is administered using remote sessions connected to the server and the administrators do not log out of their sessions, they simply disconnect.

What is happening is that the user shell is slowly consuming more and more resources.  This especially gets high if you open the VM consoles using the console application or if you leave the consoles open and disconnect.

What I have noticed is that the simple practice of logging out of your remote session cleans up all this extra used RAM – and this simply enforces that this is a user level behavior.

My recommendation to you – always log out.  If you have Administrators that don’t comply well use an old fashioned Group Policy to automatically logout disconnected sessions.  Since these are VMs and VM console sessions nothing will be lost.

The other little trick – if this is a situation where the system won’t let you power on a VM – simply try to power it on three times.  That magical third attempt forces RAM recovery and the system will reclaim resources.

Also, this further links into the behavior that the RAM of the management OS is dynamic (always has been) and that it is also limited in that it cannot consume all of the RAM of the hardware.

No comments: