Wednesday, July 11, 2012

Server 2012 Windows Network Virtualization part 1 of many

I have spent quite an extensive amount of time working with the Network Virtualization that is in Server 2012.
Most folks that are interested in this have caught the article on Jeff Snover's blog. And have possibly seen a few TechEd presentations that keep working the same graphics.  And, if you explore a bit you can find some interesting articles from folks in the networking community like over at about how NVGRE is a ‘right thing’.  And a primer on NVGRE over at
Oh, and lets not forget the Network Virtualization Survival Guide by the MSFT folks.
That all sets the basis for the technology.  Now the interesting bits.  The implementation.
First of all, lets get a big question out of the way:  Is SCVMM required to manage Windows Network Virtualization?  No, it isn’t.  However, you want some management layer to manage the complexities of keeping all of the routes and what not up to date.   If you look at Jeff Snover’s graphics, there is this whole set of PA and CA addresses that need to be kept straight.  And then there is the LookupRecords.
Really, this is not too bad in a static environment.  If the VMS never move or change, you set all these rules once and never touch them again.  However, if you move a VM (and Server 2012 makes that super easy) then you need to update the proper records or that VM is off the network.  SCVMM 2012 has a similar challenge of keeping LookupRecords and PAs updated if a VM is moved by Failover Clustering due to an HA event that it did not do.
In other words, this is not for the faint of heart.  There is actually a lot to this.  If you move beyond an environment where your VMs are static then you need to maintain these rules.  This is what management layers (like SCVMM, CloudStack) do for you, manage the complex stuff.
Okay, thanks for the warning, now what?
Step one:
Enable Windows Network Virtualization (WNV).  This is a binding that is applied to the NIC that you External Virtual Switch is bound to.  This can be a physical NIC, it can be an LBFO NIC team.  Either way, it is the network adapter that your External Virtual Switch uses to exit the server.
This also means that if you have multiple virtual networks or multiple interfaces that you can pick and choose and it is not some global setting.
If you have one External Virtual Switch this is fairly easy:
$vSwitch = Get-VMSwitch -SwitchType External
# Check if Network Virtualization is bound
# This could be done by checking for the binding and seeing if it is enabled
ForEach-Object -InputObject $vSwitch {
if ((Get-NetAdapterBinding -ComponentID "ms_netwnv" -InterfaceDescription $_.NetAdapterInterfaceDescription).Enabled -eq $false){
  # Lets enable it
  Enable-NetAdapterBinding -InterfaceDescription $_.NetAdapterInterfaceDescription -ComponentID "ms_netwnv"
I have a ForEach in the example above.  In case $vSwitch is an array, I assume I want to enable it on all virtual switches.  We will deal with the other details later.
Before me move on, I need to know some constants.  Such as my Tenant (Customer).  And some topology.  We will get there as we work through defining everything.
More to come.  Lots to learn about PA, CA, lookup routes, virtual subnet ids, and customer routes.  Lots of concepts, options, and rules to keep straight.


BrianEh said...

The link to the current (right now - July 17) Internet Draft is:

It is interesting to watch this evolve.

Dave said...

Great Series! I learned a lot! Thank you for detailing each part of NVGRE!

One question:

Is it possible to implement the solution outlined below:

There are 3 VMs. 2 out of 3 participating in NVGRE. 1 VM is configured as a router/firewall/etc. Virtual Switch is configured with PA address for these two VMs.

What I want is to allow 2 VMs to talk to Router VM and then Router VM is able to forward the traffic to outside world via the same switch. Is it possible?

Or what should we do configure such setup?

Thanks a lot!

BrianEh said...

The VM that is the router / firewall must be on both the NVGRE network and the LAN.
From a networking standpoint it is rather straightforward.
Beyond that, the configuration is all in the router VM.

For example, inside the NVGRE network, the router VM NVGRE vNIC would be the gateway IP - this is fundamentally no different than how you set up your router in your home.

The MSFT model at the moment is to use SCVMM to manage the NVGRE and Gateway VM.

One thing to be aware of. If your Hyper-V Server reboots, all NVGRE configuration is lost and must be re-applied. Not difficult, and it can be scripted to be re-established, but it does need to be re-established.

Dave said...

Thank you very much for your reply! Yes, you're right about that - NVGRE configuration is lost! Thanks for reminding me!

Actually, we actually want to avoid using R2 gateway which requires a dedicated host. We thought of using VVayyta router for connecting to outside world using OpenVPN.

Thank you once again!

BrianEh said...

Here is where I differ with MSFT - does it "require' a dedicated host? No. Not unless you are going the dedicated cluster / HA route.
Even then, it may be overkill for your personal SLA.
It is the ultimate design, and IMHO, targeted at the very large enterprise / hoster style user.

Dave said...

Your reply to my last statement takes me to another question! Sorry for asking question again!

Do you really think R2-GW can be hosted as a VM on the same NVGRE network? If yes, I would really like to know how you would that.

Thank you once again!

BrianEh said...

The VM is the gateway. You put one NIC in the NVGRE network and the other NIC on the LAN.
The VMM provided gateway is Server 2012 R2 with the proper roles installed and then the proper forwarding rules.
No different than you would do with a routing VM.

The gateway is not the Hyper-V server itself in any way. I tis always a VM.

Dave said... is same as Router VM.. Good to know that! So in case of a DR Cloud (One Primary and Replica Site), hosted at provider, how I client should connect? For example, Primary Site goes down and VM comes online at DR Site. Do I need to implement gateway at both sites? Do clients need to reconnect to Replica Site to access their VMs?

You have been very helpful to me! I truly appreciate your help!

BrianEh said...

Yes. It is a different physical site. So you still have to connect. If your primary site goes down that implies that the gateway at the primary site goes down.

When I lived in the Midwest our DR scenario for a site was a tornado. As in, gone. How do you move on? How do you provide services? How do you keep the company functioning?

Anonymous said...

Your assumptions are incorrect! You cannot have a gateway VM running other than 2012 R2 GW to allow virtual machines to access outside network!

We have tested it and it doesn't work!


BrianEh said...

I wrote this long before MSFT developed its VM titled "Gateway".

And I also ran VMs acting as a gateway / router long before this that were no more than Vyatta routers, or pfsense routers.

So, if you want to get caught up in language and names and titles - then yes the MSFT "Gateway VM" is the way MSFT wants you to do it - as it is driven by SCVMM.

But, you can do it other ways. And it is actually supported in other ways. There is no reason (the design allows it) that your 'gateway' cannot be a physical router that knows how to talk NVGRE. This is a designed for scenario, you don't see it, and it is not talked about in the literature, but it exists.

So, I ask, how did you do your test? How did you configure your forwarding rules? The MSFT infrastructure will only automatically configure your forwarding rules if you use their Gateway VM Service Template, they won't auto configure any other type of gateway - you would have to.

Dbcomp said...

Thank you much for the post! You have really saved my lots of time. I’ve just bookmarked this blog site