Wednesday, August 1, 2012

My Windows Network Virtualization demo script

After all this work with WNV I wrapped everything up with a lunch time training session covering the entire topic.

Quite honestly, it is a big concept to get across in 1 hour, wrapping the brain around the management IP, the PA addresses, the CA addresses and getting that all straight is the most confusing part – the first time someone has this tossed at them.

Once folks get those three items all sorted out it all seems to fall into place naturally.

The other idea is that fact that you are doing demonstrations and presentations as if you were a management layer.  Not using a management layer, but you ARE the management layer.  You have to do everything that you would expect a SCVMM / CloudStack / vCenter to normally be doing on your behalf.

Here are the assumptions for my demo:

Two 2012 Hyper-V Servers, four VMs.  The VMs are Red11, Red22, Blue11, and Blue22.  Red and Blue represent different Customers / Tenants.  They are bringing their systems to my infrastructure and I will be hosting them on my two Hyper-V Servers.


The VMs should have their IPs manually set in the following way and have PING enabled in the firewall.

  • Red11 =, mask, gateway
  • Red22 =, mask, gateway
  • Blue11 =, mask, gateway
  • Blue22 =, mask, gateway

The Hyper-V Servers should have one External Virtual network, with the VMs attached.  The Hyper-V Servers need to be joined to the same domain with Migration enabled (we will be moving one VM).  All VMs should be created on one of the Hyper-V Servers, this is also the Hyper-V Server where you will run the script.

And, the last point.  When you prepare the demo to run, be sure to power on all VMs first, but power on the Red VMs first, then the Blue VMs.  Red was on-boarded into my infrastructure first, and Blue’s VMs will catch an IP conflict.

Start-VM Red* -AsJob
sleep 60
Start-VM Blue* –AsJob

Once they are all running and report an IP address to the host, we are ready to go.

$vmNics = Get-VMNetworkAdapter *
do {
    $upCount = ($vmNics | where {$_.IPAddresses.count -ne 0})
    sleep 30
until ($vmNics.Count -eq $upCount.Count)

At this time you can begin running through the following scenario, one command set at a time.  I used the PowerShell ISE and selected a block and then ran that block (F8).  If you want to make all of these commands into one-liners then you can use Start-Demo.ps1.

Here is the meat, with all my commentary in the comments.  Taking all the previous 6 posts and rolling them into something to demonstrate being the management layer.

   This is the scripted demonstration. Of Windows Network Virtualization
   This assumes that there are two Server 2012 Hyper-V machines, joined to the
   same domain.  Both hypervisors have one External Virtual Switch.
   This script is executed on one hypervisor, and has portions against the
   remote hypervisor.
   That local hypervisor has 4 (Server 2012) VMs named Red11, Red22, Blue11, Blue22.
   On the remote hypervisor I used a PowerShell dedicated endpoint created
   to work around CredSSP issues -
   For the demonstration the PowerShell v3 ISE was used to execute individual blocks of
   Brian Ehlert
   Senior Software Test Engineer 2
   Citrix Labs, Redmond

# Open a session to my remote host that I will simply re-use
$waldorf = New-PSSession waldorf.brianeh.local -ConfigurationName HVRemoteAdmin

# This is just a demo, no fancy stuff.  Everything is hard coded.
# Fancy scripting is for other examples.

# Red and Blue are customers that want me to host their VMs.

# There are four VMs Red1 Red2 Blue1 Blue2. (they need to live on one host and get along)

# They happen to have overlapping IP address schemes of 192.168.1.x

# IP Conflict - second powered up / set will not work. The OS auto disables and APIPA
Get-VMNetworkAdapter * | Format-Table VMName, Name, MACAddress, VirtualSubnetID, IPAddresses -AutoSize

# I could open a ticket with the Network folks to build a VLAN. This will take too long.

# I could force one of the clients to reconfigure their IP scheme. This is too complicated.

# Instead, I have set unique VirtualSubnetIDs for each customer (isolation boundary)

Get-VMNetworkAdapter Red* | Set-VMNetworkAdapter -VirtualSubnetId 445566
Get-VMNetworkAdapter Blue* | Set-VMNetworkAdapter -VirtualSubnetId 7788990

Get-VMNetworkAdapter * | Format-Table VMName, Name, MACAddress, VirtualSubnetID, IPAddresses -AutoSize

# IP Conflict is resolved.

# To clear the APIPA status and the default behavior of taking the conflicting IP offline:
Get-VMNetworkAdapter -VMName Blue* | Disconnect-VMNetworkAdapter
Get-VMNetworkAdapter -VMName Blue* | Connect-VMNetworkAdapter -SwitchName VMs

Get-VMNetworkAdapter * | Format-Table VMName, Name, MACAddress, VirtualSubnetID, IPAddresses -AutoSize

# Start a ping from Red11 to Red22
vmconnect Statler.brianeh.local Red11

# This works until I have to distribute VMs because one host is not enough capacity to carry everything for both customers.

Move-VM -Name “Red22” -DestinationHost Waldorf.brianeh.local –IncludeStorage –DestinationStoragePath D:\Red22

# ping is now broken as there is no way to route this additional tag

# So I set up Windows Network Virtualization

# Enable the WNV binding
# At my local host
$vSwitch = Get-VMSwitch -SwitchType External
Enable-NetAdapterBinding -InterfaceDescription $vSwitch.NetAdapterInterfaceDescription -ComponentID "ms_netwnv"

# At my Remote host
Invoke-Command -Session $waldorf {
$vSwitch = Get-VMSwitch -SwitchType External
Enable-NetAdapterBinding -InterfaceDescription $vSwitch.NetAdapterInterfaceDescription -ComponentID "ms_netwnv"

# Define the Provider Address in my address space
# At my local host
$vSwitch = Get-VMSwitch -SwitchType External
$swPhysIf = Get-NetAdapter -InterfaceDescription $vSwitch.NetAdapterInterfaceDescription
New-NetVirtualizationProviderAddress -InterfaceIndex $swPhysIf.InterfaceIndex -ProviderAddress -PrefixLength 21

# At my Remote host
Invoke-Command -Session $waldorf {
$vSwitch = Get-VMSwitch -SwitchType External
$swPhysIf = Get-NetAdapter -InterfaceDescription $vSwitch.NetAdapterInterfaceDescription
New-NetVirtualizationProviderAddress -InterfaceIndex $swPhysIf.InterfaceIndex -ProviderAddress -PrefixLength 21

# Generate a GUID for each customer
$redGUID = [system.guid]::newguid()
$blueGUID = [system.guid]::newguid()

# Format the GUID string properly
$redGUID = "{" + [string]$redGUID + "}"
$blueGUID = "{" + [string]$blueGUID + "}"

# Define a Customer Route for each customer
# A the local host
New-NetVirtualizationCustomerRoute -RoutingDomainID $redGUID -VirtualSubnetID 445566 -DestinationPrefix "“ -NextHop -Metric 255
New-NetVirtualizationCustomerRoute -RoutingDomainID $blueGUID -VirtualSubnetID 7788990 -DestinationPrefix "“ -NextHop -Metric 255

# At the remote host
Invoke-Command -Session $waldorf -ArgumentList $redGUID, $blueGUID {
Param($redGUID, $blueGUID)
New-NetVirtualizationCustomerRoute -RoutingDomainID $redGUID -VirtualSubnetID 445566 -DestinationPrefix "“ -NextHop -Metric 255
New-NetVirtualizationCustomerRoute -RoutingDomainID $blueGUID -VirtualSubnetID 7788990 -DestinationPrefix "“ -NextHop -Metric 255

# Define the Lookup Routes on all switches that require it
# First local
New-NetVirtualizationLookupRecord -VMName Red11 -VirtualSubnetID 445566 -CustomerAddress -MACAddress 00155D002009 -ProviderAddress -Rule TranslationMethodEncap -CustomerID $redGUID
New-NetVirtualizationLookupRecord -VMName Red22 -VirtualSubnetID 445566 -CustomerAddress -MACAddress 00155D002008 -ProviderAddress -Rule TranslationMethodEncap -CustomerID $redGUID
New-NetVirtualizationLookupRecord -VMName Blue11 -VirtualSubnetID 7788990 -CustomerAddress -MACAddress 00155D00200B -ProviderAddress -Rule TranslationMethodEncap -CustomerID $blueGUID
New-NetVirtualizationLookupRecord -VMName Blue22 -VirtualSubnetID 7788990 -CustomerAddress -MACAddress 00155D00200A -ProviderAddress -Rule TranslationMethodEncap -CustomerID $blueGUID

# Then remote
Invoke-Command -Session $waldorf -ArgumentList $redGUID, $blueGUID {
Param($redGUID, $blueGUID)
New-NetVirtualizationLookupRecord -VMName Red11 -VirtualSubnetID 445566 -CustomerAddress -MACAddress 00155D002009 -ProviderAddress -Rule TranslationMethodEncap -CustomerID $redGUID
New-NetVirtualizationLookupRecord -VMName Red22 -VirtualSubnetID 445566 -CustomerAddress -MACAddress 00155D002008 -ProviderAddress -Rule TranslationMethodEncap -CustomerID $redGUID
New-NetVirtualizationLookupRecord -VMName Blue11 -VirtualSubnetID 7788990 -CustomerAddress -MACAddress 00155D00200B -ProviderAddress -Rule TranslationMethodEncap -CustomerID $blueGUID
New-NetVirtualizationLookupRecord -VMName Blue22 -VirtualSubnetID 7788990 -CustomerAddress -MACAddress 00155D00200A -ProviderAddress -Rule TranslationMethodEncap -CustomerID $blueGUID

# Ping is working again

# Take a look and the LookupRoutes
Get-NetVirtualizationLookupRecord | ft CustomerAddress, CustomerID, Provideraddress, macaddress, rule, vmname -AutoSize

# Break one of the routes, this can be due to a VM HA event, or a mis-configuraiton.
Remove-NetVirtualizationLookupRecord -CustomerAddress -VMName Red22
New-NetVirtualizationLookupRecord -VMName Red22 -VirtualSubnetID 445566 -CustomerAddress -MACAddress 00155D002008 -ProviderAddress -Rule TranslationMethodEncap -CustomerID $redGUID

# Now the route is incorrect, what happens? (check ping)

# Repair the route

Remove-NetVirtualizationLookupRecord -CustomerAddress -VMName Red22
New-NetVirtualizationLookupRecord -VMName Red22 -VirtualSubnetID 445566 -CustomerAddress -MACAddress 00155D002008 -ProviderAddress -Rule TranslationMethodEncap -CustomerID $redGUID


Anonymous said...

Hey Brian
Just wondering if theres a way of troubleshooting this as I cant get the transition to another host working. I can ping between Red11-Red22, and Blue11-Blue22 whilst hosted on the same VHost, but as soon as Red22 goes over to Vhost2 and I set up the LookupRecords it stops working.
Does the Provider IP Address have to match a physical IP on the host?



BrianEh said...

The PA address is not the same as the management ip address. (It took me a bit to get my brain past this at first).

The PA address is assigned to the physical NIC of the External virtual switch.

The PA addresses assigned must be routable between the two hosts. This is all about sub-netting.

The LookupRecords on host 1 and host 2 should be identical. If the VM is running when it is migrated then the hardware is not changed, if the VM is powered off the NIC of the VM might get changed.

The key is that not a single step is missed in the setup.

Each Host needs a unique PA, but past that, the rules on each side should be identical - the same CustomerRoute, the same LookupRecords.

And the LookupRecords are the equivalent of a Routing Table in a router. So they must be correct of rhte data to flow.

Anonymous said...

Nice - thanks for that. I got it going eventually - pretty cool feature when you understand how it works! Now to get it going via VMM...


Anonymous said...

Thank you for your post, Brian!

I am trying to set it up now. You stated that "The Hyper-V Servers should have one External Virtual network, WITH the VMs attached."

Would you be so kind to advise on the following items?
Is it absolute necessity to have VMs attached to it and why? By any chance is there a way to use "Internal Network" virtual switch?

Thank you for your help, in advance!


BrianEh said...

By definition and by the design of how the feature works, it must be an External Virtual Switch.

The WNV filter is actually bound tot he physical NIC, between the physical NIC and the Virtual Switch.

This is why if you add just the virtual subnet ID to the vNIC you get the internal virtual switch behavior of creating a VLAN that is local to the virtual switch.

Anonymous said...

Brain, I was able to sucessfully execute your demo script and the Vms in the same subnet communicate. But i was facing issue for the VMs in different subnets communicating each other. I have added both different virtual subnets to same routing domain Id, but still i do not see VM's communicating across virtual subnets. Do i need to configure any gateway VM for VMs in different subnets (either on same host/differenthost) to communicate? or do you any demo script for it?

BrianEh said...

individual virtual subnets are no different than VLANs in concept - there needs to be routing between them. otherwise they are isolated.

so yes. you need to provide the router.

Anonymous said...

Thanks for clarification Brain, i have come across demo script that builds gateway as in below

Is the above script helps in building gateway between virtualized and non-virtualized traffic or does this enable routing between different virtual subnets?

i am actually looking for some script that enables routing between vm in different virtual subnets with nvgre enabled traffic.

BrianEh said...

The MSFT example that you found expects a dual homed VM that you must install Server 2012 into and properly set up RRAS for forwarding (making it a router essentially).

Their model (and lingo) is that the 'gateway' is out of the NVGRE tunnel to the physical. The designed for scenario is that the machines within the NVGRE tunnel need to talk to some other machines, someplace else, and that traffic needs to cross territory where NVGRE is not supported.

Take the scenario where you have VMs in Azure on an Azure Virtual Network. And you use the Azure Gateway to connect to your VMs in your datacenter. They use the Gateway in the same way. They just enforce an endpoint type.

So, if you have a subnet that is 192.168.0.X - the assumption is that .1 is your gateway. You simply place a VM on that IP within your vSubnet (it could be a Vyatta router) and the other NIC of the Vyatta router is on your other vSubnet at Then you set your routing rules between the two within the router.

I think that folks expect the Network Virtualization to also do the job of a router, and it doesn't. It does the job of an intelligent Switch. More similar to port forwarding.

Does that help?

I have not played with this in months, so you have me thinking a bit.

The other key to this is that if you reboot your host - this configuration is all gone. You need to re-build all the routing rules. This is why MSFT is pushing SCVMM for this - as it does just that for you. So, unless you are the management layer, you need to consider this.

Anonymous said...

Thanks for clarification brian

BrianEh said...

IMHO - it is the lack of documentation (today) about this combined with the \\BUILD 2011 hype combined with the marketing of "use SCVMM" that makes all the confusion here.

I think that the MSFT scripts are great for folks that really know what they are doing.

And it is the lack of persistence / self-healing that make managing this a big value add for SCVMM, the NEC openVSwitch, the Cisco Nexus, etc.

I mean - just consider the scenario of a clustered hypervisor - you reboot it and all the VMs have evacuated. Those running hosts now need all their NVGRE route tables fixed up. And that response to that event has to happen pretty quickly or you have lost pings.

Clustering would need to manage it. But clustering is not always involved. So the switch would need to manage it. But then it would need its own management layer. Oh wait, what is what SCVMM does, and these third party's with switch extensions (and replacements).

So, unless you are building a management layer - this is fun for demonstration and play and a very static world. But gets messy really quick.

Anonymous said...

Can i use windows server 2012 VM running RRAS to act as router between two virtual machines in diifferent subnets ?

BrianEh said...

absolutely. If you script the RRAS set-up I would love to se it

Anonymous said...

Brain, i tried to set up rras LAN router with one management nic to connect from host, one nic with (which is in 5001 vsubnet) and another nic with is in 7001 vsubnet) and provided them as gateways for Vms in the corresponding subnets. But i could not ping from subnet. Am i missing something here? Do i need to configure any static routes

In nvgre environment, since the VMs in each subnet are in different virtual subnet ids,how does the rras router know about virtual subnet ids?

Let me know your thoughts on this

BrianEh said...

As long as your gateways are set up properly and you have forwarding enabled it should route.

BTW- if you are using Server 2008 or newer, ping won't reply without enabling it.

the RRAS router has to have one vNIC associated with each virtual subnet.
This is at the VM vNIC configuration level.

Anonymous said...

Brain, interestingly VMs of different virtual subnets were able to communicate without having separate router. The configuration i have is
Host-1 ---> VM1 ---> vSubnetId:5001 -
Host-2 ---> VM2 ---> vSubnetId:7001 -

By assigning same routing domain id to both vSubnets and creating gateway records for , on both hosts, they were able to communicate

BrianEh said...

That is the way it is supposed to work. However, I never had time to try.

Can you post your configuration? As examples of defining the gateway?

Anonymous said...

The hosts are restarted and all the configuration is gone.It is really cumbersome to have all my configuration back. I have tried it again but i was not successful. will post you know once i get it working.

Have you tried having DHCPVM on host1 and able to assign IPAddress to VM1 on another host2 (both DHCPVM and VM1 are in subnet 5001)in nvgre environment without SCVMM? Because unless we create lookup records it does not create DHCPVM and VM1 in the same virtual network which requires customer IPaddress. But if it is DHCP environment you will not have IPaddress. It is like chicken and egg problem.

With SCVMM , i see they recommend DHCP extension to be installed on hosts and it manages assigning IP to hosts. But is it possible to manage with DHCPVM on one host without SCVMM in nvgre environment?

BrianEh said...

DHCP running in a VM does not work and will not work. I spent a lot of time with that, and have it confirmed from the MSFT folks.
In theory it might if you define the broadcast domains, but it doesn't.

SCVMM uses a custom switch extension to perform their DHCP magic. They intercept the DHCP request, and respond to it from an IPPool

And the reboot / rules wipe. Yes, I have mentioned that pitfall in the blog here.

I did create a script that simply queries host a and host b and builds the rules based on the current location of the VMs.

BrianEh said...

If you missed this; Here is my setup script.
Note I have hardcoded a few things that you might want to use and XML or TXT file to define.

Anonymous said...

Im really lost with testing of NVGRE. I was able to test it with host (Provider Addresses) connected on access ports on the switch, but when im using teamed adapters connected on 802.1q trunk ports NVGRE seems not working. I really believe that Microsoft does not develop it in the way NVGRE to work only on access ports, but how? What is different in the configuration? What if im using 2 teamed converged adapters for cluster, management and tenant network?

BrianEh said...

So, lets gather some information.

You have an LBFO Team. These are trunked ports. (though I believe that NVGRE operates on VLAN 0, it is not VLAN tagged traffic).

Where is the WNV filter enabled? (which NIC, the physical or the Team) Since the switch should be attached to the Team the WNV filter needs to be enabled here as well.

Anonymous said...

Yes, WNV is enabled on the Teamed adapter. vSwitch is attached on the teamed adapter. NVGRE PA adapter operates between vSwitch and the teamed adapter. So is it possible this PA adapter (NVGRE) to operate as different than VLAN 0 - there is a VlanID switch that could be set:

ProviderAddress :
InterfaceIndex : 12
PrefixLength : 0
VlanID : 11 <---------------------------------------
AddressState : Preferred

Or the only way to establish NVGRE between hosts is to use VlanID: 0 ?

BrianEh said...

I have not tried using the VLAN ID on the PA.

Your InterfaceIndex should be the teamed network adapter where you enabled WNV, which should be the teamed adapter that the virtual switch is attached.
InterfaceIndex 12, is generally a physical device, not a teamed adapter.

Technically, WNV does all the routing, so it is a filter driver above the network adapter and below the virtual switch. In between, if you will.

Anonymous said...

We used Wireshark to investigate the problem the answer was KB2779768 :) no comment... we spend 2 days on this.

the problem is explained on high level from one of my collegues at this blog post

I appreciate your help!

Best Regards Yordan

BrianEh said...

The feedback that I received from the product team on this is:

The KB MUST be installed on all WNV hosts.
Otherwise the packet formats will differ between the patched and un-patched hosts.

So if you either all hosts patched or all hosts unpatched.

I hope that helps.

Anonymous said...

Dear , I can n't routing between two VSID .
How can i Creat Gateway Records .plz help me

BrianEh said...

There is no built-in router. so defining a gateway means that you must have a vm that serves as a gateway at that address.

work through my entire blog series. you will see how to define everything.

Anonymous said...

Hi, thanks for posting this blog, It really help me much.
And I have a question to ask:
Where to generate the guid? On customer's VMs, as in this article, they are four VMs and would generate four different GUIDs. I am a little confuse.
Could you please send me an e-mail to "" when you see this?
Thanks in advanced.

BrianEh said...

You have to generate a GUD for the customer, as shown here:
# Generate a GUID for each customer
$redGUID = [system.guid]::newguid()
$blueGUID = [system.guid]::newguid()

Beyond that, each VM gets a GUID from the hypervisor at the time the VM is created on the hypervisor (they all do this). You can only query the VM GUID, not create it.

Anonymous said...

Hi Brian,

I'm missing one part of the Gateway (vvyatta router) configuration. One vNIC is connected to NVGRE network and other vNIC is connected to an external Virtual Switch bound to a physical NIC which in turn is connected to PLAN.

My questions are:

1. What IP Address I should assign to vNIC (eth0) of Router which is connected to NVGRE network?
2. What IP Address should be assigned to vNIC (eth1) of Router which is connected to another External Virtual Switch? Is it going to be a public IP Address
3. And what IP address should be assigned to Virtual Switch which connects Router VM?

And also in one of your posts you say that NVGRE PA address should be routable. But IMHO, PA address will be a non-routable IP Address if we are going to use VM router?

Thank you for your time for looking into this for me!


BrianEh said...

The vNIC within the NVGRE network must have an IP on that subnet. The vNIC on the LAN needs an IP on that subnet.

PA addresses are routable. They exist on the LAN subnet and they are assigned to the virtual switch. No different than a physical router that has multiple IP addresses on one port. PA addresses are the wrapper IP addresses that transport the encapsulated NVGRE packets between the individual hosts that participate in the NVGRE subnet. Networking rules still apply.

Anonymous said...

Thanks for the reply! However, there are a few things which are still bothering me..I tried to search everywhere but no luck!

My understanding about NVGRE is that when a packet arrives from a NVGRE VM, the packet is received by the PA Switch, the switch uses LookupRecords table to check for the destination. It finds that the packet is destined for a VM which is located on another Hyper-V Host. at this point, the packet's destination is known!

Now the thing is that:

-In case of non-Gateway VM, the packet is sent to wire via PNIC of NVGRE Switch (PA address is routable). Basically, the packet exists via PNIC bound to NVGRE Switch.

-In case of a VM Gateway (vNIC assigned with NVGRE gateway IP : x.x.50.1), the packet is re-routed to the VM Gateway? Is that the case?

Would I be accurate if I describe that the NVGRE traffic will "always" be sent to default gateway (x.x.50.1) whether a Gateway VM is implemented or not? But that depends on the ProviderRoute created as below:

New-NetVirtualizationProviderRoute -InterfaceIndex $NIC.InterfaceIndex -DestinationPrefix "" -NextHop ""


Another thing bothering me is the IP address configuration:

The configuration looks like below in case of non-VM gateway:
All VMs - 192.168.50.x...
PA: public IP Address (which is a routable IP)

Very clear on this part if no Gateway VM implemented.

but in case of a Gateway VM, the configuration looks like below:

-All VMs - 192.168.50.x
-PA IP - (which is a non-routable IP)
-VM Gateway - vNIC1 - (connected to NVGRE)
-VM Gateway - vNIC2 - this is where the confusion is. Should I assign public IP Address to this vNIC?
if I assign Public IP Address then what IP address should be assigned to the non-NVGRE virtual Switch. Should it be left blank?

Thank you for clarifying my doubts!

sunny said...

Hi guys,

can any one help me what is cmdlet in Windows 2012 R2 to connect the vm like "New-VMConnectSession" in Windows 2008 R2

Thanks to all


BrianEh said...

Unfortunately, MSFT did not release the built-in PowerShell cmdlets until Server 2012.
The best that you can do is invoke vmconnect.exe directly with the connection to option.

Anonymous said...

Hi Brian,

Nice and detailed Article !

Actually I am stuck in Provider Address (PA), where should I punch the PA IP?

as you mentioned in your post that this is assigned to the physical NIC of the External Switch - You meant the Virtual adapter? because we can't set IP on P NIC once it is used for vSwitch.

BrianEh said...

Take a look at the section here: New-NetVirtualizationProviderAddress -InterfaceIndex $swPhysIf.InterfaceIndex -ProviderAddress -PrefixLength 21

The PA IP can be thought of as an alternate IP to the external port of the virtual switch (you can do this with physical switches, why not virtual ones).

Technically, it is not applied to a vNIC that you can actually 'see', it is applied to a hidden port.

Since virtual devices are conceptually similar to physical devices with ports, patch cables (virtual connections between ports), vNICs, etc.

A virtual switch has ports. And in reality, a vNIC is a virtual switch port.
But I am probably explaining too much in an attempt to help you visualize.

There is a blog series here, and PA addresses are a topic of an entire post themselves.