In part one of this post I covered the basic theory of operations and functionality of VXLAN (http://www.definethecloud.net/vxlan-deep-dive.) This post will dive deeper into how VXLAN operates on the network.
Let’s start with the basic concept that VXLAN is an encapsulation technique. Basically the Ethernet frame sent by a VXLAN connected device is encapsulated in an IP/UDP packet. The most important thing here is that it can be carried by any IP capable device. The only time added intelligence is required in a device is at the network bridges known as VXLAN Tunnel End-Points (VTEP) which perform the encapsulation/de-encapsulation. This is not to say that benefit can’t be gained by adding VXLAN functionality elsewhere, just that it’s not required.
Providing Ethernet Functionality on IP Networks:
As discussed in Part 1, the source and destination IP addresses used for VXLAN are the Source VTEP and destination VTEP. This means that the VTEP must know the destination VTEP in order to encapsulate the frame. One method for this would be a centralized controller/database. That being said VXLAN is implemented in a decentralized fashion, not requiring a controller. There are advantages and drawbacks to this. While utilizing a centralized controller would provide methods for address learning and sharing, it would also potentially increase latency, require large software driven mapping tables and add network management points. We will dig deeper into the current decentralized VXLAN deployment model.
VXLAN maintains backward compatibility with traditional Ethernet and therefore must maintain some key Ethernet capabilities. One of these is flooding (broadcast) and ‘Flood and Learn behavior.’ I cover some of this behavior here (http://www.definethecloud.net/data-center-101-local-area-network-switching) but the summary is that when a switch receives a frame for an unknown destination (MAC not in its table) it will flood the frame to all ports except the one on which it was received. Eventually the frame will get to the intended device and a reply will be sent by the device which will allow the switch to learn of the MACs location. When switches see source MACs that are not in their table they will ‘learn’ or add them.
VXLAN is encapsulating over IP and IP networks are typically designed for unicast traffic (one-to-one.) This means there is no inherent flood capability. In order to mimic flood and learn on an IP network VXLAN uses IP multi-cast. IP multi-cast provides a method for distributing a packet to a group. This IP multi-cast use can be a contentious point within VXLAN discussions because most networks aren’t designed for IP multi-cast, IP multi-cast support can be limited, and multi-cast itself can be complex dependent on implementation.
Within VXLAN each VXLAN segment ID will be subscribed to a multi-cast group. Multiple VXLAN segments can subscribe to the same ID, this minimizes configuration but increases unneeded network traffic. When a device attaches to a VXLAN on a VTEP that was not previously in use, the VXLAN will join the IP multi-cast group assigned to that segment and start receiving messages.
In the diagram above we see the normal operation in which the destination MAC is known and the frame is encapsulated in IP using the source and destination VTEP address. The frame is encapsulated by the source VTEP, de-encapsulated at the destination VTEP and forwarded based on bridging rules from that point. In this operation only the destination VTEP will receive the frame (with the exception of any devices in the physical path, such as the core IP switch in this example.)
In the example above we see an unknown MAC address (the MAC to VTEP mapping does not exist in the table.) In this case the source VTEP encapsulates the original frame in an IP multi-cast packet with the destination IP of the associated multicast group. This frame will be delivered to all VTEPs participating in the group. VTEPs participating in the group will ideally only be VTEPs with connected devices attached to that VXLAN segment. Because multiple VXLAN segments can use the same IP multicast group this is not always the case. The VTEP with the connected device will de-encapsulate and forward normally, adding the mapping from the source VTEP if required. Any other VTEP that receives the packet can then learn the source VTEP/MAC mapping if required and discard it. This process will be the same for other traditionally flooded frames such as ARP, etc. The diagram below shows the logical topologies for both traffic types discussed.
As discussed in Part 1 VTEP functionality can be placed in a traditional Ethernet bridge. This is done by placing a logical VTEP construct within the bridge hardware/software. With this in place VXLANs can bridge between virtual and physical devices. This is necessary for physical server connectivity, as well as to add network services provided by physical appliances. Putting it all together the diagram below shows physical servers communicating with virtual servers in a VXLAN environment. The blue links are traditional IP links and the switch shown at the bottom is a standard L3 switch or router. All traffic on these links is encapsulated as IP/UDP and broken out by the VTEPs.
Summary:
VXLAN provides backward compatibility with traditional VLANs by mimicking broadcast and multicast behavior through IP multicast groups. This functionality provides for decentralized learning by the VTEPs and negates the need for a VXLAN controller.
Hey Joe
Thanks for these articles.
Do you know if there are any physical VTEP bridge capable devices available today?
Andy,
See Jon’s comment below. As he stated Brocade and Arista are the only options I’m aware of but you can expect to see more over the next few months.
Joe
I “think” just Arista and Brocade. But I may be missing someone.
Jon,
Thanks for reading and the reply. To my knowledge you are correct.
Joe
Hi Joe,
Thanks very much for this great summary of the VXLAN stuff.
One minor comment, in ‘unknown MAC example’ above you use the address 224.0.0.1 which is reserved for ‘ All Hosts multicast group’. I suggest you pick an address which is not part of the reserved range 224.0.0.x.
Alon
Tanning accelerators which contain Tyrosine are also available in the market.
You can get positive results of golden brown color with
the help these spray tanning products. The coloring
initially comes from bronzers, until DHA has time to react with
skin and gradually darkens dead skin cells.
One of the benefits of purchasing a self tanning lotion that has a bronzer included is that you can purchase a lotion that works for your complexion.
Some salons offer unlimited tanning packages deals, while with others the cost of
tanning for a month can run you several hundred dollars, so call ahead.
With this in mind, staff have compiled a list of the main things that you need to know before starting out.
Because cut-throat competition exists between them, as we are
all aware, popular brands of beauty products are all improving their tanning products day by
day, allying comfort, lightness and hydration. First of all, THE
most important part of a spray tan is the solution being used by your technician. Sprays: Sunless tanning
sprays are very popular today and are a method to achieve
a sunless tan. Custom coatings are regularly engineered by APS Materials,
Inc.
-1 person holding the ball, while the rest (or just attackers) run into space
and try to shake off the defenders. However, there are some potential downsides to consider
as well. However, financial help is always readily available for autistic summer camps kids who need
it. The activities at the day camp include
computer based assignments, scientific research and sports etc.
Children and teens will benefit most from their experience if they are in a safe, and therefore, happy environment.
Good summer camps allow kids living with Down syndrome to practice independence so that
they can develop physically and mentally. Alternatively, there are
lots of camps which require the kids to go overnight or stay for days away from home.
Session one campers will study and perform “Stuart LIttle: The Musical” June 12-25.
If you don’t, pickpocket the disarm codes off of Curtis. You
literally lie on the floor like a board propping your self up with your
elbows and you’ll feel your abs burning like by no means before.
And if you want to lose weight by exercising, then you must establish a program and not everyone can do this.
Keeping yourself fit is not an easy job if you understand
what to eat and what not to. Unlike grapes or ripe bananas, apples stay firm in a backpack or purse,
requiring no special protection. So, whenever you take
a capsule of Pure Garcinia Cambogia, you are actually waging
war against the fat cells that are stubbornly clinging to your organs,
making you sicker by the day. Every system has an individual
way of adapting to diets, effort level changes and programs meant to accelerate metabolism.
If I’m not sitting in front of the screen working, I’m normally surfing the net looking for healthy recipes for the evening meal, or, I
am probably checking out sites on weight loss & fitness tips.
Hawthorne – Hawthorne tells the story regarding
Christina Hawthorne, a widow with a daughter who functions
because Chief Nursing Officer; or head nurse at one of the surrounding hospitals.
How effective are Garcinia Cambogia weight loss products.
Doctors who opposed the measure argued that obesity is based on overeating and not getting enough exercise.
To be physically fit, the first thing you
must need is a balanced system.
Hi! I understand this is kind of off-topic however I had to ask.
Does building a well-established website such as yours take a
large amount of work? I am completely new to writing a blog but I do write in my diary on a daily basis.
I’d like to start a blog so I will be able to share my own experience and views
online. Please let me know if you have any kind of suggestions or tips for brand new aspiring blog
owners. Thankyou!
What if VM sends IP packet to other VM insted of ethernet frame?
Well, an IP packet will eventually be contained in an Ethernet frame if the servers are using ethernet NICs.
what happens when a machine sends traffic to an IP address which does not belong to vxlan group. How the switch differentiates the packet whether to handle it to vtep or regular switching or routing path?
Joe, thank you for the information. I appreciate your dedication and desire to educate folks very much. I must give a bit of criticism, though. I’m not sure I totally agree with the manner in which you make some points, but overall it’s fine. That having been said, IMHO, you should have someone proof-read your writing to ensure proper grammar and punctuation. It is sometimes necessary to re-read a sentence several times before its meaning becomes clear.
Below is an example sentence that showcases both problems that involve delivery and punctuation/grammar.
“VXLAN is encapsulating over IP and IP networks are typically designed for unicast traffic (one-to-one.) This means there is no inherent flood capability. In order to mimic flood and learn on an IP network VXLAN uses IP multi-cast. ”
“It should read something along the lines of “VxLAN encapsulates traffic and adds an IP header.”
And the following sentence is where I disagree with your explanation. IP can be EITHER/OR unicast or multi-cast. Your sentence makes it sounds as though multicast is some sort of add-on technology or technique that helps IP mimic Ethernet broadcasts. That is not true. IP is either unicast or multicast – both fall under the rubric of IP.
And then there is the final sentence:
“In order to mimic flood and learn on an IP network VXLAN uses IP multi-cast. ”
It should include a comma after the word “learn.” That word ends a prepositional phrase, hence the need for a comma. Finally, “flood and learn” should be hyphenated, as in flood-and-learn, because both words describe a single type of behavior. Lastly, to be a bit clearer, the word “Ethernet’s” should probably precede “flood-and-learn” to help differentiate between the two, and the word “behavior” should come after “flood-and-learn.”
Corrected sentence:
“In order to mimic Ethernet’s flood-and-learn behavior on an IP network, VXLAN uses IP multi-cast. ”
I hope this helped…