CN115174312B

CN115174312B - Broadcast information transmission method, tunnel endpoint device, electronic device, and medium

Info

Publication number: CN115174312B
Application number: CN202210789970.9A
Authority: CN
Inventors: 张余
Original assignee: China United Network Communications Group Co Ltd
Current assignee: China United Network Communications Group Co Ltd
Priority date: 2022-07-06
Filing date: 2022-07-06
Publication date: 2023-04-18
Anticipated expiration: 2042-07-06
Also published as: CN115174312A

Abstract

The present disclosure provides a broadcast information sending method, a first tunnel endpoint device, a second tunnel endpoint device, an electronic device, and a storage medium, to solve the problem of repeated sending of broadcast information, where the method is applied to a first tunnel endpoint device having a single-hanging device in a cross-device link aggregation group, and includes: the method comprises the steps of establishing tunnels by using a virtual IP address and an IP real address of a second tunnel endpoint device of a non-identical cross-device link aggregation group, increasing a device identification of a first tunnel endpoint device in sent routing information when the tunnels are established, so that when the second tunnel endpoint device sends broadcast information, judging that two tunnels from the second tunnel endpoint device to the first tunnel endpoint device belong to the same VNI (virtual network interface) of the same device through the device identification of the first tunnel endpoint device in the routing information, selecting one tunnel when a head end copy list is set, and only sending one piece of broadcast information to the device, thereby avoiding sending repeated broadcast information and reducing loads of the tunnels and the device.

Description

Broadcast information transmission method, tunnel endpoint device, electronic device, and medium

Technical Field

The present disclosure relates to the field of communications technologies, and in particular, to a broadcast information sending method, a first tunnel endpoint device, a second tunnel endpoint device, an electronic device, and a computer-readable storage medium.

Background

A Multi-chassis Link Aggregation Group (M-LAG) is a mechanism for implementing inter-chassis Link Aggregation, and can implement Link Aggregation among multiple devices, thereby improving the Link reliability from a single board level to a device level.

In a VXLAN (Virtual Extensible Local Area Network) Network, after two devices VTEP (VXLAN Tunnel End Point) 1 and VTEP2 in a cross-device link aggregation mode complete configuration, VTEP1 and VTEP2 can determine whether a Media Access Control (MAC) address is a single-hanging device through a cross-device link aggregation link identifier when synchronizing MAC entries. Meanwhile, different addresses are used between the VTEP1 and the VTEP2 as BGP (Border Gateway Protocol) peer addresses, and BGP EVPN (Ethernet Virtual Private Network) neighbors are mutually established.

When VM (Virtual Machine) 1 is a single-hanging device accessed to VTEP2, VTEP2 in a cross-device link aggregation group establishes two VXLAN tunnels with VTEP3 by using a Virtual VTEP IP address and a real address of VTEP2 respectively when establishing a tunnel with a device VTEP3 which does not belong to the same cross-device link aggregation group, wherein the Virtual VTEP IP address and the real address of VTEP2 notify routing information of VM1 by the tunnel of the real address of VTEP2, so that VTEP2 and VTEP3 have two VXLAN tunnels, when VTEP3 is to send broadcast information, the broadcast information is repeatedly sent to VTEP2 by the two tunnels, thereby increasing the load of the link between VTEP2 and VTEP3 and consuming the resources of VTEP2 and VTEP3 devices.

Disclosure of Invention

In order to at least solve the technical problem that link resources and device processing capacity between devices are wasted due to repeated transmission of broadcast information possibly existing when VTEP broadcast information is aggregated by a single-hanging device across-device link in the prior art, the present disclosure provides a broadcast information transmission method, a first tunnel endpoint device, a second tunnel endpoint device, an electronic device, and a computer-readable storage medium, which can avoid transmission of repeated broadcast information and reduce loads of tunnels and devices.

In a first aspect, the present disclosure provides a broadcast information sending method, applied to a first tunnel endpoint device in a cross-device link aggregation group, where the first tunnel endpoint device has a first single-hanging device, and the method includes:

establishing two VXLAN tunnels with the virtual IP address of the first tunnel endpoint device and the real IP address of the first tunnel endpoint device and a second tunnel endpoint device of a non-identical cross-device link aggregation group respectively;

when two VXLAN tunnels are established, type3 routing information is respectively sent to the second tunnel endpoint device, and device identification information of a first tunnel endpoint device is respectively added in the two sent Type3 routing information, so that after the second tunnel endpoint device receives the two types 3 routing information, the second tunnel endpoint device judges that two tunnels from the second tunnel endpoint device to the first tunnel endpoint device are both VXLAN tunnels which are to the same device and belong to the same VNI (VXLAN Network Identifier) through the device identification information of the first tunnel endpoint device in the two types 3 routing information, and selects one of the tunnels when a head end copy list is set, so that information broadcasted to the first tunnel endpoint device is sent to the first tunnel endpoint device through one selected tunnel only.

Further, the method further comprises:

receiving a BGP EVPN route sent by second tunnel endpoint equipment, and acquiring an IP address of the second tunnel endpoint equipment;

when two VXLAN tunnels are respectively established, the IP address of the second tunnel endpoint equipment is used as the IP address of the opposite end to establish the tunnel;

and judging that the two tunnels from the first tunnel endpoint device to the second tunnel endpoint device belong to the VXLAN tunnel belonging to the same VNI as the same device through the IP addresses of the opposite ends in the two VXLAN tunnels, and selecting one of the tunnels when a head end copy list is set so as to send the information broadcasted to the second tunnel endpoint device through the selected tunnel only when the broadcast information is sent.

Further, the device identification information of the first tunnel endpoint device is the MAC address of the first tunnel endpoint device.

Further, establishing a VXLAN tunnel with the virtual IP address of the first tunnel endpoint device and the second tunnel endpoint device of the non-identical cross device link aggregation group includes:

sending first Type3 routing information to a second tunnel endpoint device, and adding device identification information of the first tunnel endpoint device in the first Type3 routing information, wherein a home terminal IP address in the first Type3 routing information is set as a virtual IP address of the first tunnel endpoint device;

establishing a first VXLAN tunnel with a second tunnel endpoint device through the first Type3 routing information, wherein the source address of the tunnel is the virtual IP address of the first tunnel endpoint device, and the destination address is the IP address of the second tunnel endpoint device;

sending a first Type2 route to a second tunnel endpoint device through a first VXLAN tunnel so as to inform the second tunnel endpoint device of an MAC Address, a host ARP (Address Resolution Protocol) and host routing information of a host connected to the first tunnel endpoint device, wherein the host routing information is added with a next hop routing attribute, the next hop Address is set as a virtual IP Address of the first tunnel endpoint device, and the host does not comprise the first single-hanging device.

Further, establishing a VXLAN tunnel with the IP real address of the first tunnel endpoint device and the second tunnel endpoint device of the non-identical cross device link aggregation group includes:

sending second Type3 routing information to a second tunnel endpoint device, and adding device identification information of the first tunnel endpoint device in the second Type3 routing information, wherein the home terminal IP address in the second Type3 routing information is set as the IP real address of the first tunnel endpoint device;

establishing a second VXLAN tunnel with a second tunnel endpoint device through the second Type3 routing information, wherein the source address of the tunnel is the IP real address of the first tunnel endpoint device, and the destination address is the IP address of the second tunnel endpoint device;

and sending a second Type2 route to a second tunnel endpoint device through a second VXLAN tunnel, wherein the second Type2 route comprises the MAC route of the first single-hanging device so as to inform the second tunnel endpoint device of the route information of the first single-hanging device, a next hop route attribute is added, and the next hop address is set as the IP real address of the first tunnel endpoint device.

In a second aspect, the present disclosure provides a broadcast information sending method, which is applied to a second tunnel endpoint device, where the method includes:

establishing two VXLAN tunnels with the first tunnel endpoint device according to the virtual IP address of the first tunnel endpoint device and the IP real address of the first tunnel endpoint device respectively;

respectively receiving Type3 routing information sent by the first tunnel endpoint device when two VXLAN tunnels are established, wherein the first tunnel endpoint device is a device in a cross-device link aggregation group and is provided with a first single-hanging device, the second tunnel endpoint device and the first tunnel endpoint device are devices in a non-identical cross-device link aggregation group, and the two pieces of Type3 routing information which are respectively received both comprise device identification information of the first tunnel endpoint device;

after receiving the two types of Type3 routing information, judging that the two tunnels from the second tunnel endpoint device to the first tunnel endpoint device are VXLAN tunnels which are to the same device and belong to the same VNI through the device identification information of the first tunnel endpoint device in the two types of Type3 routing information, and selecting one of the tunnels when setting a head end copy list, so that information broadcasted to the first tunnel endpoint device is only transmitted to the first tunnel endpoint device through the selected tunnel when transmitting broadcast information.

Further, the method further comprises:

sending a BGP EVPN route to the first tunnel endpoint equipment so that the first tunnel endpoint equipment acquires an IP address of second tunnel endpoint equipment, establishing tunnels by taking the IP address of the second tunnel endpoint equipment as an opposite end IP address when two VXLAN tunnels are respectively established, judging that the two tunnels from the first tunnel endpoint equipment to the second tunnel endpoint equipment belong to the same VNI (virtual network interface) by taking the opposite end IP addresses of the two VXLAN tunnels as the IP address of the second tunnel endpoint equipment, selecting one of the tunnels when a head end copy list is set, and sending the information broadcasted to the second tunnel endpoint equipment only through the selected tunnel when the broadcast information is sent.

In a third aspect, the present disclosure provides a first tunnel endpoint device, the first tunnel endpoint device being a device in a cross-device link aggregation group, the first tunnel endpoint device having a first single-hanging device, the first tunnel endpoint device comprising:

the first tunnel establishment module is set to establish two virtual extensible local area network VXLAN tunnels respectively by using the virtual IP address of the first tunnel endpoint equipment, the IP real address of the first tunnel endpoint equipment and the second tunnel endpoint equipment of the non-same cross-equipment link aggregation group;

a first sending module, configured to send Type3 routing information to the second tunnel endpoint device respectively when two VXLAN tunnels are established, and add device identification information of a first tunnel endpoint device in the two sent Type3 routing information respectively, so that after the second tunnel endpoint device receives the two types 3 routing information, it determines, through the device identification information of the first tunnel endpoint device in the two types 3 routing information, that two tunnels from the second tunnel endpoint device to the first tunnel endpoint device are VXLAN tunnels that are to the same device and belong to the same virtual extensible local area network identifier VNI, and selects one of the tunnels when a head-end copy list is set, so that when broadcast information is sent, information broadcasted to the first tunnel endpoint device is sent to the first tunnel endpoint device only through the selected tunnel.

In a fourth aspect, the present disclosure provides a second tunnel endpoint device comprising:

a second tunnel establishing module, configured to establish two VXLAN tunnels with the first tunnel endpoint device by using the virtual IP address of the first tunnel endpoint device and the IP real address of the first tunnel endpoint device, respectively;

a receiving module, configured to receive Type3 routing information sent by the first tunnel endpoint device when two VXLAN tunnels are established, where the first tunnel endpoint device is a device in a cross-device link aggregation group and has a first single-hanging device, and the second tunnel endpoint device and the first tunnel endpoint device are devices in a non-same cross-device link aggregation group, and both the two pieces of Type3 routing information received respectively include device identification information of the first tunnel endpoint device;

a second broadcast module, configured to, after receiving the two pieces of Type3 routing information, determine, through device identification information of a first tunnel endpoint device in the two pieces of Type3 routing information, that two tunnels from the second tunnel endpoint device to the first tunnel endpoint device are both VXLAN tunnels to the same device and belonging to the same VNI, and select one of the tunnels when setting a head end copy list, so that, when sending broadcast information, the information broadcasted to the first tunnel endpoint device is sent to the first tunnel endpoint device only through the selected tunnel.

In a fifth aspect, the present disclosure provides an electronic device comprising a memory and a processor, wherein the memory stores a computer program, and when the processor runs the computer program stored in the memory, the processor executes the broadcast information transmitting method according to any one of the first and second aspects.

In a sixth aspect, the present disclosure provides a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, implements the broadcast information transmitting method of any one of the first and second aspects.

Has the beneficial effects that:

according to the broadcast information sending method, the first tunnel endpoint device, the second tunnel endpoint device, the electronic device and the computer readable storage medium, the device identification attribute is added in Type3 routing information, and after tunnels are established with the opposite terminal VTEP respectively by the virtual IP address and the real address, the opposite terminal VTEP can judge that two tunnels to the VTEP in the cross-device link aggregation group belong to the same VNI tunnel of the same device through the device identification attribute, and selects one tunnel for broadcast sending when a head end copy list is set during broadcast information sending, so that repeated broadcast information sending can be avoided, and loads of the tunnels and the devices are reduced.

Drawings

Fig. 1 is a schematic flow chart of broadcast information transmission according to an embodiment of the present disclosure;

fig. 2 is a schematic flow chart of broadcast information transmission according to a second embodiment of the present disclosure;

fig. 3 is a schematic flow chart of broadcast information transmission according to a third embodiment of the present disclosure;

fig. 4 is an architecture diagram of a first tunnel endpoint device according to a fourth embodiment of the present disclosure;

fig. 5 is an architecture diagram of a second tunnel endpoint device according to a fifth embodiment of the present disclosure;

fig. 6 is an architecture diagram of an electronic device according to a sixth embodiment of the disclosure.

Detailed Description

In order to make the technical solutions of the present disclosure better understood by those skilled in the art, the present disclosure is further described in detail below with reference to the accompanying drawings and examples. It is to be understood that the specific embodiments and figures described herein are merely illustrative of the invention and are not limiting of the invention.

It should be noted that the terms "first," "second," and the like in the description and claims of the present disclosure and in the above-described drawings are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order; also, the embodiments and features of the embodiments in the present disclosure may be arbitrarily combined with each other without conflict.

Wherein the terminology used in the embodiments of the disclosure is for the purpose of describing particular embodiments only and is not intended to be limiting of the disclosure. As used in the disclosed embodiments and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise.

In the following description, suffixes such as "module", "component", or "unit" used to denote elements are used only for the convenience of explanation of the present disclosure, and have no specific meaning in themselves. Thus, "module", "component" or "unit" may be used mixedly.

The following are corresponding names and explanations of terms that may be mentioned in the present disclosure:

VXLAN (Virtual eXtensible Virtual local area network) is a two-layer VPN (Virtual private network) technology based on IP networks and in the form of "main UDP" encapsulation. VXLAN can provide two-layer interconnection for dispersed physical sites based on existing service provider or enterprise IP networks and can provide service isolation for different tenants. VXLAN is used primarily in data center networks. VXLAN has the following characteristics:

a. support a large number of tenants: by using the 24-bit identifier, at most 24 power (16777216) VXLANs (VXLANs) of 2 can be supported, so that the number of supported tenants is increased on a large scale, and the problem of insufficient resources of the traditional two-layer network VLAN is solved.

b. Easy maintenance: a large two-layer network is established based on an IP network, so that the network deployment and maintenance are easier, and the existing IP network technology can be fully utilized, such as load sharing by utilizing an equivalent route; only the edge device of the IP core network needs to carry out VXLAN processing, and the network intermediate device only needs to forward the message according to the IP header, thereby reducing the difficulty and the cost of network deployment.

The VXLAN technology takes the existing three-layer physical network as an Underlay network, and a virtual two-layer network, namely an Overlay network, is constructed on the three-layer physical network. The Overlay network realizes the transfer of the two-layer message of the tenant between different sites by crossing a three-layer network through a packaging technology and by utilizing a three-layer forwarding path provided by an Underlay network. For the tenant, the Underlay network is transparent, and different sites of the same tenant work in a local area network. A typical network model for VXLAN includes the following sections:

VM (Virtual Machine): multiple virtual machines can be created on one server, and different virtual machines can belong to different VXLANs. Virtual machines belonging to the same VXLAN are in the same logic two-layer network and are communicated with each other in two layers; two levels of isolation between virtual machines belonging to different VXLANs. VXLAN is identified by VXLAN ID, also known as VNI (VXLAN Network Identifier), which is 24 bits long.

VTEP (VXLAN Tunnel End Point ): edge device of VXLAN. The VXLAN processing is performed on the VTEP, for example, to identify the VXLAN to which the ethernet data frame belongs, to perform two-layer forwarding on the data frame based on the VXLAN, and to encapsulate/decapsulate the packet. The VTEP may be an independent physical device or a server where the virtual machine is located.

VXLAN tunnel: a point-to-point logical tunnel between two VTEPs. After encapsulating a VXLAN header, a UDP header and an IP header for a data frame, the VTEP forwards the encapsulated message to a far-end VTEP through a VXLAN tunnel, and the far-end VTEP decapsulates the encapsulated message.

Core equipment: devices in an IP core network. The core device does not participate in VXLAN processing, and only needs to forward the message in three layers according to the destination IP address of the encapsulated message.

VSI (Virtual Switch Instance): a virtual switching instance on the VTEP provides a two-layer switching service for VXLAN. The VSI can be viewed as a VXLAN-based virtual switch on a VTEP that performs layer two forwarding, with all the functions of a traditional ethernet switch, including source MAC address learning, MAC address aging, flooding, etc. VSIs correspond one-to-one to VXLANs.

AC (Attachment Circuit, access Circuit): the VTEP connects physical or virtual circuits of the local site. On a VTEP, the three-tier interface or Ethernet service instance (service instance) associated with a VSI is referred to as the AC. Wherein an ethernet service instance is created on a layer two ethernet interface that defines a series of matching rules for matching data frames received from the layer two ethernet interface. The service instance AC is configured under 1 two-layer physical port.

An EVPN (Ethernet Virtual Private Network) is a two-layer VPN technology, where a control plane uses MP-BGP (Border Gateway Protocol) to announce EVPN routing information, and a data plane uses VXLAN encapsulation to forward packets. EVPN has advantages over VXLAN:

A. the configuration is simplified: the automatic discovery of VTEP, the automatic establishment of VXLAN tunnel and the automatic association of VXLAN tunnel and VXLAN are realized through MP-BGP, the manual configuration of a user is not needed, and the difficulty of network deployment is reduced.

B. Separating the control plane from the data plane: the control plane is responsible for issuing routing information, and the data plane is responsible for forwarding messages, so that the division of labor is clear, and the management is easy.

The following describes the technical solutions of the present disclosure and how to solve the above problems in detail with specific examples. These several specific embodiments may be combined with each other below, and details of the same or similar concepts or processes may not be repeated in some embodiments.

Fig. 1 is a broadcast information sending method provided in an embodiment of the present disclosure, and is applied to a first tunnel endpoint device in a cross-device link aggregation group, where the first tunnel endpoint device has a first single-hanging device, as shown in fig. 1, the method includes:

step S101: establishing two VXLAN tunnels with the virtual IP address of the first tunnel endpoint device and the real IP address of the first tunnel endpoint device and a second tunnel endpoint device of a non-identical cross-device link aggregation group respectively;

step S102: when two VXLAN tunnels are established, respectively sending Type3 routing information to the second tunnel endpoint device, and respectively adding device identification information of a first tunnel endpoint device in the two pieces of Type3 routing information sent, so that after the second tunnel endpoint device receives the two pieces of Type3 routing information, the second tunnel endpoint device judges that two tunnels from the second tunnel endpoint device to the first tunnel endpoint device are both VXLAN tunnels which are to the same device and belong to the same VNI through the device identification information of the first tunnel endpoint device in the two pieces of Type3 routing information, and selects one of the tunnels when a head end copy list is set, so that when broadcast information is sent, information broadcasted to the first tunnel endpoint device is sent to the first tunnel endpoint device through the selected tunnel.

In the VXLAN network, after two devices VTEP1 and VTEP2 in a cross-device link aggregation mode are configured, the VTEP1 and VTEP2 can judge whether the MAC address is a single-hanging device or not through a cross-device link aggregation link identifier when synchronizing MAC table entries. Meanwhile, different addresses are used between the VTEP1 and the VTEP2 as BGP peer addresses, BGP EVPN neighbors are mutually established, the VTEP3 is equipment which does not belong to a cross-equipment link aggregation mode, and the VM1 is single-hanging equipment which is accessed to the VTEP2.

In the prior art, before communication, VTEP1 and VTEP2 establish a VXLAN tunnel with VTEP3;

VTEP1 sends Type3 routing information to VTEP3, the routing information home terminal VTEP IP address sets up as the fictitious VTEP IP address;

VTEP1 and VTEP3 establish VXLAN tunnel, the source address is the virtual VTEP IP address of VTEP1, the destination address is the IP address of VTEP3;

VTEP1 sends Type2 route MAC/IP route to VTEP3, is used for announcing the host computer MAC address connected to this apparatus, host computer ARP and host computer routing information; adding a next hop routing attribute in the routing information, and setting a next hop address as a virtual VTEP IP address;

VTEP1 does not send the Type2 routing information of VM1 to VTEP3;

VTEP2 sends Type3 routing information to VTEP3, route information this end VTEP IP address set up as fictitious VTEP IP address;

VTEP2 and VTEP3 establish VXLAN tunnel, the source address is the virtual VTEP IP address of VTEP2, the destination address is the IP address of VTEP3;

VTEP2 sends Type2 route MAC/IP route to VTEP3, is used for announcing the host computer MAC address connected to this apparatus, host computer ARP and host computer routing information; adding a next hop routing attribute in the routing information, and setting a next hop address as a virtual VTEP IP address;

VTEP2 does not send the Type2 routing information of VM1 to VTEP3;

VTEP2 sends Type3 routing information to VTEP3, the IP address of VTEP IP address of home terminal is set up as the IP address different from fictitious VTEP IP address in the routing information is called the real address;

VTEP2 and VTEP3 establish VXLAN tunnel, the source address is the real address of VTEP2, the destination address is IP address of VTEP3;

VTEP2 sends Type2 route VM1MAC route to VTEP3, used for announcing VM1 routing information; and adding a next hop routing attribute in the routing information, and setting a next hop address as a real address.

VTEP2 and VTEP3 have two VXLAN tunnels, when VTEP3 wants to send the broadcast information, will send the broadcast information to VTEP2 repeatedly through two tunnels, increase the load of the link between VTEP2 and VTEP3, consume VTEP2 and VTEP3 resources of the apparatus.

In order to solve the problem, an embodiment of the present disclosure provides a method for transmitting broadcast information in a single-hanging device cross-device link aggregation manner. After two devices VTEP1 and VTEP2 in a cross-device link aggregation mode are configured, the device firstly sends a Hello message of a cross-device link aggregation group through a peer-link. After receiving the Hello message of the opposite terminal, the device judges whether the number of the cross-device link aggregation group carried in the message is the same as that of the local terminal, and if the number of the cross-device link aggregation group of the two devices is the same, the cross-device link aggregation group of the two devices is successfully paired. After the pairing is successful, the two devices send device information messages of the cross-device link aggregation group to the opposite terminal, and the devices determine the active and standby states of the cross-device link aggregation group according to the cross-device link aggregation group priority and the system MAC address carried in the messages. After the cross-device link aggregation group negotiates a master/standby state, the two devices send information messages through the peer-link, wherein the messages carry configuration information of member interfaces. And after the synchronization of the member interface information is completed, determining the master/standby states of the member interfaces. After the main device and the standby device are negotiated, the two devices send double main detection messages according to a 15s period through the double main detection links, once the devices sense a peer-link fault, the three double main detection link messages are sent according to a 100ms period, and detection is accelerated. When both the two devices can receive the message sent by the opposite terminal, the dual-active system starts to work normally. After normal operation, the two devices send a synchronization message through a peer-link to synchronize information of the opposite end in real time, the synchronization message includes MAC table entries, ARP table entries, and the like, and sends states of member ports, and synchronizes STP (Spanning Tree Protocol), VRRP (Virtual Router Redundancy Protocol) Protocol messages. VTEP1 and VTEP2 can judge whether the MAC address is a single-hanging device through the cross-device link aggregation link identification when synchronizing MAC table items. Meanwhile, different addresses are used between VTEP1 and VTEP2 as BGP peer addresses, and BGP EVPN neighbors are mutually established.

After VTEP1 and VTEP2 establish BGP EVPN neighborhood, VTEP1 and VTEP3 establish VXLAN tunnel, while sending routing information, can carry on normally according to the existing mode;

of course, when VTEP1 sends Type3 routing information to VTEP3, an attribute may also be added in the Type3 routing information as a device identifier, and the device identifier is set as the MAC address of VTEP1 and is used to identify VTEP1.

When a first tunnel endpoint device (i.e. VTEP 2) and a second tunnel endpoint device (i.e. VTEP 3) establish a tunnel, the first tunnel endpoint device establishes VXLAN tunnel with a virtual IP (i.e. virtual VTEP IP) address of the first tunnel endpoint device and an IP real address of the first tunnel endpoint device and a second tunnel endpoint device of a non-same cross-device link aggregation group respectively, during the establishment of the two tunnels, VTEP2 sends Type3 routing information to VTEP3 respectively with different local VTEP IP addresses (virtual VTEP IP address and IP real address of VTEP 2), and adds an attribute to the Type3 routing information as device identification, i.e. adds device identification information of the first tunnel endpoint device to the Type3 routing information, wherein after the tunnel established by the IP real address of VTEP2, VTEP2 sends VM1MAC routing of the Type2 routing for notifying of the routing information, and VTEP2 and VTEP3 establish two VXLAN tunnels. When VTEP3 wants to send the broadcast information (send the information of VTEP within a certain range), VTEP3 judges two tunnels from VTEP3 to VTEP2 belong to the same VNI tunnel to the same device through the device identification information (the device identification information of VTEP 2) in the Type3 routing information, VTEP3 selects one of the tunnels when setting up the head end copy list, only send a copy of the broadcast information to VTEP2, and the broadcast information sent to other VTEPs is not changed.

According to the embodiment of the disclosure, the device identifier attribute is added to Type3 routing information, after tunnels are established with the opposite-end VTEP respectively by using the virtual IP address and the real address, the opposite-end VTEP can judge that two tunnels to the VTEP in the cross-device link aggregation group belong to a tunnel belonging to the same VNI (virtual network interface) of the same device through the device identifier attribute, and when the broadcast information is sent, one tunnel is selected for broadcast sending when the head-end copy list is set, so that the sending of repeated broadcast information can be avoided, and the loads of the tunnels and the devices can be reduced.

Further, the method further comprises:

receiving a BGP EVPN route sent by a second tunnel endpoint device, and acquiring an IP address of the second tunnel endpoint device;

When a tunnel is established between a first tunnel endpoint device (i.e., VTEP 2) and a second tunnel endpoint device (i.e., VTEP 3), a BGP EVPN peer is first established between VTEP2 and VTEP3. Then, a two-layer broadcast domain is created on VTEP2 and VTEP3, respectively, and the associated VNI is configured under the two-layer broadcast domain. And then, an EVPN instance is created in a second-layer broadcast domain, and the RD, the outgoing direction VPN-Target (ERT) and the incoming direction VPN-Target (IRT) of the local-end EVPN instance are configured. After the IP address of the local terminal VTEP is configured, the VTEP2 and the VTEP3 generate a BGP EVPN route and send the BGP EVPN route to the opposite terminal. After receiving the route, VTEP2 and VTEP3 will obtain opposite terminal VTEP IP address and VNI carried therein, if the VTEP IP address of opposite terminal is reachable by three-layer route, establish a VXLAN tunnel to the opposite terminal; meanwhile, if the VNI of the opposite end is the same as that of the local end, a head end copy table is created for forwarding subsequent BUM (broadcast or unknown unicast or multicast) messages.

The source addresses of two VXLAN tunnels established by VTEP2 and VTEP3 are respectively the virtual IP address of VTEP2 and the real address of VTEP2, and the destination addresses of the tunnels are both the IP address of VTEP3, therefore, when sending broadcast information, VTEP2 judges that two tunnels from VTEP2 to VTEP3 belong to the same VNI tunnel to the same device through opposite end IP address (IP address of VTEP 3) information in the two VXLAN tunnels, VTEP2 selects one of the tunnels when setting up the head end copy list, and only sends one piece of broadcast information to VTEP3.

After two VXLAN tunnels are established, the VTEP in the cross-device link aggregation group judges that two tunnels to the opposite end belong to the same VNI tunnel of the same device through the IP address of the opposite end, so that only one piece of broadcast information is sent to the VTEP of the opposite end, and the repeated broadcast information can be avoided being sent.

The equipment identification information is used for identifying the equipment, and the tunnel to the VTEP at the opposite end can be well determined to belong to the same VNI tunnel to the same equipment through the MAC address of the opposite end in the two tunnels, so that only one piece of broadcast information is sent.

sending first Type3 routing information to second tunnel endpoint equipment, and adding equipment identification information of the first tunnel endpoint equipment in the first Type3 routing information, wherein the home terminal IP address in the first Type3 routing information is set as the virtual IP address of the first tunnel endpoint equipment;

sending a first Type2 route to a second tunnel endpoint device through a first VXLAN tunnel so as to inform the second tunnel endpoint device of the MAC address, the ARP of the host and host route information of the host connected to the first tunnel endpoint device, wherein the host route information is added with a next hop route attribute, the next hop address is set as the virtual IP address of the first tunnel endpoint device, and the host does not comprise the first single-hanging device.

VTEP2 sends Type3 routing information to VTEP3, adds an attribute in Type3 routing information as the equipment identification, and the equipment identification sets up the MAC address as VTEP2, and the home terminal VTEP IP address sets up as virtual VTEP IP address in the routing information.

VTEP2 establishes a VXLAN tunnel with VTEP3, with the source address being the virtual VTEP IP address of VTEP2 and the destination address being the IP address of VTEP3.

The VTEP2 sends a Type2 routing MAC/IP route to a VTEP3, and the Type2 routing MAC/IP route is used for notifying a host MAC address, a host ARP and host routing information which are connected to the device, wherein the host is a non-single-hanging device, a next hop routing attribute is added in the routing information, the next hop address is set as a virtual VTEP IP address, and the Type2 routing information of the VM1 is not included in the MAC/IP route of the Type2 route.

Further, establishing a VXLAN tunnel with the IP real address of the first tunnel endpoint device and the second tunnel endpoint device of the non-identical cross-device link aggregation group includes:

sending second Type3 routing information to a second tunnel endpoint device, and adding device identification information of a first tunnel endpoint device in the second Type3 routing information, wherein the home terminal IP address in the second Type3 routing information is set as the IP real address of the first tunnel endpoint device;

VTEP2 establishes a VXLAN tunnel with VTEP3, with the source address being the real address of VTEP2 and the destination address being the IP address of VTEP3.

VTEP2 sends VM1MAC route of Type2 route to VTEP3 for advertising VM1 route information. And adding a next hop routing attribute in the routing information, and setting a next hop address as a VTEP2 real address.

VTEP2 thus establishes two VXLAN tunnels with VTEP3. For the single-hanging device and the non-single-hanging device, information transmission is carried out through two tunnels respectively, and for the broadcast information, when VTEP2 and VTEP3 transmit the broadcast information, only one of the tunnels needs to be selected, and only one piece of broadcast information is transmitted to the other side.

According to the embodiment of the disclosure, by judging that two tunnels to the VTEP in the cross-device link aggregation group belong to a tunnel belonging to the same VNI of the same device, and selecting one of the tunnels for broadcast transmission when the head end copy list is set during broadcast information transmission, the transmission of repeated broadcast information can be avoided, and the loads of the tunnels and the devices can be reduced.

Fig. 2 is a flowchart of a broadcast information sending method provided in the second embodiment of the present disclosure, which is applied to a second tunnel endpoint device, and as shown in fig. 2, the method includes:

step S201: establishing two VXLAN tunnels with the first tunnel endpoint device according to the virtual IP address of the first tunnel endpoint device and the IP real address of the first tunnel endpoint device respectively;

step S202: respectively receiving Type3 routing information sent by the first tunnel endpoint device when two VXLAN tunnels are established, wherein the first tunnel endpoint device is a device in a cross-device link aggregation group and is provided with a first single-hanging device, the second tunnel endpoint device and the first tunnel endpoint device are devices in a non-identical cross-device link aggregation group, and the two pieces of Type3 routing information which are respectively received both comprise device identification information of the first tunnel endpoint device;

step S203: after receiving the two types of Type3 routing information, judging that the two tunnels from the second tunnel endpoint device to the first tunnel endpoint device are VXLAN tunnels which are to the same device and belong to the same VNI through the device identification information of the first tunnel endpoint device in the two types of Type3 routing information, and selecting one of the tunnels when setting a head end copy list, so that information broadcasted to the first tunnel endpoint device is only transmitted to the first tunnel endpoint device through the selected tunnel when transmitting broadcast information.

When a second tunnel endpoint device (VTEP 3) of the non-cross device link aggregation group establishes two tunnels with a VTEP2 in the cross device link aggregation group with a single-hanging device, it can be determined that the two tunnels belong to a VXLAN tunnel belonging to the same VNI to the same device through device identification information (such as a MAC address of the VTEP 2) of the first tunnel endpoint device in Type3 routing information received respectively, and therefore, only one piece of broadcast information is sent to the VTEP2 when sending the broadcast information.

Further, the method further comprises:

When two tunnels are established with a second tunnel endpoint device (VTEP 3) of a non-cross device link aggregation group, the source addresses are the virtual VTEP IP address of the VTEP2 and the real address of the VTEP2, and the destination addresses are the IP addresses of the VTEP3. Therefore, two tunnels from the VTEP2 to the VTEP3 can be judged to belong to the same VNI of the same device through the opposite terminal IP address information in the Type3 routing information, and therefore, only one piece of broadcast information is sent to the VTEP3 when the broadcast information is sent.

The embodiments of the present disclosure correspond to the embodiments, and specific reference may be made to the relevant description in the first embodiment.

The device in the cross-device link aggregation mode according to the embodiment of the present disclosure adds the device identifier attribute to the Type3 routing information, does not broadcast the route information of the single-hanging device that is not connected to the device, establishes a tunnel with other VTEPs by using a real address, and broadcasts the route of the single-hanging device, where the route information is added with the next hop route attribute, and the next hop address is set as the real address. Other VTEP and devices in a cross-device link aggregation mode can determine that two tunnels belong to the same VNI as the same device through the device identifier attribute and the peer IP address, and only one of the tunnels is selected when the head-end copy list is set. Repeated broadcast information can be prevented from being transmitted, and the load of the tunnel and the equipment is reduced.

In order to describe the technical solution of the present disclosure more clearly and completely, a third embodiment of the present disclosure further provides a broadcast information sending method, where VTEP1 and VTEP2 form a cross-device link aggregation mode, VM1 is connected to VTEP2 in a single-hanging manner, VTEP3 does not join in the cross-device link aggregation mode, VTEP1 and VTEP2 use different addresses as BGP peer addresses, and BGP EVPN neighbors are established with each other, as shown in fig. 3, the method includes:

s1: VTEP1 sends Type3 routing information to VTEP3, add an attribute in Type3 routing information as the equipment label, the equipment label sets up as the MAC address of VTEP1, route information home end VTEP IP address sets up as the virtual VTEP IP address;

s2: VTEP1 and VTEP3 establish VXLAN tunnel, the source address is the virtual VTEP IP address of VTEP1, the destination address is the IP address of VTEP3;

s3: VTEP1 sends Type2 route MAC/IP route to VTEP3, is used for announcing and connecting to host computer MAC address, host computer ARP and host computer routing information of this apparatus; adding a next hop routing attribute in the routing information, and setting a next hop address as a virtual VTEP IP address; furthermore, VTEP1 does not send the Type2 routing information of VM1 to VTEP3;

s4: VTEP2 sends Type3 routing information to VTEP3, add an attribute in Type3 routing information as the equipment label, the equipment label sets up as the MAC address of VTEP2, the home terminal VTEP IP address sets up as the virtual VTEP IP address in the routing information;

s5: the VTEP2 and the VTEP3 establish a VXLAN tunnel, the source address is the virtual VTEP IP address of the VTEP2, and the destination address is the IP address of the VTEP3;

s6: VTEP2 sends Type2 route MAC/IP route to VTEP3, is used for announcing and connecting to the host machine MAC address, host machine ARP and host machine routing information of this apparatus. Adding a next hop routing attribute in the routing information, and setting a next hop address as a virtual VTEP IP address; VTEP2 does not send the Type2 routing information of VM1 to VTEP3;

s7: VTEP2 sends Type3 routing information to VTEP3, add an attribute in Type3 routing information as the apparatus identification, the apparatus identification sets up as the MAC address of VTEP2, the IP address of home terminal VTEP in the routing information is set up as the IP address different from fictitious VTEP IP address to call as the real address;

s8: VTEP2 and VTEP3 establish VXLAN tunnel, the source address is the real address of VTEP2, the destination address is IP address of VTEP3;

s9: VTEP2 sends Type2 route VM1MAC route to VTEP3, used for announcing VM1 routing information; adding a next hop routing attribute in the routing information, and setting a next hop address as a real address;

s10: the VTEP2 judges that two tunnels from the VTEP2 to the VTEP3 belong to the same VNI of the same equipment through opposite end IP address information in the Type3 routing information, the VTEP2 selects one tunnel when setting a head end copy list, and only one piece of broadcast information is sent to the VTEP3 when the broadcast information is sent;

s11: the VTEP3 judges that two tunnels from the VTEP3 to the VTEP2 belong to the same VNI of the same device through the device identification information in the Type3 routing information, the VTEP3 selects one of the tunnels when setting a head end copy list, and only one piece of broadcast information is sent to the VTEP2 when the broadcast information is sent.

In the embodiment of the present disclosure, VTEP2 and VTEP3 respectively determine, through device identification information in the peer IP address information and the routing information, that the two tunnels belong to the same VNI to the same device, select one of the tunnels when setting the head end copy list, and send only one piece of broadcast information to the peer when sending the broadcast information. And repeated broadcast information is prevented from being sent, and the load of the tunnel and the equipment is reduced.

Fig. 4 is an architecture diagram of a first tunnel endpoint device provided in a fourth embodiment of the present disclosure, where the first tunnel endpoint device is a device in a cross-device link aggregation group, and the first tunnel endpoint device has a first single-hanging device, as shown in fig. 4, the first tunnel endpoint device includes:

a first tunnel establishing module 11, configured to establish two virtual extensible local area network VXLAN tunnels with a virtual IP address of a first tunnel endpoint device, an IP real address of the first tunnel endpoint device, and a second tunnel endpoint device of a non-identical cross-device link aggregation group, respectively;

a first sending module 12, configured to send Type3 routing information to the second tunnel endpoint device respectively when two VXLAN tunnels are established, and add device identification information of the first tunnel endpoint device in the two sent Type3 routing information respectively, so that after the second tunnel endpoint device receives the two types 3 routing information, it determines, through the device identification information of the first tunnel endpoint device in the two types 3 routing information, that two tunnels from the second tunnel endpoint device to the first tunnel endpoint device are VXLAN tunnels which are to the same device and belong to the same virtual extensible local area network identifier VNI, and selects one of the tunnels when a head-end copy list is set, so that when broadcast information is sent, information broadcasted to the first tunnel endpoint device is sent to the first tunnel endpoint device only through the selected tunnel.

Further, the first tunnel endpoint device further includes a first receiving module 13 and a first broadcasting module 14;

the first receiving module 13 is configured to receive a BGP EVPN route sent by the second tunnel endpoint device to obtain an IP address of the second tunnel endpoint device;

the first tunnel establishing module 11 is specifically configured to establish a tunnel by using the IP address of the second tunnel endpoint device as the IP address of the opposite end when two VXLAN tunnels are respectively established;

the first broadcast module 14 is configured to determine, through the two VXLAN tunnels, that both the peer IP addresses are the IP address of the second tunnel endpoint device, that the two tunnels from the first tunnel endpoint device to the second tunnel endpoint device belong to the VXLAN tunnel belonging to the same VNI as the same device, and select one of the tunnels when setting the head end copy list, so that when sending broadcast information, information broadcasted to the second tunnel endpoint device is sent to the second tunnel endpoint device only through the selected tunnel.

Further, the device identification information of the first tunnel endpoint device is a media access control MAC address of the first tunnel endpoint device.

Further, the first tunnel establishing module 11 includes:

a first sending unit, configured to send first Type3 routing information to a second tunnel endpoint device, and add device identification information of the first tunnel endpoint device in the first Type3 routing information, where a home IP address in the first Type3 routing information is set as a virtual IP address of the first tunnel endpoint device;

a first establishing unit, configured to establish a first VXLAN tunnel with a second tunnel endpoint device through the first Type3 routing information, where a source address of the tunnel is a virtual IP address of the first tunnel endpoint device, and a destination address is an IP address of the second tunnel endpoint device;

the first sending unit is further configured to send a first Type2 route to the second tunnel endpoint device through the first VXLAN tunnel, so as to notify the second tunnel endpoint device of an MAC address of a host connected to the first tunnel endpoint device, a host address resolution protocol ARP, and host route information, where a next hop route attribute is added to the host route information, the next hop address is set as a virtual IP address of the first tunnel endpoint device, and the host does not include the first single-hanging device.

Further, the first tunnel establishing module 11 includes:

a second sending unit, configured to send second Type3 routing information to a second tunnel endpoint device, and add device identification information of the first tunnel endpoint device in the second Type3 routing information, where an IP address of a home terminal in the second Type3 routing information is set as an IP real address of the first tunnel endpoint device;

a second establishing unit, configured to establish a second VXLAN tunnel with a second tunnel endpoint device through the second Type3 routing information, where a source address of the tunnel is an IP real address of the first tunnel endpoint device, and a destination address is an IP address of the second tunnel endpoint device;

the second sending unit is further configured to send a second Type2 route to the second tunnel endpoint device through the second VXLAN tunnel, where the second Type2 route includes an MAC route of the first single-hanging device, so as to notify the second tunnel endpoint device of the routing information of the first single-hanging device, where a next hop routing attribute is added, and a next hop address is set as an IP real address of the first tunnel endpoint device.

Fig. 5 is an architecture diagram of a second tunnel endpoint device according to a fifth embodiment of the present disclosure, and as shown in fig. 5, the second tunnel endpoint device includes:

a second tunnel establishing module 21, configured to establish two VXLAN tunnels with the first tunnel endpoint device according to the virtual IP address of the first tunnel endpoint device and the IP real address of the first tunnel endpoint device, respectively;

a second receiving module 22, configured to receive Type3 routing information sent by the first tunnel endpoint device when two VXLAN tunnels are established, where the first tunnel endpoint device is a device in a cross-device link aggregation group and has a first single-hanging device, and the second tunnel endpoint device and the first tunnel endpoint device are devices in a non-same cross-device link aggregation group, and both the two pieces of Type3 routing information received respectively include device identification information of the first tunnel endpoint device;

the second broadcast module 23 is configured to, after receiving the two types of Type3 routing information, determine, through the device identification information of the first tunnel endpoint device in the two types of Type3 routing information, that both of the two tunnels from the second tunnel endpoint device to the first tunnel endpoint device are VXLAN tunnels which are to the same device and belong to the same VNI, and select one of the tunnels when setting the head end copy list, so that when sending broadcast information, the information broadcasted to the first tunnel endpoint device is sent to the first tunnel endpoint device only through the selected tunnel.

Further, the second tunnel endpoint device further includes a second sending module 24;

the second sending module 24 is configured to send a BGP EVPN route to the first tunnel endpoint device, so that the first tunnel endpoint device obtains an IP address of the second tunnel endpoint device, establishes tunnels with the IP address of the second tunnel endpoint device as an opposite-end IP address when two VXLAN tunnels are respectively established, determines that two tunnels from the first tunnel endpoint device to the second tunnel endpoint device belong to the same VNI as the same device by using both the opposite-end IP addresses of the two VXLAN tunnels as the IP address of the second tunnel endpoint device, and selects one of the tunnels when a head-end copy list is set, so that when broadcast information is sent, information broadcasted to the second tunnel endpoint device is sent to the second tunnel endpoint device only through the selected tunnel.

The first tunnel endpoint device and the second tunnel endpoint device in the embodiments of the present disclosure are used to implement the broadcast information sending methods in the first to third embodiments of the methods, so the description is relatively simple, and reference may be specifically made to the related descriptions in the first to third embodiments of the methods, and details are not repeated here.

Furthermore, as shown in fig. 6, a sixth embodiment of the present disclosure further provides an electronic device, which includes a memory 10 and a processor 20, where the memory 10 stores a computer program, and when the processor 20 runs the computer program stored in the memory 10, the processor 20 executes the above-mentioned various possible methods.

The memory 10 is connected to the processor 20, the memory 10 may be a flash memory, a read-only memory or other memories, and the processor 20 may be a central processing unit or a single chip microcomputer.

Furthermore, the disclosed embodiments also provide a computer-readable storage medium, on which a computer program is stored, the computer program being executed by a processor to perform the above-mentioned various possible methods.

The computer-readable storage media includes volatile or nonvolatile, removable or non-removable media implemented in any method or technology for storage of information such as computer-readable instructions, data structures, computer program modules or other data. Computer-readable storage media include, but are not limited to, RAM (Random Access Memory), ROM (Read-Only Memory), EEPROM (Electrically Erasable Programmable Read-Only Memory), flash Memory or other Memory technology, CD-ROM (Compact disk Read-Only Memory), digital versatile disks (DVD, digital Video disks) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the desired information and which can be accessed by a computer.

It is to be understood that the above embodiments are merely exemplary embodiments that are employed to illustrate the principles of the present disclosure, and that the present disclosure is not limited thereto. It will be apparent to those skilled in the art that various changes and modifications can be made therein without departing from the spirit and scope of the disclosure, and these are to be considered as the scope of the disclosure.

Claims

1. A broadcast information sending method is applied to a first tunnel endpoint device in a cross-device link aggregation group, wherein the first tunnel endpoint device is provided with a first single-hanging device, and the method comprises the following steps:

establishing two virtual extensible local area network VXLAN tunnels by using the virtual IP address of the first tunnel endpoint equipment, the IP real address of the first tunnel endpoint equipment and the second tunnel endpoint equipment of the non-identical cross-device link aggregation group respectively;

when two VXLAN tunnels are established, respectively sending Type3 routing information to the second tunnel endpoint device, and respectively adding device identification information of a first tunnel endpoint device in the two pieces of Type3 routing information sent, so that after the second tunnel endpoint device receives the two pieces of Type3 routing information, the second tunnel endpoint device judges that two tunnels from the second tunnel endpoint device to the first tunnel endpoint device are both VXLAN tunnels which are to the same device and belong to the same virtual extensible local area network identifier VNI through the device identification information of the first tunnel endpoint device in the two pieces of Type3 routing information, and selects one of the tunnels when a head end copy list is set, so that when broadcast information is sent, the information broadcast to the first tunnel endpoint device is sent to the first tunnel endpoint device through the selected tunnel.

2. The method for transmitting broadcast information according to claim 1, further comprising:

3. The method of claim 1, wherein the device identification information of the first tunnel endpoint device is a Media Access Control (MAC) address of the first tunnel endpoint device.

4. The method for sending broadcast information according to claim 1 or 2, wherein the step of establishing a VXLAN tunnel with the virtual IP address of the first tunnel endpoint device and the second tunnel endpoint device of the non-identical cross-device link aggregation group comprises:

sending a first Type2 route to a second tunnel endpoint device through a first VXLAN tunnel so as to inform the second tunnel endpoint device of the MAC address, the ARP (host Address resolution protocol) and host route information of a host connected to the first tunnel endpoint device, wherein the host route information is added with a next hop route attribute, the next hop address is set as a virtual IP address of the first tunnel endpoint device, and the host does not comprise the first single-hanging device.

5. The method according to claim 1 or 2, wherein the step of establishing a VXLAN tunnel with the IP real address of the first tunnel endpoint device and the second tunnel endpoint device of the non-identical cross-device link aggregation group comprises:

6. A broadcast information transmitting method, applied to a second tunnel endpoint device, the method comprising:

7. The method of claim 6, wherein the method further comprises:

8. A first tunnel endpoint device, wherein the first tunnel endpoint device is a device in a cross-device link aggregation group, wherein the first tunnel endpoint device has a first single-hanging device, and wherein the first tunnel endpoint device comprises:

a first tunnel establishing module, configured to establish two virtual extensible local area network VXLAN tunnels with a virtual IP address of a first tunnel endpoint device, an IP real address of the first tunnel endpoint device, and a second tunnel endpoint device of a non-identical cross-device link aggregation group, respectively;

9. A second tunnel endpoint device, the second tunnel endpoint device comprising:

a second receiving module, configured to receive Type3 routing information sent by the first tunnel endpoint device when two VXLAN tunnels are established, where the first tunnel endpoint device is a device in a cross-device link aggregation group and has a first single-hanging device, and the second tunnel endpoint device and the first tunnel endpoint device are devices in a non-same cross-device link aggregation group, and both the two pieces of Type3 routing information received respectively include device identification information of the first tunnel endpoint device;

10. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program stored by the memory, the processor performs the broadcast information transmitting method according to any one of claims 1 to 5 or any one of claims 6 to 7.

11. A computer-readable storage medium, characterized in that a computer program is stored thereon, which computer program, when being executed by a processor, implements the broadcast information transmitting method according to any one of claims 1-5 or any one of claims 6-7.