Skip to content
Apr 24 / Greg

Using BGP For QoS – MUM 2015

This presentation is an example of using BGP for your QoS policies.

The slides are available here: BGP For QoS Slides (1844 downloads)
The configs are available here: BGP For QoS (1589 downloads)
The video is here:

You can use route filters to mark incoming BGP routes. In this case we use route comments. This allows you to also manually add your own entries in with the correct route comment.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

#configure our route filters for twitch, vudu, and our open IX
/routing filter
#Here we use regex matching inside of the AS Path attribute.  We are finding out 
#where the traffic is sourced from, then adding a route comment based on that.
add bgp-as-path="^.*,46489\$" chain=bgp-qos-in comment=Twitch \
    set-route-comment=RCstreamingvideo
add bgp-as-path="^.*,40582\$" chain=bgp-qos-in comment=Vudu set-route-comment=\
    RCstreamingvideo
#Here we are using community strings to mark these routes.  On top of that we are
#also setting the BGP LP above the default of 100 so it will be the preferred route.
add bgp-communities=65101:10 chain=bgp-qos-in comment="OIX set LP 110" \
    set-bgp-local-pref=110 set-route-comment=RCoix
#This is a catch all rule you can enable.  This rule will remove all route comments
#from above.
add chain=bgp-qos-in comment="*clear all*" disabled=yes set-route-comment=""

You will then need to add these to your BGP peer incoming.

1
2
3

/routing bgp peer
add in-filter=bgp-qos-in name=peer-ISP out-filter=bgp-out remote-address=\
    x.x.x.x remote-as=x ttl=default

*Remember that when you apply a filter to a peer, it resets the peer completely.
**Remember that when you adjust these lists, all of your routes from this peer become momentarily disabled while they run through the adjusted filter.

Once this is done we have a script that searches out these specially commented routes. In this case, the route comments must start with “RC”, as in “RCstreamingvideo”. Once it finds a matching entry, it creates an address-list entry using that route’s destination address and naming it whatever the comment was. If we have a route to 10.10.10.0/24 that has a route comment of “RCoffnet”, then it will add an address-list entry named RCoffnet with address 10.10.10.0/24. Before adding the entry, the script will check that this entry doesn’t currently exist, if it does it will remove, then re-add it. All of these entries are set to expire after 24.5 hours.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43

# create the script
/system script
add name=bgp-qos policy=\
    ftp,reboot,read,write,policy,test,password,sniff,sensitive source=":log info\
    \_\"BGP QoS script start\";\r\
    \n#Define Local Var and load data\r\
    \n#loop variables\r\
    \n:local i 0;\r\
    \n#route ip address\r\
    \n:local ipAddress;\r\
    \n#is it marked for us\r\
    \n:local routeMark \"null\";\r\
    \n#route comment\r\
    \n:local routeComment \"null\";\r\
    \n#check the beginning of our routeComment\r\
    \n:local listName \"null\";\r\
    \n\r\
    \n#loop to check the entire routing table\r\
    \n:foreach i in=[/ip rou find] do={\r\
    \n  #grab the route's comment\r\
    \n  :set routeComment [/ip route get \$i comment]\r\
    \n  #check if to make sure the route comment isn't null\r\
    \n  :if (\$routeComment!=\"\") do={\r\
    \n    #grab the first two letters off of the route comment\r\
    \n    set listName [:pick \$routeComment 0 2]\r\
    \n    #make sure the first two letters are RC\r\
    \n    :if (\$listName=\"RC\") do={\r\
    \n      #get the IP address of the route\r\
    \n      :set ipAddress [/ip route get \$i dst-address]\r\
    \n      #log debug info to the log\r\
    \n#      :log info \"\$i - \$routeSize - \$routeMark - \$routeComment - \$li\
    stName - \$ipAddress\";\r\
    \n        #if it is the default gateway don't add it, otherwise add it to th\
    e addresslist for 24.5 hours\r\
    \n          :if (\$ipAddress!=0.0.0.0/0) do={\r\
    \n          /ip firewall address-list rem [find where list=\$routeComment ad\
    dress=\$ipAddress];\r\
    \n          /ip firewall address-list add list=\$routeComment address=\$ipAd\
    dress timeout=88200;}\r\
    \n    }\r\
    \n  }\r\
    \n}\r\
    \n:log info \"BGP QoS script complete\";"

Here’s the script in a human readable form, just to make life a little easier 🙂

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35

:log info "BGP QoS script start";
#Define Local Var and load data
#loop variables
:local i 0;
#route ip address
:local ipAddress;
#is it marked for us
:local routeMark "null";
#route comment
:local routeComment "null";
#check the beginning of our routeComment
:local listName "null";
 
#loop to check the entire routing table
:foreach i in=[/ip rou find] do={
  #grab the route's comment
  :set routeComment [/ip route get $i comment]
  #check if to make sure the route comment isn't null
  :if ($routeComment!="") do={
    #grab the first two letters off of the route comment
    set listName [:pick $routeComment 0 2]
    #make sure the first two letters are RC
    :if ($listName="RC") do={
      #get the IP address of the route
      :set ipAddress [/ip route get $i dst-address]
      #log debug info to the log
#      :log info "$i - $routeSize - $routeMark - $routeComment - $listName - $ipAddress";
        #if it is the default gateway don't add it, otherwise add it to the addresslist for 24.5 hours
          :if ($ipAddress!=0.0.0.0/0) do={
          /ip firewall address-list rem [find where list=$routeComment address=$ipAddress];
          /ip firewall address-list add list=$routeComment address=$ipAddress timeout=88200;}
    }
  }
}
:log info "BGP QoS script complete";

We then schedule the script to run every 24 hours. In this way, if new entries are advertised from our peers, we will pick them up every 24 hours. If they stop advertising a route, it will timeout and be removed after 24.5 hours. If the entry still exists, it will be refreshed for another day. This prevents us from having any gaps without our specialized routing.
**Be sure to schedule this in non-peak times; the script can be CPU intensive.

1
2
3
4
5

# schedule our script
/system scheduler
add interval=1d name=bgp-qos on-event=bgp-qos policy=\
    ftp,reboot,read,write,policy,test,password,sniff,sensitive start-date=\
    apr/22/2015 start-time=04:00:00

Now that we have address-lists created, we can use these in our traditional QoS policies.

First we create mangle rules to mark traffic based on these address-lists.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25

# our mangle rules to mark OIX connections, then packets
/ip firewall mangle
add action=mark-connection chain=prerouting comment="oix con mark" \
    dst-address-list=RCoix new-connection-mark=oix
add action=mark-packet chain=prerouting comment="oix-in packet mark" \
    connection-mark=oix in-interface=gre-oix new-packet-mark=oix-in \
    passthrough=no
add action=mark-packet chain=prerouting comment="oix-out packet mark" \
    connection-mark=oix new-packet-mark=oix-out passthrough=no
# our mangle rules to mark streaming video based on address lists
# ######streaming mark for twitch
add action=mark-connection chain=prerouting comment=\
    "streaming video connection mark BGP" dst-address-list=RCstreamingvideo \
    new-connection-mark=streaming-video protocol=tcp src-address-list=\
    internal-nets
add action=mark-connection chain=prerouting comment=\
    "streaming video connection mark L7" connection-mark=no-mark dst-port=80 \
    layer7-protocol=video new-connection-mark=streaming-video protocol=tcp \
    src-address-list=internal-nets
add action=mark-packet chain=prerouting comment=\
    "streaming video in packet mark" connection-mark=streaming-video \
    in-interface=ether3 new-packet-mark=streaming-video-in passthrough=no
add action=mark-packet chain=prerouting comment=\
    "streaming video out packet mark" connection-mark=streaming-video \
    new-packet-mark=streaming-video-out passthrough=no

Next we create or utilize existing queue tree entries to act upon those marked packets.

1
2
3
4
5
6
7
8
9
10
11

# #####the open exchange queue that is outside other queues
# ##streaming video inside the 10mb queue
/queue tree
add max-limit=10M name=in parent=global
add max-limit=10M name=out parent=global
add name=oix-in packet-mark=oix-in parent=global queue=default
add name=oix-out packet-mark=oix-out parent=global queue=default
add limit-at=4M max-limit=10M name=streaming-video-in packet-mark=streaming-video-in parent=in priority=3 \
    queue=default
add limit-at=4M max-limit=10M name=streaming-video-out packet-mark=streaming-video-out parent=out \
    priority=3 queue=default

Here’s all of the CLI’s in one block:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108

# #####the open exchange queue that is outside other queues
# ##streaming video inside the 10mb queue
/queue tree
add max-limit=10M name=in parent=global
add max-limit=10M name=out parent=global
add name=oix-in packet-mark=oix-in parent=global queue=default
add name=oix-out packet-mark=oix-out parent=global queue=default
add limit-at=4M max-limit=10M name=streaming-video-in packet-mark=streaming-video-in parent=in priority=3 \
    queue=default
add limit-at=4M max-limit=10M name=streaming-video-out packet-mark=streaming-video-out parent=out \
    priority=3 queue=default
 
# setting up this local router's BGP instance - don't use outside of lab
/routing bgp instance
set default as=65100
 
# our mangle rules to mark OIX connections, then packets
/ip firewall mangle
add action=mark-connection chain=prerouting comment="oix con mark" \
    dst-address-list=RCoix new-connection-mark=oix
add action=mark-packet chain=prerouting comment="oix-in packet mark" \
    connection-mark=oix in-interface=gre-oix new-packet-mark=oix-in \
    passthrough=no
add action=mark-packet chain=prerouting comment="oix-out packet mark" \
    connection-mark=oix new-packet-mark=oix-out passthrough=no
# our mangle rules to mark streaming video based on address lists
# ######streaming mark for twitch
add action=mark-connection chain=prerouting comment=\
    "streaming video connection mark BGP" dst-address-list=RCstreamingvideo \
    new-connection-mark=streaming-video protocol=tcp src-address-list=\
    internal-nets
add action=mark-connection chain=prerouting comment=\
    "streaming video connection mark L7" connection-mark=no-mark dst-port=80 \
    layer7-protocol=video new-connection-mark=streaming-video protocol=tcp \
    src-address-list=internal-nets
add action=mark-packet chain=prerouting comment=\
    "streaming video in packet mark" connection-mark=streaming-video \
    in-interface=ether3 new-packet-mark=streaming-video-in passthrough=no
add action=mark-packet chain=prerouting comment=\
    "streaming video out packet mark" connection-mark=streaming-video \
    new-packet-mark=streaming-video-out passthrough=no
 
#setting up our peers - particularly our route filters
/routing bgp peer
add in-filter=bgp-qos-in name=peer-ISP out-filter=bgp-out remote-address=\
    x.x.x.x remote-as=x ttl=default
add in-filter=bgp-qos-in name=peer-OIX out-filter=bgp-out remote-address=\
    172.17.1.2 remote-as=65101 ttl=default
 
#configure our route filters for twitch, vudu, and our open IX
/routing filter
add bgp-as-path="^.*,46489\$" chain=bgp-qos-in comment=Twitch \
    set-route-comment=RCstreamingvideo
add bgp-as-path="^.*,40582\$" chain=bgp-qos-in comment=Vudu set-route-comment=\
    RCstreamingvideo
add bgp-communities=65101:10 chain=bgp-qos-in comment="OIX set LP 110" \
    set-bgp-local-pref=110 set-route-comment=RCoix
add chain=bgp-qos-in comment="*clear all*" disabled=yes set-route-comment=""
 
# schedule our script
/system scheduler
add interval=1d name=bgp-qos on-event=bgp-qos policy=\
    ftp,reboot,read,write,policy,test,password,sniff,sensitive start-date=\
    apr/22/2015 start-time=04:00:00
 
# create the script
/system script
add name=bgp-qos policy=\
    ftp,reboot,read,write,policy,test,password,sniff,sensitive source=":log info\
    \_\"BGP QoS script start\";\r\
    \n#Define Local Var and load data\r\
    \n#loop variables\r\
    \n:local i 0;\r\
    \n#route ip address\r\
    \n:local ipAddress;\r\
    \n#is it marked for us\r\
    \n:local routeMark \"null\";\r\
    \n#route comment\r\
    \n:local routeComment \"null\";\r\
    \n#check the beginning of our routeComment\r\
    \n:local listName \"null\";\r\
    \n\r\
    \n#loop to check the entire routing table\r\
    \n:foreach i in=[/ip rou find] do={\r\
    \n  #grab the route's comment\r\
    \n  :set routeComment [/ip route get \$i comment]\r\
    \n  #check if to make sure the route comment isn't null\r\
    \n  :if (\$routeComment!=\"\") do={\r\
    \n    #grab the first two letters off of the route comment\r\
    \n    set listName [:pick \$routeComment 0 2]\r\
    \n    #make sure the first two letters are RC\r\
    \n    :if (\$listName=\"RC\") do={\r\
    \n      #get the IP address of the route\r\
    \n      :set ipAddress [/ip route get \$i dst-address]\r\
    \n      #log debug info to the log\r\
    \n#      :log info \"\$i - \$routeSize - \$routeMark - \$routeComment - \$li\
    stName - \$ipAddress\";\r\
    \n        #if it is the default gateway don't add it, otherwise add it to th\
    e addresslist for 24.5 hours\r\
    \n          :if (\$ipAddress!=0.0.0.0/0) do={\r\
    \n          /ip firewall address-list rem [find where list=\$routeComment ad\
    dress=\$ipAddress];\r\
    \n          /ip firewall address-list add list=\$routeComment address=\$ipAd\
    dress timeout=88200;}\r\
    \n    }\r\
    \n  }\r\
    \n}\r\
    \n:log info \"BGP QoS script complete\";"

When I recorded the video I hadn’t eaten in 12 hours, so I might be just a tad bit loopy…hehe
Please let me know in the comments what you think about life and the pursuit of happiness. Thanks and happy routing.

Filed under Mikrotik, Networking, Script
Leave a Comment

 

CAPTCHA

*