Tech Journal Back to Tech Journal

Setting up QoS (to prioritize various types of communications)

After setting up a Linux box as a NAT proxy, I've found that various family members were complaining that they couldn't surf the Internet very smoothly. This was due to bandwidth limitations. My own download programs were using up more of the available bandwidth than would allow for a smooth browsing experience in the remaining bandwidth.

I wanted to have an automated solution, so that even if I weren't home, my own use of the Internet wouldn't impair the others' use of it. The solution? QoS!

QoS as supported by the the linux kernel can be managed by a tool called tc, to create a prioritized tree to classify all the packets into. The theory is the tricky bit, so here goes. We have some kind of pipe to the Internet. Say it's got a max upload speed of 9KB, and down of 150KB. We want to split that among all the types of communication going across that pipe, in a fair manner. I'm not saying that TCP/IP doesn't attempt to be fair, but its main purpose is to get as much data across a line as possible. That means that if you have a very long download going at a nice speed (i.e. with a big transmission window), a new client trying to transfer data across the same pipe will find itself queuing behind the packets of the larger transfer. This will cause the window for the new client to stay small, and the xfer speed low.

In my network, I have several types of transfers:

So what I want to do, is allocate most the of available bandwidth to low latency (Telnet and VoIP), the remaining to Web-surfing, and any that's left to other stuff (including Bittorrent and KazaA). This will result in the mass-transfer programs (like Bittorrent) to take up all the bandwidth when nobody else is around. Perfect. (But this doesn't really work too well. I've found that using the SpeedScheduler plugin for Azureus is a good idea. Limit it to 3 up and 70 down, and it's all smooth.)

                           -----
                           |1:0|
                           -----
                             |
                           -----
                           |1:1|
                           -----
                             |
                   ------+---+---+------
                   |     |       |     |
                ------ ------ ------ ------
                |1:10| |1:15| |1:20| |1:30|
                ------ ------ ------ ------

    

1:0 is attached to the PPP device (ppp0, ppp1, etc.), with 1:20 set as the default child. Then, 1:1 is connected to as a child to 1:0, and defined to be limited to 70Kbit upload (I only want to use 2/3 of the upstream bandwidth).

Then, I define 1:10 for low-latency (Telnet, and other TOS Min Delay), 1:15 for 2nd best min latencey (Web browsing), 1:20 is the default 'bulk' and also where all unknown traffic ends up in. Finally, 1:30 is the low-priority 'bulk' traffic - anything marked TOS Min Cost (which I made sure Azureus marked its packets as such.)

This is all done with this script

#!/bin/bash

# The Ultimate Setup For Your Internet Connection At Home
# [from http://lartc.org/howto/lartc.cookbook.ultimate-tc.html]
# 
#
# Set the following values to somewhat less than your actual download
# and uplink speed. In kilobits
DOWNLINK=1200   # it's supposed to be 1.5mbit, but that doesn't work at all...
UPLINK=70       # really the max is 96kbps, but to give some leeway
DEV=ppp0

if [ ! -e /proc/sys/net/ipv4/conf/$DEV ]; then
        DEV=ppp1
fi
if [ ! -e /proc/sys/net/ipv4/conf/$DEV ]; then
        echo "Error, couldn't find either ppp0 or ppp1!"
        exit
fi

# clean existing down- and uplink qdiscs, hide errors
tc qdisc del dev $DEV root    2> /dev/null > /dev/null
tc qdisc del dev $DEV ingress 2> /dev/null > /dev/null

###### uplink

# install root HTB, point default traffic to 1:20:

tc qdisc add dev $DEV root handle 1: htb default 20

# shape everything at $UPLINK speed - this prevents huge queues in your
# DSL modem which destroy latency:
tc class add dev $DEV parent 1: classid 1:1 htb rate ${UPLINK}kbit burst 6k

# high prio class 1:10:
tc class add dev $DEV parent 1:0 classid 1:10 htb rate ${UPLINK}kbit \
   burst 5k prio 1

# web browsing class 1:15 - 2nd best latency
tc class add dev $DEV parent 1:0 classid 1:15 htb rate ${UPLINK}kbit \
   burst 6k prio 2

# bulk & default class 1:20 - gets slightly less traffic, 
# and a lower priority:
tc class add dev $DEV parent 1:0 classid 1:20 htb rate $[8*$UPLINK/10]kbit \
   burst 15k prio 3

# low priority bulk 1:30 - anything set a "lowest monetary cost"
tc class add dev $DEV parent 1:0 classid 1:30 htb rate $[7*$UPLINK/10]kbit \
   burst 6k prio 4

# all get Stochastic Fairness:
tc qdisc add dev $DEV parent 1:10 handle 10: sfq perturb 10
tc qdisc add dev $DEV parent 1:15 handle 15: sfq perturb 10
tc qdisc add dev $DEV parent 1:20 handle 20: sfq perturb 10
tc qdisc add dev $DEV parent 1:30 handle 30: sfq perturb 10

# TOS Minimum Delay (ssh, NOT scp) in 1:10:
tc filter add dev $DEV parent 1:0 protocol ip prio 10 u32 \
      match ip tos 0x10 0xff  flowid 1:10

# outgoing telnet packets in 1:10:
tc filter add dev $DEV parent 1:0 protocol ip prio 10 u32 \
      match ip sport 23 0xffff flowid 1:10
# incoming telnet packets in 1:10:
tc filter add dev $DEV parent 1:0 protocol ip prio 10 u32 \
      match ip dport 23 0xffff flowid 1:10

# skype in 1:10 - port 14067 for Shalom
tc filter add dev $DEV parent 1:0 protocol ip prio 10 u32 \
      match ip sport 14067 0xffff flowid 1:10
tc filter add dev $DEV parent 1:0 protocol ip prio 10 u32 \
      match ip dport 14067 0xffff flowid 1:10
# skype in 1:10 - port 14068 for kids
tc filter add dev $DEV parent 1:0 protocol ip prio 10 u32 \
      match ip sport 14069 0xffff flowid 1:10
tc filter add dev $DEV parent 1:0 protocol ip prio 10 u32 \
      match ip dport 14068 0xffff flowid 1:10

# ICMP (ip protocol 1) in the interactive class 1:10 so we 
# can do measurements & impress our friends:
tc filter add dev $DEV parent 1:0 protocol ip prio 10 u32 \
        match ip protocol 1 0xff flowid 1:10

# To speed up downloads while an upload is going on, put ACK packets in
# the interactive class:

tc filter add dev $DEV parent 1: protocol ip prio 10 u32 \
   match ip protocol 6 0xff \
   match u8 0x05 0x0f at 0 \
   match u16 0x0000 0xffc0 at 2 \
   match u8 0x10 0xff at 33 \
   flowid 1:10

# web browsing packets in 1:15:
tc filter add dev $DEV parent 1:0 protocol ip prio 5 u32 \
      match ip dport 80 0xffff flowid 1:15

# rest is 'non-interactive' ie 'bulk' and ends up in 1:20

# ... except for the Minimize Monetary Cost, which are 1:30
# TOS Minimize Monetary Cost, in 1:30
tc filter add dev $DEV parent 1:0 protocol ip prio 10 u32 \
      match ip tos 0x02 0x02 flowid 1:30

########## downlink #############
# slow downloads down to somewhat less than the real speed  to prevent 
# queuing at our ISP. Tune to see how high you can set it.
# ISPs tend to have *huge* queues to make sure big downloads are fast
#
# attach ingress policer:

tc qdisc add dev $DEV handle ffff: ingress

# filter *everything* to it (0.0.0.0/0), drop everything that's
# coming in too fast:

tc filter add dev $DEV parent ffff: protocol ip prio 50 u32 match ip src \
   0.0.0.0/0 police rate ${DOWNLINK}kbit burst 10k drop flowid :1

    

Finally, some resources on HOWTO Prioritize packets in NAT with QoS:
http://lartc.org/howto/lartc.cookbook.fullnat.intro.html
http://luxik.cdi.cz/~devik/qos/htb/

Last updated on 2005-10-18 15:00:00 -0700, by Shalom Craimer

Back to Tech Journal