Linux Traffic Control – QoS
I recently started using VoIP, using NodePhone with Internode. This post describes how I implemented QoS on my router to make VoIP work well. It also allows me to prioritise some traffic on my network, such as SSH, while deprioritising others, such as BitTorrent and SFTP.
Background
You may want to skip this section initially and refer back to it if you come across a concept that you do not fully understand later in the document.
I assume that you have a working understanding of networking. This means you must know what an IP address is, what TCP is, and what the relationship of these to application level protocols HTTP and SSH is. IPv6 is not covered here but probably will be in the future.
QoS, or Quality of Service, is any mechanism for guaranteeing a particular amount of throughput or latency for some type of traffic.
VoIP is basically about making phone calls over your internet connection. NodePhone uses the SIP standard for this. It supports incoming and outgoing calls to the PSTN by giving you an extra phone number. Internode recommends getting a QoS equipped router, but I use a Linux PC as my router and was unwilling to switch for various reasons. Using VoIP without QoS is much like talking on a mobile phone while going through a subway. The call is jittery with frequent voice dropouts.
IP packets have a Type-Of-Service (TOS) field in them. Applications can set this field to values such as Minimise-Delay, or Maximise-Throughput. This allows applications such as BitTorrent to signal that they are more interested in throughput than latency, while an interactive SSH session (think typing at a shell) could signal the opposite. The default is for normal service.
Shaping is when we prioritise outgoing traffic. It is possible to do this really well because our router is in full control of what data is sent out onto the internet. It is possible to specify what rate packets should be sent at and in what order. Eg, VoIP packets before BitTorrent is a sensible rule.
Policing is attempting to enforce rules upon incoming data. This is not possible because we cannot directly influence how much data other computers will send our way. Instead what we do is drop packets that are coming in too fast in the hope that the sender will slow down. Thankfully this is exactly what TCP is designed to do, so this strategy works reasonably well. The aim here is to make your router into a bottleneck that is slightly slower than your internet connection so you influence over the rate at which different classes of packets come in.
Goal
I needed to set up QoS to prioritise the VoIP traffic on my network. The existing documentation is not exactly plentiful and while it was invaluable, I found it difficult to understand. There don’t seem to be any turnkey solutions that do QoS for Linux in the way that I wanted.
I have four categories of traffic.
- VoIP traffic – guaranteed 64Kbit, highest priority
- High priority – interactive SSH sessions (not SFTP)
- Normal priority – the default
- Low priority – bulk data transfers like SFTP or BitTorrent
If I’m on a phone call, that traffic absolutely needs priority over everything else. The packets from the VoIP phone call should never be dropped or delayed.
Immediately after that comes other high priority traffic. This is anything with the Minimise-Delay TOS bit set in the IP header. SSH sets Minimise-Delay for interactive shells, but not for SFTP/SCP. Very convenient. Be aware that if you use ControlMaster to piggyback SFTP on an existing SSH connection that the TOS bits are set per TCP connection. I also put ACK packets in here; one of these is sent for every packet that you download. If you’re doing a large upload then these tiny packets can get delayed, causing your download to slow down. I find it works well to give them a high priority.
Next priority is general traffic, this is the default bucket. By default all traffic goes in here. This includes web traffic, instant messaging, email, etc
Finally we have the data that we don’t care about at all. I’m a little nasty and dump any traffic with the Maximise-Throughput TOS bit set in here. That includes my BitTorrents (using rtorrent) and SFTP/SCP traffic. This means that the traffic doesn’t really get it’s throughput maximised at all, but it works well for my purposes.
Linux Traffic Control
This is all done using the Linux Traffic Control system. It’s made up of a tree of queues, each with a specific algorithm for dequeuing packets.
Have a look at the shaper script. All of the rates in this file are specified in Kilobits/sec. You may need to read this text more than once.
I’m using the Hierarchical Token Bucket (HTB) for each of the four categories mentioned above. Each bucket has an associated rate, maximum rate and priority. The available outgoing bandwidth is measured and divided up amongst the different categories of traffic. The bucket gets a minimum throughput specified by ‘rate’, and a maximum specified by ‘ceil’. This maximum could be reached if one of the other buckets is not using its allowance. For example, while there is no VoIP traffic other buckets can use that 64Kbit allowance. Packets are dequeued from the buckets in order of the priority that each is given. Lower priorities first.
Next we add a Stochastical Fairness Queue (SFQ) to each bucket. The SFQ organises all packets it receives into sessions, like TCP connections, using a hashing algorithm. It dequeues packets from these sessions in a round-robin fashion. This means that if you have two connections they each receive an equal share of the available resources.
Packets are assigned to a particular queue/bucket by inspecting their header for source/dest addresses/ports as well as the TOS field.
Finally, some basic policing is applied to incoming traffic. Incoming VoIP traffic is never policed, but all other incoming traffic is policed at slightly less than the link’s maximum throughput. This forces the router to be the bottleneck and encourages the sender of any traffic that has been dropped to slow down.
QoS on incoming traffic really needs to be handled by your ISP, and while Internode claims that they do this, I found that I needed these policing rules for VoIP to work while large downloads were occurring. I suspect this is due to Internode not being able to provide proper QoS for my ADSL1 connection on a 1.5Mbit Telstra port.
References
- Shaper script – latest version
- Shaper script – original version
- Linux Advanced Routing & Traffic Control HOWTO
- RFC1349 – Type of Service in the Internet Protocol Suite