Now the Netgear factory firmware actually is a heavily stripped down old version of OpenWRT. After reading a few of forum posts and wiki pages I decided that I should go for a bleeding edge snapshot version of OpenWRT (r27153). It’s all working very well now :)

Installation

First thing, I grabbed openwrt-ar71xx-generic-wndr3700-squashfs-factory.img from the website and flashed it onto the router using the stock firmware’s upgrade page. This process went smoothly and rebooted the router. I then used telnet to connect and was prompted to set a password. That disabled telnetd and enables the dropbear ssh server. I also put an ssh public key into /etc/dropbear/authorized_keys. So far so good.

OpenWRT uses a squashfs as the base read-only filesystem, with jffs2 set up as an overlay filesystem so you can write to anywhere. You can use the builtin opkg package manager to install a bunch of useful software beyond what is built in. Currently I have 2.3M used with 4.2M free :)

I didn’t bother installing a web interface, instead preferring to do all the configuration using the UCI config files in /etc/config. See the UCI docs for a description of what to put in all these files. Whenever you start a service, eg dnsmasq using the /etc/init.d/dnsmasq script, the appropriate UCI files are read, a temporary native config file for the service is created (if appropriate) and any necessary command line args are generated to start the service.

Syslog

If you’re trying to debug something, you can view the syslog on the device using the logread command. The logger command will echo its arguments to syslog.

Wireless Setup

Most of the /etc/config/wireless settings are automatically detected. Just set the encryption to psk2 (WPA2 AES) and put in your preferred passphrase and ssid.

The radios on this device support multiple SSIDs. Each of these shows up as a separate network interface in Linux. I chose ‘guest’ as the name for this interface.

config wifi-iface
  option device radio0
  option network guest
  option mode ap
  option wmm 0
  option encryption none
  option ssid 'your-guest-ssid'

Quick note, I discovered that leaving wmm (Wireless MultiMedia extensions?) enabled (the default) made SIP VoIP calls from my Nokia wifi clients unusable. It’s easy to disable as seen above.

Firewall (iptables)

Now we have our guest wifi network interface, lets set up some firewall rules to isolate it from the main lan.

I’m not using the WAN port on my WNDR3700. There are three interfaces that are relevant. The ethernet switch and my main wifi are both bridged as ‘lan’. This network is served by DHCP and DNS from my Debian router. It is 192.168.1.0/24. I also have the ‘guest’ network, which is not bridged, and gets DHCP and DNS from dnsmasq on the WNDR3700, it uses 192.168.2.0/24. dnsmasq is set to forward DNS requests onto my main router and to explicitly ignore the ‘lan’ interface.

The firewall policy is to disallow everything on the guest network except:

• ICMP
• DHCP/DNS requests to dnsmasq on the WNDR3700
• Packets from 192.168.2.0/24 not addressed to 192.168.0.0/16

That last rule ensures that anything coming from the guest network must be using one of the expected addresses, and also that the guest network cannot send packets to hosts on my main lan. By default OpenWRT allows all related,established packets using Linux iptables’ conntrack module.

Here’s the complete firewall config I’m using: /etc/config/firewall.

Routing

My Debian router also needs to know how to route back to the guest wlan subnet.

# route add -net 192.168.2.0 netmask 255.255.255.0 gw 192.168.1.3

This can be put into /etc/network/interfaces as a post-up line. It tells the Debian router that whenever it needs to forward traffic from the internet back to the guest wlan it should do so through 192.168.1.3. This is the address of my WNDR3700.

Cron and Button events

I wanted to be able to turn the guest network on and off with the push of a button. I also wanted it to automatically disable itself each night at 3am if I forgot to turn it off.

First thing was to write a simple /root/guest-wifi script to enable/disable the guest wifi network from the command line. Then I’d hook this up to the button hotplug system and cron.

Cron is installed but not enabled by default:

cat > /etc/crontabs/root <<EOT
# m h  dom mon dow   command
  0 3   *   *   *     /root/guest-wifi disable
EOT
ln -s /etc/crontabs/root /etc/crontab
/etc/init.d/cron enable
/etc/init.d/cron start

Finally, set up the push button:

mkdir /etc/hotplug.d/button
cat > /etc/hotplug.d/button/guest-wifi-toggle <<EOT
#!/bin/sh
if [ "$BUTTON" = "BTN_2" -a "$ACTION" = "pressed" ]; then
  /root/guest-wifi toggle
fi
EOT
chmod +x /etc/hotplug.d/button/guest-wifi-toggle

System upgrades

You can update and install new packages with opkg, however at some point you may want to upgrade the kernel. Also if you’re using the squashfs images, anything you upgrade is taking up valuable space on the squashfs and the jffs2 overlay.

OpenWRT has a neat solution: sysupgrade. Always make sure to add any files that you want to keep across upgrades to the /etc/sysupgrade.conf file. Any files or directories listed here will be preserved by sysupgrade.

Background

You may want to skip this section initially and refer back to it if you come across a concept that you do not fully understand later in the document.

I assume that you have a working understanding of networking. This means you must know what an IP address is, what TCP is, and what the relationship of these to application level protocols HTTP and SSH is. IPv6 is not covered here but probably will be in the future.

QoS, or Quality of Service, is any mechanism for guaranteeing a particular amount of throughput or latency for some type of traffic.

VoIP is basically about making phone calls over your internet connection. NodePhone uses the SIP standard for this. It supports incoming and outgoing calls to the PSTN by giving you an extra phone number. Internode recommends getting a QoS equipped router, but I use a Linux PC as my router and was unwilling to switch for various reasons. Using VoIP without QoS is much like talking on a mobile phone while going through a subway. The call is jittery with frequent voice dropouts.

IP packets have a Type-Of-Service (TOS) field in them. Applications can set this field to values such as Minimise-Delay, or Maximise-Throughput. This allows applications such as BitTorrent to signal that they are more interested in throughput than latency, while an interactive SSH session (think typing at a shell) could signal the opposite. The default is for normal service.

Shaping is when we prioritise outgoing traffic. It is possible to do this really well because our router is in full control of what data is sent out onto the internet. It is possible to specify what rate packets should be sent at and in what order. Eg, VoIP packets before BitTorrent is a sensible rule.

Policing is attempting to enforce rules upon incoming data. This is not possible because we cannot directly influence how much data other computers will send our way. Instead what we do is drop packets that are coming in too fast in the hope that the sender will slow down. Thankfully this is exactly what TCP is designed to do, so this strategy works reasonably well. The aim here is to make your router into a bottleneck that is slightly slower than your internet connection so you influence over the rate at which different classes of packets come in.

Goal

I needed to set up QoS to prioritise the VoIP traffic on my network. The existing documentation is not exactly plentiful and while it was invaluable, I found it difficult to understand. There don’t seem to be any turnkey solutions that do QoS for Linux in the way that I wanted.

I have four categories of traffic.

1. VoIP traffic – guaranteed 64Kbit, highest priority
2. High priority – interactive SSH sessions (not SFTP)
3. Normal priority – the default
4. Low priority – bulk data transfers like SFTP or BitTorrent

If I’m on a phone call, that traffic absolutely needs priority over everything else. The packets from the VoIP phone call should never be dropped or delayed.

Immediately after that comes other high priority traffic. This is anything with the Minimise-Delay TOS bit set in the IP header. SSH sets Minimise-Delay for interactive shells, but not for SFTP/SCP. Very convenient. Be aware that if you use ControlMaster to piggyback SFTP on an existing SSH connection that the TOS bits are set per TCP connection. I also put ACK packets in here; one of these is sent for every packet that you download. If you’re doing a large upload then these tiny packets can get delayed, causing your download to slow down. I find it works well to give them a high priority.

Next priority is general traffic, this is the default bucket. By default all traffic goes in here. This includes web traffic, instant messaging, email, etc

Finally we have the data that we don’t care about at all. I’m a little nasty and dump any traffic with the Maximise-Throughput TOS bit set in here. That includes my BitTorrents (using rtorrent) and SFTP/SCP traffic. This means that the traffic doesn’t really get it’s throughput maximised at all, but it works well for my purposes.

Linux Traffic Control

This is all done using the Linux Traffic Control system. It’s made up of a tree of queues, each with a specific algorithm for dequeuing packets.

Have a look at the shaper script. All of the rates in this file are specified in Kilobits/sec. You may need to read this text more than once.

I’m using the Hierarchical Token Bucket (HTB) for each of the four categories mentioned above. Each bucket has an associated rate, maximum rate and priority. The available outgoing bandwidth is measured and divided up amongst the different categories of traffic. The bucket gets a minimum throughput specified by ‘rate’, and a maximum specified by ‘ceil’. This maximum could be reached if one of the other buckets is not using its allowance. For example, while there is no VoIP traffic other buckets can use that 64Kbit allowance. Packets are dequeued from the buckets in order of the priority that each is given. Lower priorities first.

Next we add a Stochastical Fairness Queue (SFQ) to each bucket. The SFQ organises all packets it receives into sessions, like TCP connections, using a hashing algorithm. It dequeues packets from these sessions in a round-robin fashion. This means that if you have two connections they each receive an equal share of the available resources.

Packets are assigned to a particular queue/bucket by inspecting their header for source/dest addresses/ports as well as the TOS field.

Finally, some basic policing is applied to incoming traffic. Incoming VoIP traffic is never policed, but all other incoming traffic is policed at slightly less than the link’s maximum throughput. This forces the router to be the bottleneck and encourages the sender of any traffic that has been dropped to slow down.

QoS on incoming traffic really needs to be handled by your ISP, and while Internode claims that they do this, I found that I needed these policing rules for VoIP to work while large downloads were occurring. I suspect this is due to Internode not being able to provide proper QoS for my ADSL1 connection on a 1.5Mbit Telstra port.

References

1. Shaper script – latest version
2. Shaper script – original version
3. Linux Advanced Routing & Traffic Control HOWTO
4. RFC1349 – Type of Service in the Internet Protocol Suite