Two of my roommates let me know Friday the Internet sucked in the kitchen. I knew this used to be the case, and I suspected our metal framed kitchen table of causing issues, but I thought it was solved.

Last month we got new Internet service and I redid our network. Since we had scaffolding on the side of the building for painting, I took advantage and ran some cables from the living room to the kitchen and to my room upstairs. This would let me have three routers in different rooms but hardwired, which would be much more reliable than wireless repeaters. I got that all set up mostly fine, though I had some hangups when I realized the information I followed for wiring Ethernet jacks was inaccurate.

Well, having been told the Internet was still having issues in the kitchen, our new temporary roommate and I did some ping tests and saw some quite irregular behavior. There were some packets lost, but not a ton. But more annoying, the ping times would vary from a few milliseconds to almost a second, and it seemed pretty sporadic. I decided this must be related to the miswiring of the cable connecting to the kitchen router, and I redid the ends of all those cables. I had a scare then when the cable went from being possibly faulty to definitely not transmitting on any path according to my wire tester. This ended up being due to a problem on the end I didn’t touch. Apparently there was a loose wire that got jiggled. Once I got that sorted, I decided to replace the kitchen access point, an ASUS RT-N16 I bought in 2012, with the sort of extra Apple AirPort Extreme access point I had up in Paul’s old room, acting as a wireless repeater. That would allow people in the kitchen to connect over 5Ghz channels, for whatever benefit that might get them.

After making those changes, the pinging was still unusual, and I observed it from my room with my hardwired computer, though I was going through a switch, adding to the complexity. There were times when the Internet would stop completely for less than a minute. I noticed this a few times on my computer, but I thought it was related to my torrents freezing my computer, and didn’t know it affected the whole network. There had been network wide outages I noticed after first setting this all up, but I thought that stopped when I upgraded the firmware on our main router, an ASUS RT-AC68U.

Briefly for the record: I had been using what I thought was the latest DD-WRT firmware for the router. I searched the official router database for “RT-AC68U” and found the router listed:

RT-AC68U Firmware - Webflash image asus_rt-ac68u-firmware.trx 2014-04-22 26,30 MB

This is apparently build 23940, and obviously is two years old. That’s not to say it should be filled with problems, though. But when I did a Google search for "rt-ac68u" "dd wrt", the top result was a link to Asus RT-AC68U - DD-WRT Wiki. That page listed a bunch of different firmware links, and it was not at all apparent to me what the differences were. I tried the BRAINSLAYER BUILD: Firmware: DD-WRT v3.0-r29193 std (03/02/16) since it was clearly very new. It had a bit of a refresh on the interface, and a few new features, but largely it was the same, which was fine. I hoped the new firmware would solve all the problems, and when I didn’t notice any Internet outages for two weeks and didn’t need to reboot the router in that time, I thought we were smooth sailing. Till Friday, when I learned the kitchen sucked.

Well, back on that wiki page, I saw another firmware listed as “Merlin Firmware” and linking to http://asuswrt.lostrealm.ca/. I recalled seeing ASUSWRT mentioned on a newer ASUS router, so I thought perhaps ASUS adopted DD-WRT, but I guess that’s just the name of their own firmware. Anyway, the Merlin version is a separately adapted version of the stock ASUS firmware, with some extra features. It appeared to do everything I needed, so I decided to go with that one. Even though the BRAINSLAYER was recently updated, I guess the firmware closer to the manufacturer one would be expected to have fewer bugs? Maybe.

My evening was also complicated by an actual Comcast outage, which threw me for a loop when I thought I really screwed up the router.

The firmware update did not go as smoothly as hoped, though I did get it eventually. I was getting transfer errors no matter what firmware I tried to flash, except a DD-WRT image. I had no idea why the router was only taking DD-WRT firmware, but suspected maybe something was wrong with the DD-WRT interface. I was thinking of trying a more manual method when that became mandatory. One of the flashes failed so badly the router would no longer boot. Sometimes I could get it into this basic recovery mode with a simple web page to upload a new firmware, but that still didn’t work. I ended up flashing via tftp according to the directions in Chris Hardie’s blog, Recovering ASUS router firmware without Windows. I tried that a bunch of times but was still getting transfer errors. I ended up resorting to booting my Windows partition and using the official ASUS firmware flashing tool. That still had issues, but after a dozen attempts, I was able to flash the Merlin ASUSWRT I wanted. I got that set up, and everything seems to be working well!

I then set up a Raspberry Pi as a network monitor, and had a whole bunch of issues getting that set up in master-slave mode when I decided I couldn’t leave well enough alone! But now it’s all working, and I have lovely graphs to prove it, with alerts as well!

Smokeping network monitoring