Partial Page Loading Resolved

Back in February, I talked about an issue with certain people getting only partial pages on this site, Cocoa Dev Central, Tree House Ideas, and some others. It was hard to debug since the vast majority of vistors had no problems. The good news is that I think I've finally solved it.

It looks like this was an issue with OpenBSD's PF packet filter, which is also used by FreeBSD. Specifically, the problem was with PF's "state keeping" functionality. The idea is that keeping track of a client "session" reduces the amount of validation work the firewall has to do.

The affected clients would consistently only receive the first 4096 bytes of .html or .css files, followed by a single garbage byte (looks like a memory stomp), and then nothing. For some reason, files parsed by PHP would only get to byte 3884 before getting chomped.

Fortunately, the W3 validator was in the affected group, so I could use it to figure out what was going on. The interesting thing is that its requests would never show up in Apache's logs, even though it was clearly getting new content within that first 4k.

In at least a few of the cases, the clients were behind some sort of proxy and/or virtual network. It's possible that a router/proxy and PF would disagree about the client address of a request and PF would just stop delivering packets, though this is just speculation on my part.

If anyone has some insight into whether this is actual bug in PF or not, I'd be interested in hearing about it.

Comments Temporarily Disabled

I had to temporarily disable comments due to spam. I'll re-enable them soon.