Thursday, December 13, 2012

Information Super Highway

We can rebuild him...we have the technology - Oscar Goldman
I have been a customer of ATT High Speed DSL over the past two years and I've had a mediocre experience with it. It wasn't horrible, but it was a long way from being good. We paid for the maximum speed available to us, which was 3Mbps down and 256Kbps up. It's not the fastest out there, but it should be good enough for what we need to.
After running on this for about a year we were having a lot of problems streaming movies in the evening. I grew up in a world starting with 300 baud modems and I have been doing large content files since the days of 56k. So having to buffer a video for ten or fifteen minutes before watching it was no big deal to me. I was used to the early days of spending hours downloading a video to watch it later.Mrs.Chaos expects more from her internet. What we eventually realized was that during the evenings our connection speed dropped to 0.5 - 0.8 Mbps and that is too slow to effectively stream much. If we both wanted to watch movies at the same time? No go. Even during the working day I would have a online meeting and come running out, "Stop using the Internet!". No big deal to me, but I suppose in the modern era these things shouldn't happen.
We called ATT and told them the internet was slow and asked if they could do anything about it. ATT customer care reported that we had the fastest internet available to us. We tried again at a later time and got the same response. We looked at getting SureWest but they said they couldn't supply our home. SureWest could supply five homes down in each direction, but not ours. So we resigned ourselves to having a barely usable internet in during the evening "Netflix hours."When I got my iPhone 4s with its psuedo-4G speed, I ran a speed test, and it was blazingly fast. I tweeted it about it, sarcastically congratulating ATT on its ability to deliver fast internet to me through a highly saturated wireless spectrum but not through a copper wire running into my home. Something happened. @ATTMike tweeted back an offer to help. One of his team gave me a call, escalated me up to Level-2 support and they fixed things. "Looks like the upstream router is completely saturated during peak load times, let me just move you over to a different router." For the next eight month, we had pretty good internet speeds.Then came the dropped connection saga. The internet would drop unexpectedly for about 1-5 minutes. This would happen on average once an hour, but that meant sometimes it was happening every twenty minutes. You cannot have work teleconferences. You cannot stream shows. The Internet was mostly unusable. This went on for a few days and then everything was fine for a week and then it happened it again. I tweeted with ATT and Mrs.Chaos called ATT.I did what I expect anyone would do. I connected the DSL modem directly to my laptop, put the DSL modem into bridged mode, ran PPPoE on the laptop, increased the debug logging levels and looked at what was going on in the console log. AHA! The ATT PPPoE server was sending disconnect commands downstream and my client was happily disconnected and then immediately reconnecting whenever that happened.
I told this to each ATT person I talked with, but I was never able to gain access to one of the Grey Beards in the back room and so these comments fell on deaf ears. One of the ATT people on the phone said, "It looks like your DSL modem is about two years old. Those things tend to give out after a couple years so we'll send you a new one." This problem was intermittent, so as soon as everything was fine for a couple days, they would close the ticket. The problem would come back and I'd start over.They sent a technician to the house to make sure all of our wiring was fine. He was the first person who seemed like he understood my comment about the PPPoE server and said, "that makes sense. I'll put a note about that into the ticket so the server-side guys can take a look." Each time the problem would go away I hoped it would be the last and that there would be some person at ATT who looked at some dashboard and said, "Oh hey! That red light there is blinking and it should be green. Let me just reboot this." Everything would then start working magically. His last thought would be, "I wonder if there are open tickets about this issue? Whatever, I'm going home." But each time we had a few days of perfect service it would follow up with a few days of useless service.
We spoke a few times with SureWest. Surely they could pull their fiber six houses down? When complaining to one of my friends over this whole ordeal he talked to me about ATT U-Verse and I learned something very interesting. ATT runs two different internet services from two different companies that it acquired. It runs the "ATT High Speed DSL" over its copper wires. It runs the "U-Verse Internet Service" over its newer fiber cables. While our 3Mbps speed was the fastest "ATT High Speed DSL" available to us, "U-Verse Internet Service" ran with speeds of up 32Mbps running fiber to the home.I called to switch. It required a technician to come on site, but no problem, they setup the appointment for three weeks later. The cheapest way to do the install was to get TV for one month along with the service. Weird, but okay. Obviously, during those three weeks of time, the DSL line didn't have any issues. High Speed DSL was scared and making a last ditch effort to stay connected.
The installation was a bit more of an ordeal than I expected. The first ATT technician showed up at 9am to "run fiber" from the pedestal at the intersection down to the pedestal in front of the house. I assume this means lighting up some dark fiber, but maybe he dragged a wire through half a mile of smurf tubes. I can't say. I do know that it broke my home internet at 9am. He left around 10am and our internet didn't work.The second technician showed up at 1pm (when ATT told me he would) and started the home installation process. It was interesting to watch as he installed the fiber termination point into our garage through the wiring box and run the cables into it. He then plugged our home phone line and coax into the fiber termination device. Three and a half hours later and he the new ATT provided 2WIRE router installed and providing internet to the home. I let him go.Once he was out the door it was time to put the home internet back into non-standard configuration. I turned off the 2WIRE wireless network, plugged in my Airport Extreme and then set 2WIRE to provided the Airport Extreme an "External IP" and to use the Airport as the DMZ machine. It worked flawlessly (I'm thankful!).The end of the story? I now have 12Mbps down and 1.5Mbps up for the same price I was paying before. All of my home services are running exactly as they were before. Knock on wood for me that everything continues to be awesome. Stay classy ATT.(Update: The two days after installation, the internet dropped to less than 1Mbps during the day. But after those days it has been fine.)