Jeff Simmermon, the Digital Communications Director for Time Warner Cable, has responded to the charges that TWC is responsible for the lags and disconnections plaguing East Coast World of Warcraft players. He took a look at the traceroutes posted on Blizzard’s user forums and sent the response.
I’m the director of digital communications at Time Warner Cable, and I’d like to bring some clarity to this discussion. We’re happy to to take our lumps when we’ve earned them, but it doesn’t seem to be the case in this instance.
Take a look at some of the traceroutes posted to the thread in question … starting here, at comment #446: http://tinyurl.com/5gqe27
If you follow the commenter’s posted trace results, you’ll notice that it’s only on TWC’s Roadrunner (rr) network for the first 6 hops — with maximum response times of 10 ms. The response time jumps drastically at hop # 11 — when the trace is no longer on the Roadrunner network.
Scroll down further on the same page to comment #456, and you’ll see something similar — a giant leap in lag times. However, this trace never touches our network. It starts at Verizon, goes to Alter.net at hop #5, and then jumps to ATT.net’s network at hop #8. Hop #9 shows a response time of 114 ms — quite a jump from the 49ms at hop #8.
On the first page of the thread, you’ll see something similar: http://tinyurl.com/3hfs9k
At comment #10, the lag time leaps from 18ms on our network at hop #6 to 150ms at hop #7 — on Level3, an Internet backbone.
At comment #18 (same page), the trace again never touches our network. The lag jumps from 15 ms at hop #3 to 261 ms at hop #4, while on the Verizon network. The hops vacillate between high and low response times throughout the trace.
Blizzard’s comment at the top of the thread that “Unfortunately this means that the only commonality between all the players experiencing these disconnects and extreme latency is Time Warner/Road Runner” is a pretty interesting choice of words, in light of the fact that several of the troublesome traceroutes posted in the forum itself never touch our network.
Jeff Simmermon
Director, Digital Communications
Time Warner Cable
So, is Blizzard just trying to pass the buck to the best available scapegoat? If both companies aren’t responsible for the lags and disconnects, whos is? Does anyone know how this crazy Internet works?
(Photo: ashley_dryden)
More From Consumerist
-
There’s A Thin Line Between Having Fun At Time Warner Cable’s Expense And Just Being A Jerk
-
Time Warner Technicians Rescue Kitten Snuggled Up Against Cable Lines
-
Time Warner Cable CEO: "There Are Too Many Networks"
-
Time Warner Cable Impersonators Ask: What Can We Do Worse For Our Customers?
-
My Parents’ Time Warner Cable Installation Will Cost Either $3,000 Or $40
This would never have happened if we had elected Al Gore.
Sounds like there might be a bad router in the routing tables. RR COULD adjust their tables to remove the bad router.
Time Warner Cable doesnt care about WoW people
Anyone else having ATT connection issues outside of iPhone problems?
My tethering stopped being able to connect for my laptop, but the PDA still sees the Internet. Likewise, my backup dial-up account was also unable to connect.
Reason is, since Monday, any of my customers with ATT/SBC/PacBell/Yahoo/Prodigy accounts or connections have also been reporting connectivity issues, not counting any iPhone issues.
Calls to tech support net a “not our equipment” from ATT, which is the stock answer whether true or not.
Well, that is the problem with TCP/IP, it guarantees delivery, not performance.
Eventually, the packet will get there…
It may very well be that neither TW or Blizzard’s provider is in control of the offending router. In fact, there may be no direct connection between the two networks, leaving them at the mercy of the 3rd party. In this case it is Level 3, the backbone provider.
They should hit their routers with a Sword of Swift Transfer.
MOAR TUBES!
I heard it was just a series of tubes. Better get some draino.
Well, I’d say the problem is with Time Warner’s backbone, level 3. Level 3 provides fiberoptic & wireless based bandwidth pipes to ISP’s, and some of their routing looks screwy. Methinks Time Warner needs to step up, and give their service provider a boot to the groin.
Time Warner certainly has a choice in who their upstream providers are, so they aren’t completely void of fault.
I bet Geek Squad can fix it.
@backbroken:
Just give him a pair of tights and a mask and he’ll fix this problem just like he fixed the environment…Are there Nobel Prizes for internet saving?
Well, this probably isn’t the best forum to discuss the traceroute results, but TWC may be right. It also may be that there’s some kind of config problem on all the TWC edge routers. It’s strange that there’s that much latency right after the TWC edge, especially if it happens across multiple peers…so a common config may be the culprit.
Maybe they’re doing QoS at the edge and WoW stuff is on the losing end of that?
Blizzard is known for passing the buck as fast as possible. When they were suffering massive lag and usability problems a couple years ago, they tried to pass it off as a backbone problem, and finally admitted that it was a problem with their servers. It was fixed, eventually, but they were very quick to defer blame.
“This would never have happened if we had elected Al Gore.”
We did.
Does anyone know how this crazy Internet works?
Of course…..ask Ted Stevens. I am sure you can get a straight answer from him about how the internet works.
See, the actual problem here is that the internet is not a truck, it’s a series of tubes. And right now the tubes are clogged with enormous amounts of material.
@Canino: “I bet Geek Squad can fix it.”
I don’t know why but I busted out laughing when I read your comment…LOL!
@MyPetFly:
“This would never have happened if we had elected Al Gore.”
“We did.”
This is so true.
@magic8ball: Ted Steven’s e-mail, mostly.
Blizzard has turned into a money grubbing horrible company since Sierra bought them.
money money money
@Lithium542: Umm…what? L3 is a different company than TWC/RR. I’m not sure what you expect TWC to do about it other than to notify L3 about the latency experienced through some of their routers. L3 would most likly respond two TWC along the lines of, “Your customer’s satisfaction is important to us…”
TW response sounds reasonable. If one of the routers between the users and blizzard is blocking it, it’s likely a whole region would be blocked as well.
Time warner takes you to the internet. Once you get there, you bounce around on tons of different routers until you get to the ISP of the destination. It’s hard to explain in non-technical terms.
All I can say is, a long time ago, I was blocked from Yahoo Mail, and it was neither my ISP nor Yahoo. Some idiot in the middle had a badly configured router.
@Dobernala: unfortunately, it’s not as easy for them to change backbones as it is for us to change cell phone companies or ISPs (I can’t imagine what their ETF cost would be).The fact that TCW laid out the details of the latency issues shows that (a) they researched it, and(b) identified the problem.
HOWEVER, since this proves that nothing is completely network-specific (that they all share backbone), then maybe they can all stop pretending that we pay out the nose for service because they own the lines?
Sounds like ATT’s hub is having issues………again…….
Have someone in your family who lives somewhere else host a vpn server for you, connect to that, and pipe your WoW through the VPN. That’ll work
Sorry one more: I hear the all the time when lag times between hops are high. “The latency is astronomical”
Who is to blame here? Well with networks you can always pass the buck to the ISP, the routers and switches along the path, the backbone provider, the end servers, etc. Who knows what the real problem is? Maybe just too many WoW heads trying to battle it out at the same time?
I think that Time Warner might be right for once. However I bet it’s something they can talk to those providers about instead of sitting on their butts. They might have more influence on Level 3 than us regular consumers.
A few years back I experienced a similar problem on the a section of the Adelphia network that ended up with Time Warner.
There’s multiple problems with all this, and none will help find the root cause.
First, the traceroute utility in Windows runs the trace using ICMP. ICMP traffic is rate limited across just about every network now, so a Windows trace may not give you a clear picture. Your ‘live’ data may get somewhere in 40ms, but your Windows traceroute could show 100ms if some rate limits were being hit.
Secondly, the Blizzard tech support folks in their forums are not technically proficient enough to understand the underlying network issues that could cause something like this. They don’t have those people on staff because they don’t really need them. All the WoW realms are co-located in AT&T cages, they don’t need a lot of network experience to set that up.
Finally, you have a classic situation where and end user blames their ISP, the ISP blames the content provider, and the root problem is on a carrier network somewhere in the middle. Until one of them mans up and has someone competant look seriously at the problem, nothing will change.
In my case, the problem was a sketchy interface on an AT&T router that most of my area ended up passing through. I was only able to find that out because of my professional contacts. Blizzard support was no help, nor was Adelphia. I was actually threatened by Adelphia’s Director of network security at the time, because he thought I was contacting providers and pretending to be employed by them.
It’s a crappy situation, unfortunately there’s no easy resolutions to it.
@Canino:
you forgot the “‘t” after can
that is pretty lol, I’m glad they atleast investigated the issue and owned those WoW fags in the face
Sounds like TWC is putting in a delay or something between 10 and 11. I have no problems on my local DSL provider.
[upitall.googlepages.com]
looking at that thread…it does seem like all of them occur at the edge of TWC. My guess is that to try and stretch bandwidth to the nth degree, TWC is slashing QoS. I bet money this isn’t going on with just WOW they just have a big enough community to make noise.
I think Ted Stevens should have to clean the inter-tubes as parrt of whatever punishment he gets for being a corrupt bastard.
Trace Route is fucking meaningless. It uses ICMP packets not UDP. With TW’s deep packet inspection, it could be passing ICMP unmollested (as screwing with them would be a dead give away) but hosing UDP.
Games use UDP, so if thier DPI is throttling UDP and not ICMP, traceroute would show nothing.
I blame the Russians and their cyber war.
My best guess is that TWC is lying. They say they can’t pick up the phone and call other networks and say “hey your lag time is unacceptable”? Bullshit.
Ha! My diabolical plan to destroy the “World of Warcraft” is coming to fruition…thank you, Time Warner for your help.
OK, seriously though…while I somewhat agree with the spirit of the TWC Directors message (that it may not be entirely their fault), I don’t agree with the message…as it’s still their problem. Further, the packets do touch their network…the packets don’t magically jump from hop #1 to hop #6.
I can only begin to guess what the root cause of the problem is, but, coming across like you’re passing the buck will only put fuel to the fire…
“If you follow the commenter’s posted trace results, you’ll notice
that it’s only on TWC’s Roadrunner (rr) network for the first 6 hops —
with maximum response times of 10 ms. The response time jumps
drastically at hop # 11 — when the trace is no longer on the Roadrunner
network.”
And if he’d followed that line of thought to hope #12, he’d see the
ping times drop back down, which means traffic going THROUGH hop #11 is
fine, but the ping time goes up going to 4.68.16.133 (hop#11)
“At comment #18 (same page), the trace again never touches our network.
The lag jumps from 15 ms at hop #3 to 261 ms at hop #4, while on the
Verizon network. The hops vacillate between high and low response times
throughout the trace.”
It jumps up again on one hop and then drops back down; which again means
traffic going through hop #4 (and #7 and #10) is not latent, just
traffic destined to those hops
I’m not sure this guy understands traceroute and I wouldn’t put much
weight in his troubleshooting of network issues
Can someone explain to me why it took the Digital Communications Director of TWC to come up with an answer to the problem? Each person that called into TWC should have reached a tech that should have been able to see and provide the same information. Causing a lot less frustration to TWC customers.
Also,
TW edge routers decide what back bones to use to get various places. If they are choosing poor routes, even though the slow down is outside of TW’s network, its still their fault. TW needs to call their peer up and find out what the problem is instead of shrugging and passing the buck.
Everyone’s speaking a language I do not understand.
/quietly leaves thread
@OnceWasCool: What does that have to do with this article? Please keep your comments on topic.
After looking at some notes from my tech support days, I can definately say the problem is kinked cables.
You see, your cables have to be straight or smoothly curved, not kinked. If you have a kinked cable the zeros can get through ok, but the ones get stuck and cause a log jam.
All these WoW users should make double sure they have no kinked cables. If they find a kink, they should straighten it and shake it a little to get the ones moving again. It might take a while for it to all clear out, but the speed will return as soon as the backed up bits have gone through.
You’re welcome.
@valarmorghulis
I’m guessing you’ve never worked for an ISP.
Just like you have a choice in ISP’s, they have a choice in bandwidth providers. Global Crossing, Covad, Cogent, etc. are all viable alternatives. If Level 3 is giving them poor service, then they need to move. If they have a flawed routing table, or poorly functioning network hardware in specific locations, other companies that use Level 3 for bandwidth would theoretically complain too.
When I used to work with Cogent, they’d occasionally forget to pay their routing/DNS bills, and you’d only get half the internet….
Wow, MR. Director. You really know tact… way to respond with excellent PR skills! Wahoo, YOU SUCK!
In other news, due to the rediculous lag times, the WoW geek walked outside, saw his shadow, and promptly hid. 6 more weeks of summer!
@CrazyMann — I can field that one, having recently sat in for a shift at one of our call centers in Queens. Our CSRs process a tremendous number of calls every day, and don’t have the time to do the research that I did.
If you — or anyone else — has any questions, feel free to e-mail me at jeff.simmermon@twcable.com …
@eliblack:
Every company is quick to defer blame, or do you not see what TWC is doing here? When you call up support, it always everyone else’s fault but theirs first.
Those issues with latency are really not horribly bad, not enough to cause these issues.
I want to point out one thing though. You can have lag and disconnects even with a good ping. This can have anything to do with bad equipment to traffic filtering.
It’s probably the NSA’s fault. They have to inspect every packet, WoW or not, for “terrorism” of course. Skellbasher’s comment above said “WoW realms are co-located in AT&T cages”. AT&T has allegedly provided the NSA with a room specifically for them them to inspect every packet through their network.
[en.wikipedia.org] <– Hepting v. AT&T (lawsuit filed by the EFF)
Maybe terrorists are masking their communications through WoW?
Shouldn’t TWC be able to change its routing tables to skip those hops?
I work in software and I would do exactly what blizzard said. They don’t have anything to do with how you connect to their game, they just make their servers availible. It’s the responsibility of the ISP to make sure they’re customers can get to locations on the if they are selling internet access (unless they specifically tell you, “No, we don’t let you go there”).
Ultimately this means that someone has to contact the company with bad hops, and TWC doesn’t want to do it, Blizz doesn’t want to do it, so its left in the hands of the consumer, which quite frankly is F***ing BS.
@InfiniTrent:
Me too. But look — there’s a cute kitty in the pic! I can play wif kitteh instead of playing WoW?