Mick did not initiate the "compromising" of the server so he should not have to pay for the overage. People get busy, I think the ISP is preying on him for these overages in that I see it as a failure of the ISP to protect his application which resides in their server farm.

I think when there are slight overages the company deserves to charge their exorbitant rates. However when there is a large and uncharacteristic overage they should give him the plan that better covers it. Especially if he was willing to pay.

This doesn't take into account how the account was compromised. If it was compromised because the hosting service did a bad job they should cover the entire fee.

There are very real costs associated with bandwidth usage. I don't know what kind of network his ISP has, but if they are going to be charged for them (at their wholesale rate), he should have to pay for that.

However, it is obviously profitable for them to sell a plan that provides that much bandwidth for $1,199. A good compromise would be to pay that amount and then be done with it.

This is like that story from a week or so ago where a company wanted a guy to pay thousands of dollars in data charges for a movie downloaded abroad.

In most cases the billing system only sees numbers. There may be a cutoff somewhere but a lot of the time it's a fight to get credited for usage before the cutoff, especially if there's no way to prove that you didn't use it.

Most companies that do this kind of billing put the onus on the customer to prove they DIDN'T use what the system says they did, rather than have it be their responsibility to prove that they did.

When I worked at a call center, we were actually trained to pretty much suspect everyone and trust no one when it came to customers. We were not to take their word on anything. If it could not be verified, it was to be closely scrutinized if not outright denied.

If the person was able to gain control of his server because of Mick's own negligence in coding/patching of the site, then I'd say he's SOL on this one.

I think we would have to see Mick's contract to make a determination about who is right. Mick might have allowed his hosting ISP the right to keep the 'tab' running in the event of a sudden traffic surge. Had it gone the other way around and the traffic was legitimate, he might be pissed if the hosting was suddenly shut down due to B/W limits.

I think it all depends on who is responsible for security monitoring and intrusion detection. If the OP is responsible for server security, and someone breaks in and uses a ton of resources, then the OP is also responsible to pay up.

If the hosting company is responsible for security, then they have no business charging the OP for someone breaking in and stealing bandwidth.

@ZManGT: You're kidding right? Slight overage? In this day and age bandwidth is incredibly cheap and anyone who tells you otherwise is a liar.

Put it this way. Comcast charges $60 a month for what amounts to 250 gigabytes a month in bandwidth. Thats for a home user account. That bandwidth properly ends up costing Comcast about $5, $10 when you add in maintaining the network itself. Bandwidth is dirt cheap.

Sure if this guy was 1000 gigs over his limit then they should have the right to charge a small fee but any good web host will shut your account off until you call them and approve a fee for more bandwidth

Sure if this guy was 1000 gigs over his limit then they should have the right to charge a small fee but any good web host will shut your account off until you call them and approve a fee for more bandwidth, they wont just let the charges rack up.

Time to move hosts.

I had a dedicated server that had the same problem except it wasn't a compromised server. It was a problem with the bandwidth monitoring software that was randomly adding gigs of bandwidth to my account at irregular intervals.

Demand to see hour-by-hour bandwidth logs. I was able to disprove the charge by showing that the bandwidth I was being charged for was impossible for the server to actually serve. The program was showing huge spikes that would have been impossible for a network card to achieve (a 100mb card for an hour showing traffic that would have required 250mb of throughput).

They wanted something like $9K from me and I ended up paying nothing and getting a month free for my trouble.

@axiomatic:

If Servepath offers typical dedicated servers for hosting, it's the sole responsibility of the customer to keep the server patched/updated to prevent compromises. I don't agree with this policy, but that seems to be the norm for hosting companies. Just a way of passing the buck, in a number of ways, along to the customer.

However, often times, there are no patches/updates for some vulnerabilities, especially the zero-day ones.

Also, most hosting companies, well the ones that actually care and want to keep their customers and not get bad press, will give the customer a break and allow them to pay a much lower fee as long as they get the point across that they are responsible for keeping everything updated from here on.

But, if everything above is true, I'd switch hosting companies asap...without thinking twice.

If they did not tell you, they gain $5,700. If they did tell you, Zero. Mmmm....

@axiomatic: You don't understand how hosting works.

First they are a data center and second he owns a dedicated server. That means that the server belongs to the customer and he pays "rent" to let that server sit in a rack and user electricity. Then he has to pay bandwidth costs. The fact that the server was compromised is not the data centers fault because they don't manage the server, it doesn't belong to them, it belongs to the client.

The fact that they didn't notify him of the excess bandwidth is bad on them. The customer should have received emails as he was getting close to his limit and not a week afterwards.

I am now looking for a new hosting company. Do not wish to have this kind of experience with them.

@alarmpro: The key will be the notification. Contract or not, was it set up so notification was supposed to happen sooner? Offering notification and then claiming it's not their fault you weren't notified shouldn't hold up.

Personally, I think it's odd they won't just settle for the $1,200. It sounds like a reasonable compromise.

If it's a unmanaged server (like 95% of them are), it's up to the customer to manage... that means keeping an eye on bandwidth, security issues, and mitigation. Customer accepts 100% responsibility.

If it's a managed server, than the ISP is responsible for the whole incident.

It sounds like an unmanaged server... which means the client foots the bill.

This is why most get daily (if not every few hours) bandwidth totals, cpu consumption, apache stats, etc. so they know what's going on. Anything looks unusual you check it out.

Speaking as a web developer, it IS Mick's responsibility as the server administrator to maintain security -- and to monitor it. If my server gets DoSed, I can be declared responsible for the resource usage of the attack. Not fair, since I have no control out of an outside attack, but that's unfortunately the way it works.

At the same time, as someone said, as this is a large uncharacteristic issue, the hosting provider should investigate this. If Mick is not at fault due to poor security, then it falls on the ISP. If Mick was using a poor password, it's on him.
.

@NotATool: It's a dedicated server owned by the customer completely managed by the customer.

I was in the hosting business for over 10 years. I typically dealt with shared hosting, not dedicated, and we did offer the customer to upgrade to another plan that might have supported the -- we'll call it-- "new" bandwidth needs in order to avoid the overage charges. I'm surprised this company did not accept that proposal from the customer. Some providers might argue that the server admin (the customer in general) ensure that "compromises" not be allowed to happen. At the same time, I feel that hosting companies should try to work with their clients whenever possible. It would be great if the provider in this case was able to offer their perspective here.

There is a plan they have that allows 4,00GB per month that costs $1,199 per month, and they are totally unwilling to allow me to pay that amount and be done with the issue. At least that is a more manageable amount to suck up than $5,700.

The fact that they aren't allowing him to pay for the 4,000GB one makes me think this company is pretty awful and just looking for a big pay day.

@Corporate_guy: But they have the service that allows 4,000GB and he's willing to pay for that, but they aren't willing to take it. Seems they just want a big pay day. And now they are getting bad press. Stupid.

@midwestkel: Yeah, with our setup we get e-mails ever week outlining our usage as well as warnings when we start getting close to our limits so that we can throttle back usage or turn it off altogether.

The web host is passing on their own cost to the customer based on what the customer signed up for. They get a cheaper rate from their providers for bandwidth *IF* they sign a high commitment level of bandwidth to be used on their carriers. What Mick seems to be doing is asking them to pass on the cost of a higher commit after the fact. At the same time, his host would be eating the costs of a lower commitment level based on Mick's original commitment. It also sound like Mick is trying to get that commitment price as a one-time deal so this doesn't help their profits in the future either. If he can't afford to deal with overage charges then maybe he should consider going with an unmetered bandwidth plan.

Full discloser: I work for competing web host company.

@GMFish: Only if you are dumb enough to pay it.

@nakedscience:

He agreed to pay for overages when he signed up. Here is the relevant Terms of Service agreement

"The TOS states (e) If Customer exceeds its transfer allotment, bandwidth commitment, or other pre-paid Service allotment listed in Customer’s Signup, as solely measured by ServePath, ServePath may: (i) charge Customer for such overage immediately via credit card and will notify Customer in writing of such charge; or (ii) issue a one-time invoice for such overage, with payment due by wire transfer or direct deposit in 7 days. If ServePath concludes, in its sole discretion, that Customer’s account reflects a pattern of repeated overage, ServePath may require prepayment for such overage, and Customer will make such payment on its regular payment date or on a monthly basis. The provisions of this Subsection 2(e) apply regardless of the cause of overage, even if caused by hacker activity or other third party actions."

the customer is entirely responsible in this case unless a corresponding managed services security package was included in their purchase.

The ISP probably needs to take a hike. My hosting is through Dreamhost. I've heard they're aggressive with taking down compromised accounts, but I've never had mine taken down. Even if I was compromised and didn't find out, I have a quota of 8.4TB. I wouldn't realistically expect to be able to suck that out of a shared server without doing something stupid, but maybe that's just me.

Not everybody who has a dedicated server needs one -- and sometimes virtual private servers are enough.

(Disclosure: I work for a competitor of Servepath and I have more than a decade of experience in web hosting.)

Servepath should have systems in place that automatically inform their customers when their' bandwidth utilization jumps like that (my employer's system does this, and also alerts our support, networking and policy enforcement groups so they can proactively respond when necessary).

However, a dedicated server is a very powerful tool that can be used for a lot of evil if it's not maintained and managed properly and, unfortunately, the only person who can do this is the customer. Whenever I see the, "I didn't have time because I was busy with (blank)," excuse I tend to wonder what the purpose of the server was and what would have happened if the hard drive or ram in that box had failed. Generally, dedicated servers are owned by small business owners. If your server gets broken into, and you don't notice for a couple of weeks, are you paying any attention to your customer? I don't know if that's the case here, but the question stands.

As for not letting him pay the lower amount, this is pretty standard in the industry. Hosting providers have to pay for bandwidth too. They buy bandwidth at certain "commit levels" at certain pricing, but when a provider goes over their commit, they pay bandwidth overages to their providers. My point is that if you are a provider, and you have a customer who goes over their bandwidth allocation by 400%, you may be left holding the bag on bandwidth overages yourself, so giving your customer a discount may save that customer and may make everyone feel good, but overall it was a bad business decision; you just sold bandwidth at a loss and that's a poor idea that ultimately not only harms the business, but every other customer as a result.

Let's say instead of stealing bandwidth the intruder stole or deleted valuable data. Who would be responsible for that? OP of course.

Just because the intruder stole his hosting company's bandwidth doesn't make it their responsibility to eat the cost. 4,500GB in 7 days is about 50mbit/sec sustained. That's a HUGE amount of data, and should have an equally HUGE cost.

When you don't know what you're doing, running a server can be expensive. Let this be a lesson -- pay someone to secure your server if you're unable to do it yourself. It's much cheaper than other outcomes such as this.

@rambow681: Dude:

There is a plan they have that allows 4,00GB per month that costs $1,199 per month

They "sell" 4,000GB per month for $1200. There is NO FUCKING WAY it costs them almost $6k. None, zero, zilch. At this point, they just want to be greedy, when the customer is MORE than willing to pay for the ACTUAL COST of the bandwidth.

Even if he wanted the traffic or not, the ISP will pay for that bandwidth upstream, so he should pay for that traffic.

However, Servepath should be more lenient and allow him to at least pay the $1000 because that would cover their costs.

A datacenter usually buys 1 mbps of bandwidth in 1000 mbps commitments for around 7-10$ so the datacenter's costs would be around $7*50 per month or around $350-500.

Servepath should have given him an automated email after a day or two letting him know that based on his past traffic, he will get over the limits fast. If Mick is a good customer with a good history at them, after two days they should have given him at least a call or limit his port speed to 10mbps or some reasonable value so that he won't get over the limits so easily.

Also, Mick should carefully read the terms of service because some say that if there's some kind of DOS (denial of service) they must intervene and block that or limit it.

@nakedscience: They can offer plans of that size because most people won't use all 4,000GB of transfer. The 4,000GB plan also likely has a 12 month contract. The hosting company's costs are irrelevant, Mick has a certain plan with certain costs. Does it cost T-mobile 35 cents a minute to connect my calls when I run out of plan minutes? Nope, but that's what overage is all about. Though, to be fair to T-mobile, anytime I've gone over one call to them has gotten the overage removed.

@rambow681: also I'd like to point out that it seems like the hosting company wants to get rid of him as a customer. I certainly would want to be rid of him. He doesn't watch his server and it's not secured, and he doesn't want to pay. He's a liability.

@exploded: I agree - he wants to do that but they won't.

It's funny, I see people get furious when their server shuts them down for blowing their bandwidth but that's a lot better than getting a nearly $6k bill in my opinion.

@chrysrobyn:

While Dreamhost says you have a quota of 8.4 in reality you would probably not be even to do even 1TB of transfer. This is because your hosting account is on a server with about 2000 other hosting accounts, which all use the same 100mbps network connection to the internet.

Furthermore, the web server is speed-shaping each connection - you can see that as each download that's initiated tops to about 300-500KB/s even though you have a much higher Internet connection.

It's a classic case of overselling and they even admit it here.

Wow, maybe the salary I earn managing and securing my company's web servers is justifiable after all!

your equipment used it, you gotta pay for it. Those are the rules he most likely agreed to when he signed up. This is what you have to deal with when you want your own server. No one watches it but you.

This should be something that any good hosting company could avoid easily. By monitoring the traffic per port, you can view spikes and have software in place to notify your network techs when such spikes occur. You could have turned the port down, located the source of the compromise, correct or replace the solution and then bring it back online. Servepath is in the wrong here, they should have network abuse prevention already in place.

@jamesn1: Most data centers don't buy bandwidth as a function of the amount of data transferred but rather according to the "95th percentile rule." Any data center with an Ops Mgr with a pulse would have a system in place to make sure they don't cross that 95th percentile threshold otherwise they are just as negligent IMHO. I'm not saying the guy should get off scot-free, but what they are doing is not only outrageous but stupid. The data center I use offers a chance to upgrade to whatever package covers the overage and retroact the billing, but then again, they're smaller and not thieves....

Bandwidth is dirt cheap. Anyone who tries to tell you otherwise is just propagating a myth created by the jackasses behind metered billing and caps.

Web Host should have notified Mick. If they sell bandwidth and colo space, then they have the tools necessary to track usage. Its how they earn a paycheck, after all.

It sounds like both sides are at fault for different reasons:
Mick is responsible for his own server.

The colo is responsible for cutting him off before letting his overages skyrocket to unreasonable amounts. Also, denying the request to work with Mick on the bill is suspicious. Unless this isn't the first time its happened.

@rambow681: " They can offer plans of that size because most people won't use all 4,000GB of transfer."

Bullshit excuse.

"The 4,000GB plan also likely has a 12 month contract. "

So? Still costs the same.

@rambow681: zomg he made one mistake!

That is a bit much... I have a 1/2 cab and 20Mbps of quality tier bandwidth where I colo in Denver. I pay $1000 a month. If something is slightly off about my bandwidth usage, an email is sent to me to let me know things are off. Servepath needs to man up and give the guy a break.

@BrewMe: I can recommend NearlyFreeSpeech. They have a "pay as you go" bandwidth usage model.

What means is that if for some reason, your site starts using Gigs and Gigs of traffic, once your account is depleted of funds, the website is turned off, thus limiting your financial exposure.

(In my case, I deposit about $10 at a time, so the most I could lose in such a scenario is less than $10.)

I've been a happy customer of theirs for about 4 years.

@nakedscience: We don't know that he made one mistake. He never said "this is the first time this has happened." For all we know it's a chronic problem with him. His hosting company is being hard-line about it for a reason.

@Corporate_guy: it doesn't have to be.
you can rent a dedicated server but the hosting company will be managing it.

Wow, I work for a competitor as well, first off, Mick needs to look for a new provider, my company provides servers with 4TB of bandwidth starting at $189.00 per month, second and most unfortunate, he is most likely going to have to pay it.

@rambow681: Perhaps that reason is because they are douchebags? You've been here a while, yes? Companies like to be douchebags for no reason.

I work for a hosting company(not Servepath) and here's how we handle such situations:

By default our dedicated servers are "managed"(hosting company manages the server and has root access). All OS security, patches and configurations are our responsibility. Client has just regular user account where they can upload files, add/remove websites, whatever else they need to do. Client is responsible for their installed applications (wordpress, forums, gallery, whatever else might have installed).

We also offer "self-managed" dedicated servers. In this situation we provide clients with root access and surrender all server management responsibilities to them. Before the client gets root access they have to agree to a pretty detailed disclaimer which also covers the exact situation like Mick experienced.

If we detect an unusual activity on a server(abnormally high bandwidth or cpu usage) we investigate what's going on. In the case of "managed" servers if an exploit was caused by our fault(misconfigured server or some critical software is not up-to-date/patched) we notify the client and work with them on resolving this issue. We absorb any costs that would arise from this.

However, if a server was compromised because of client's installed software(didn't change default passwords, didn't properly configure file/directory permissions, did not remove exploitable "sample" scripts) then we contact the client and definitely present them with a bill, but we will also work with them on resolving this problem. Quiet often servers are getting exploited not because of the client's direct fault but because they use badly written software that has more security holes than a swiss cheese.

In the case of "self-managed" servers getting exploited, it's totally the client's responsibility no matter what caused the exploit. We'll push much stronger for them to pay whatever overages they incurred. However, even still we will work with them on an acceptable solution. If it's obvious that they are way over their head in the 1st place, we will absorb the overages on the condition that they will have to pay the full setup fee to reformat/reinstall their server and from that point on switch it to "managed" mode. If that client will want "self-managed" hosting, they'll have to go some place else.