There’s a story we hear far too often: someone is buying a house. Before they put any money down, they do their research. They call the local cable/Internet provider to make sure they can get broadband service at this new address. They double-check. They triple-check. They search the property for wires, call back, and make sure they’ll be okay. Then they take out the mortgage, move in, and… surprise! There’s no broadband service after all, there won’t be any, and now they’re up a very expensive creek.

This problem isn’t relegated to one ISP or one region of the country. In the last year, we’ve brought you stories about consumers caught in this trap with Comcast and with AT&T, in two different states.

It might feel like there’s a legion of evil customer service representatives out there at ISP call centers just waiting for a chance to tell bald-faced lies to would-be customers, but as tempting as it is to believe that, the problem is not with the humans. Instead, CSRs and consumers alike are all pulling from one pool of data.

Whether you call an 800-number or check an ISP’s website, you’re checking an address against a single database and are effectively being told, “Yes, there is service here,” when in reality, there may not be.

That database suffers from the classic, timeless data-management problem of “garbage-in, garbage-out” — they can’t give accurate info if they don’t get it to begin with. And short of going out to look at every property in a region and measuring its distance from existing infrastructure, there is no real way to know which data you should keep, and which is junk.

So why is the information wrong, why does it keep happening, and is there anything consumes and ISPs can do?

For those answers, we need to delve a bit into the history of cable itself, and decades of regulation.

Once upon a time, television signals came to viewers over the air via antennas. But some viewers couldn’t access over-the-air signals, and so the idea of transmitting TV to them through wires — cables — was born. It turned out to be a pretty neat and lucrative idea, so it grew.

But growth gets messy… quickly. Companies were springing up that needed access to infrastructure. And consumers needed guarantees of some kind of reasonably fair environment. Throughout the ’60s and ’70s the FCC took piecemeal actions, but in 1984 Congress stepped in and passed the Cable Communications Policy Act of 1984 (the Cable Act).

The full Act (PDF), which became Title VI of the Communications Act, is about as dense as you’d expect. But, as we’ve covered before, the most important part of it for our purposes is the definition and delegation of franchise authority.

Thirty years ago, cable television was a profoundly local business. Municipalities — cities, towns, or sometimes counties — were the entities that entered into agreements with cable companies.

The entire infrastructure for cable was new at the time. Providers created networks of cable lines, either strung along telephone poles or run underground, and incurred high costs building out this brand-new system.

So franchise agreements protected business: not only did they give municipalities the right to yea or nay who could build in their backyards, but they also granted public rights-of-way access to those new companies, and, in a number of cases, effectively created a monopoly for the local market.

Building out a whole town might cost a few million bucks, but if you were going to be the sole provider inside the city limits, you could make the money back — and then some.

But those agreements also had a trade-off: if a company was going to be permitted to climb poles and dig ditches and charge residents, then they would have to serve all the residents, or near to it. Franchise agreements frequently used language like “shall serve,” and “shall be required,” which translated from the legalese loosely means, “you do this or else.”

The franchise agreement samples below — spanning from 1990 to 2005, in California, Virginia, and Massachusetts — are just three of tens of thousands of examples of the type of agreements into which municipalities and cable companies regularly entered:

This slideshow requires JavaScript.

In the three decades since passing the Cable Act, all but the most rural areas of the country have long since had their cable infrastructure built out.

And as cable technology grew to include data connectivity in the last 15 years, the law has changed to keep pace.

From its invention in the late 1940s all the way on through to the early 1990s, if you had cable, you had, well, cable. Even with the introduction of residential satellite TV, pay-TV was still just that.

But the turn of the 21st century is when things really started to get hairy. Cable companies and telephone companies were suddenly meeting in the middle, and that middle was data. By 2005, virtually all cable companies were offering customers a cable modem to access the internet at higher-speeds than your copper-wire dial-up or DSL.

Meanwhile, phone companies were going the other way. They always ran wires to everyone’s houses, but for decades could only offer voice service. But by 2005 your copper-wire phone company could not only offer you DSL, but also was expanding into fiber — and with fiber came not just internet access, but “cable” TV.

The Comcasts and Time Warner Cables of the world had longstanding franchise agreements with cities and towns around the country that not only gave them access to residents but also to infrastructure.

Verizon and AT&T, respectively launching their competing FiOS and U-verse services, did not — but wanted in. And the political winds at the time strongly favored killing off government regulations and opening up a free market wherever possible.

So when Verizon and AT&T came knocking, a new era of regulation was born. All of the cable and telecom companies that provide broadband access and linear programming (channels you can surf) still have to secure franchise agreements with a region before they can operate there. But the authority has shifted, and so too have the obligations.

In 2005, Texas became the first state to shake things up. Instead of having to go individually to every town, city, and county in the Lone Star State seeking a certificate of franchise, a cable company wishing to do business would go straight to the state’s Public Utility Commission to apply for a franchise. Over the next year, Indiana, Kansas, South Carolina, New Jersey, North Carolina, and California followed suit.

Then, in December of 2006, the FCC voted to adopt a rule (100-page PDF) that modified the process of cable franchise licensing.

As Ars Technica aptly explained way back then, Verizon and AT&T complained that the process of getting all these agreements set up was too onerous, too expensive, and too likely to tip off the competition. They refused to comply with any build-out agreements — those clauses that said, “you must serve X many residents” — and asked the FCC to tell the municipalities to knock it off.

And basically, the FCC did. The rule stipulated new conditions applied to local franchising authorities: they had to grant licenses faster and charge less for them. And most critically, the FCC found that “failing to grant a franchise when an applicant did not agree to unreasonable build-out mandates” meant a municipality was acting unlawfully.

The rule also applies very specifically to municipal-level franchise-granters. Just this year, the FCC responded to petitions asking them to revisit the rule, and confirmed (PDF) that the requirements in it apply to local, but not to state-level, franchising authorities.)

The FCC made the change in the first place in order to spur competition — which, they found, was rendered nonexistent in large part due to the history of the franchising process — and increase broadband adoption:

The dearth of competition is due, at least in part, to the franchising process.58 The record demonstrates that the current operation of the franchising process unreasonably prevents or, at a minimum, unduly delays potential cable competitors from entering the MVPD market. Numerous commenters have adduced evidence that the current operation of the franchising process constitutes an unreasonable barrier to entry. Regulatory restrictions and conditions on entry shield incumbents from competition and are associated with various economic inefficiencies, such as reduced innovation and distorted consumer choices.

If you lower the barriers to entry, the thought went, more competitors will enter. And if that just so happens to make it easier for some large businesses with heavy-hitting lobbyists, well, win-win all around.

Over the last decade, many consumers have seen some benefit from the expansion of AT&T and Verizon’s services, and new entrants like Google and CenturyLink joining in to the residential fiber game. But high tensions still exist between incumbent providers and new entrants, as we saw just this month in Minneapolis.

Minneapolis residents have been subject to a Comcast monopoly for high-speed broadband service for years. The city has recently negotiated a new agreement with CenturyLink that would bring competition to town.

Comcast, which has enjoyed not having to compete, is not pleased with the difference in franchise agreements. “We are disappointed that CenturyLink is not being held to the same terms and conditions as our existing agreement with the city, contrary to Minnesota state law,” a Comcast exec said at the time.

The existing franchise agreement between Minneapolis and Comcast (PDF) originated long before the regulatory change and was last reviewed in 2009, to run through 2021. It does clearly specify not only which residents Comcast may serve, but which residents they are required to serve:

[Comcast] shall make Cable Service available to every residential dwelling unit within the Franchise Area where the minimum density is at least thirty (30) dwelling units per mile and is within one (1) mile of the existing Cable System. Subject to the density requirement, [Comcast] shall offer Cable Service to all new homes or previously unserved homes located within 125 feet of the Grantee’s distribution cable. … This Franchise is granted for the entire corporate boundaries of the City as of the Effective Date … [Comcast] shall provide Cable Service to the entire franchise area.

So, Comcast is required to build to any willing customer who is within the city limits, within a certain distance of the existing infrastructure, and in a neighborhood of a certain minimum density, even if the house is in a new subdivision or has never had cable before.

But the brand-new CenturyLink agreement is different. Their franchise agreement (PDF) is about meeting certain minimum deployment targets: making their service available to a certain percentage of residents per year, serving every ward of the city, and especially making service available to low-income households.

On the surface, none of that sounds too bad. In general, the more options a consumer has, the better off she is — and goodness knows, we here at Consumerist are all in favor of increased broadband competition. But the franchising shift also came with some side-effects.

The changes to regulation over the past decade haven’t changed cable and telecom service areas… but they have changed the approaches to how those areas are served. And that brings us back to the original problem: an ISP thinking they serve your house, when in reality they don’t.

The spread of state-level franchise laws, and the change to what municipalities are allowed to require, has led to a shift. Those agreements that used to say a provider must provide service to residents within the county or city line are now increasingly saying that a provider may provide service to those residents.

The former is a guarantee on behalf of residents that a private entity is coming in with a service. The latter is permission to a company to gather customers as they will. That change is one half of the problem.

There are over 315 million people in the United States, living in millions of houses and condos and apartments in over 19,000 incorporated municipalities. That is an absolutely eye-crossing amount of data. Luckily, here in the 21st century we have computers, which can easily store that wealth of data, in big databases that are easy to buy, sell, and rent access to. And that’s the other half of the problem.

If you’re creating an in-house database of “addresses we can serve,” the easiest thing to do is to get a list of all the addresses inside a certain set of ZIP codes, or inside the boundaries of a certain city or county, and just import it.

When you are required to serve any address within the entire boundaries of a municipality, the list of “addresses we can serve” and “addresses inside this ZIP code” will be a Venn diagram with 100% overlap — a big ol’ circle.

But when you have the option to serve any address within the boundaries of a municipality, and you’ve reached most but not all of them, that Venn diagram suddenly reverts to being two distinct circles again. Some residents are going to end up caught on the wrong side… and nobody will realize it until someone on that fringe calls in for service, is told they can get it, and discovers they can’t.

That’s what makes the problem such a challenge for consumers. You can get a super detailed map of franchised territories from the state of California, but you can’t get a guarantee that living in one of those shaded areas will actually get you the service nominally provided.

But there are ways to find out, at least, if you’re likely to be subject to that problem.

Because all of these agreements vary widely from place to place, it’s really important to hone in on the specific rules in the place where you live or plan to move.

1. Start with your state.

Many, but not all, now grant state-level cable franchises. Your state may be on this list. You can also try searching for [state] cable franchise authority and [state] public utility commission to get to the relevant administrative site.

Once there, search for cable, video, or telecommunications regulations, or search the site for the term “franchise.” If they issue state-level franchises, you’ll find breadcrumbs — and you’ll know that it’s almost certain that the ISP you’re wondering about, in the neighborhood you’re wondering about, doesn’t have to serve 100% of addresses in its service area.

2. Then, go local.

Municipalities’ web presence varies in its usefulness. Some are great, and some are awful — but there’s no way to know without looking. So if your state delegates cable franchising authority to the municipalities, you’ll want to hit Google with [county] cable franchise agreement and [city or town] cable franchise agreement too.

If you can’t access a copy of existing local franchise agreements online, call the town or city administrators and ask humans how to get one. And when you’ve got your hands on one, you want to look for clauses about “area served” or “service area boundaries.” That’s where you’ll find out if the ISP is required to bring service to your address or not.

3. Get good data.

Many franchise agreements contain very specific wording about number of feet from a trunk line a company has to wire, or whether there’s an out for multi-unit buildings or dwellings in insufficiently dense neighborhoods. If you’re thinking of buying a house set back from the road, measure that drive! And do some quick math, if you can, on how many neighbors you have per square mile.

It stinks that companies can’t or won’t always give you truthful and correct answers when you call and ask them. But if you have any doubts, you can at least try to make regulation, as dense as it is, work for you.

Want more consumer news? Visit our parent organization, Consumer Reports, for the latest on scams, recalls, and other consumer issues.