How many times have you been asked “May I have your ZIP code?” when paying with a credit card? Many people just assume it’s for security purposes, but in reality it’s more likely that you may have just given marketers access to a wealth of knowledge about you and your shopping habits.

CNN Money’s Melanie Hicken takes a look at controversial data brokers like Acxiom. Don’t be fooled by the idiotic name; it is a big-time player in the consumer information industry, along with CoreLogic and Datalogix, all of which track shopping information and maintain databases containing data like age, marital status, education, party affiliation, and income.

In contrast to the wealth of information available about consumers, these companies often need only two pieces of information to keep track of you — your name and ZIP code. If you’re Mary Smith and live in a densely populated city, that might not be sufficient, but the director of policy and advocacy at the Privacy Rights Clearinghouse tells CNN that, “For the majority of the country, the ZIP code is going to be the piece of the puzzle that is going to enable a merchant to identify you.”

If you’re paying by credit card, the store — and whichever broker it sells its information to — has already captured your name just by swiping it. Once the ZIP is added — presuming you don’t move every few months — every place you store you provide that same information to is another note in your file.

And it’s not just about where and what you buy. Marketers can look at purchasing trends to predict what you may be interested in buying. Even if you never tell anyone you’re expecting, those maternity clothes you buy could result in retailers and marketers knowing about your baby before you’ve even told your folks.

“Some of these data brokers know us better than we know ourselves,” said Pam Dixon, executive director of the World Privacy Forum.

These brokers also get into areas that some may consider to be off-bounds to marketers. For example, while they can not legally access your medical records, companies like Acxiom can track what over-the-counter medicines and medical supplies you purchase. They can also track your online browsing behavior, so you may be revealing more to data brokers than you do to your family, friends, and maybe even your physician.

While Acxiom claims it doesn’t track “sensitive” health issues like STDs, we are sure may people would not feel comfortable knowing that some database is aware you have something as simple as a cold.

In March, the Massachusetts Supreme Court ruled that retailers in the Bay State can not require ZIP codes when making credit card purchases. The highest court in California made a similar ruling in 2011.

Last fall, Datalogix partnered with Facebook, giving rise to concerns about users’ privacy on the social networking service. In December, the Federal Trade Commission announced it was looking into the variety of information collected (and methods of collecting it) by Datalogix, Acxiom, Corelogic, eBureau, ID Analytics, Intelius, Peekyou, Rapleaf, and Recorded Future.

Many of these brokers do offer some method of opting out. You can read about Acxiom’s opt-out procedure and policy here. The Datalogix privacy page has a form for opting out — it’s in the “Choice” section.

Want more consumer news? Visit our parent organization, Consumer Reports, for the latest on scams, recalls, and other consumer issues.