Q: What is a wonder of the modern world?
A: Data warehouses
Over the last few months I’ve been working in the Land of Big Data: data warehousing. This stuff is new to me. Previously I treated data as a necessary evil of application development, thankfully abstracted from my coding landscape as nothing more than a call to some stored procedure that returned the relevant dataset of interest. Not a big deal given that the usual size of the dataset returned was a few hundred rows at most.
That was then and this is now.
Now my landscape is filled datasets the contain millions of rows. And, most of these rows are filled with pretty standard stuff—first name, last name, address, city, state, zip, stuff the person in the row bought and when they bought it—things that are pretty usual in the everyday data warehouse of a large company that has millions of customers or wants millions of customers.
Yet it turns out that not only is my landscape filled with the names and addresses of millions of people, but we also know a lot of tasty tidbits about each one: average age of each person, how many people are in the household, what type car he or she drives, what type of house he or she lives in, marital status, age of children in household, favorite grocery store and I image if we added a bit of effort to our inquiries, we could find out sexual orientation, frequency of sexual activity and if the person prefers half gallon size containers of milk as opposed to gallon size containers. It’s that comprehensive.
Entire companies such as Information Resources, Experian and A.C. Nielsen are built on the buying and selling of millions of rows of data that tells us a lot of stuff about a lot of people. Sort of freaks me out. You’d think that the Average Joe would be all up in arms about this for reasons of invasion of privacy, violation of family values or mass manipulation of the national consciousness by Big Corporations only concerned with getting the general population to forego buying Tylenol in favor of Advil because adults between the ages of 35 and 65 prefer the color blue over the color red.
But no such indignation exists about the fact that so much knowledge about the “collective us” is available to any company that has the money, technical infrastructure and analytic know-how to facilitate the inquiry. We sort of like it this way.
The mass exposure of a private individual's information to corporations is not being done to us, but by us.
In the old days, if you wanted to know a whole lot about a large population, you’d create a society filled with political informers and secret police, which resulted in kids reporting the political activities of their parents and workers ratting on each other. And, if that didn’t work, you’d put a gun to some person’s head and tell them that if they didn’t spill the beans on everybody that they knew, you’d shoot them.
But again, that was then and this now.
Today no such intrigue and violence is necessary. Just give each member of the population a credit card with a line of credit appropriate to the holder’s income level and credit score. And, on the paper that gets signed to accept and activate the card you put a little language that says, “Oh yeah, we get to sell information about you and your purchases to any interested buyer that can pay for it, whether headquartered in Cincinnati or Beijing”.
Voluntary totalitarianism. Who woulda figured?
Over the last few months I’ve been working in the Land of Big Data: data warehousing. This stuff is new to me. Previously I treated data as a necessary evil of application development, thankfully abstracted from my coding landscape as nothing more than a call to some stored procedure that returned the relevant dataset of interest. Not a big deal given that the usual size of the dataset returned was a few hundred rows at most.
That was then and this is now.
Now my landscape is filled datasets the contain millions of rows. And, most of these rows are filled with pretty standard stuff—first name, last name, address, city, state, zip, stuff the person in the row bought and when they bought it—things that are pretty usual in the everyday data warehouse of a large company that has millions of customers or wants millions of customers.
Yet it turns out that not only is my landscape filled with the names and addresses of millions of people, but we also know a lot of tasty tidbits about each one: average age of each person, how many people are in the household, what type car he or she drives, what type of house he or she lives in, marital status, age of children in household, favorite grocery store and I image if we added a bit of effort to our inquiries, we could find out sexual orientation, frequency of sexual activity and if the person prefers half gallon size containers of milk as opposed to gallon size containers. It’s that comprehensive.
Entire companies such as Information Resources, Experian and A.C. Nielsen are built on the buying and selling of millions of rows of data that tells us a lot of stuff about a lot of people. Sort of freaks me out. You’d think that the Average Joe would be all up in arms about this for reasons of invasion of privacy, violation of family values or mass manipulation of the national consciousness by Big Corporations only concerned with getting the general population to forego buying Tylenol in favor of Advil because adults between the ages of 35 and 65 prefer the color blue over the color red.
But no such indignation exists about the fact that so much knowledge about the “collective us” is available to any company that has the money, technical infrastructure and analytic know-how to facilitate the inquiry. We sort of like it this way.
The mass exposure of a private individual's information to corporations is not being done to us, but by us.
In the old days, if you wanted to know a whole lot about a large population, you’d create a society filled with political informers and secret police, which resulted in kids reporting the political activities of their parents and workers ratting on each other. And, if that didn’t work, you’d put a gun to some person’s head and tell them that if they didn’t spill the beans on everybody that they knew, you’d shoot them.
But again, that was then and this now.
Today no such intrigue and violence is necessary. Just give each member of the population a credit card with a line of credit appropriate to the holder’s income level and credit score. And, on the paper that gets signed to accept and activate the card you put a little language that says, “Oh yeah, we get to sell information about you and your purchases to any interested buyer that can pay for it, whether headquartered in Cincinnati or Beijing”.
Voluntary totalitarianism. Who woulda figured?
posted by Bob Reselman at 1:17 PM
0 Comments:
Post a Comment
<< Home