Q: What do programmers and plumbers have in common?

A: Both make the world a better place in which to live.

I've been captivated by programming and software since I wrote my first spreadsheet formula back in the mid-eighties. Despite the complexity of languages to which I have evolved, for me, programming is a compositional act in which the lines of code are more akin to musical phrases than 'just-the-facts' morsels of an instruction set. That I can put them all together and make something that works is still pretty remarkable. That it turned out that I could make living from it is nothing short of amazing. That my code has made a difference for the better is the ultimate reward.

That’s the good news.

The bad news is that coding is anesthetic. When you are in the throes of getting your stuff to work, you are completely focused on the abstract game of logical pick-up-sticks that you’ve made for yourself. Nothing else is noticed. Nothing else matters. It’s just you and the code; no food, no sleep, no kids. The state of the universe is of little concern.

Ten years ago I was of the belief that if all the coders in the world understood the awesome power we held and could unite under a common vision, we could create a place completely free of the ills that plague mankind. Turns out that our common vision is acute and episodic. It takes a lot to provoke us, but once provoked, we do act, as shown by our recent reaction to SOPA and PIPA.

Still, at the end of it all, it seemed as if we considered ourselves to be more like civilization’s plumbers rather than its guardians.  I had more than casual contempt for programmers that spent a lifetime learning the ins and outs of new techniques and products for no other purpose than to enhance one’s technical prowess in order to be the smartest person in the room. I was mad that those less versed in the complexities of the coding life, civilians -- for lack of a better term -- were articulating the Big Vision for the Planet and getting the attention due to the people that actually made the world go ‘round. My thinking was that the only action that mattered was the Big Action, that a gesture that was not big enough to get the attention of CNBC, Twitter or Slashdot, was an effort wasted.

I was wrong.

Today I understand the power of humility. Today I understand that plumbers are the true guardians of civilization. Being able to keep the sewerage from contaminating the water supply is probably a first order indicator of a civilized society.

Everyday hundreds of thousands of people sit down in a front of computers to write code that makes the world work. True, some of those people try to make the world work not-so-well. But, for the most part, those slinging the bits are good folks, with good intentions. Does it matter that many are doing so in a dream-like dance between the human mind and the artificial? Dunno. I do like to think that there is a certain advantage to being aware of one’s self and one’s situation. But, ‘so what?’, if it's otherwise.  The important thing is that the code allows the information to flow, uncontaminated and continuously to everybody, because a civilization without water and without information is no civilization at all.

In Memory of Nick Statman

My friend, Nick Statman was buried today at Hillside Memorial Park nearby in Culver City, California. He was 59 years old, two weeks short of his sixtieth birthday. My wife and I attended the memorial service and his burial.

My connection to Nick started as a commercial relationship. Nick’s and my paycheck came from the same employer. Making money was the binding theme that defined the context of our relationship, at the onset anyway.

My relationship with Nick moved beyond the commercial quickly. One day, about 6 months into my tenure at the Day Job, Nick approached me and asked would my wife and I like to join him and his family—wife, brother, in-laws and cousins, for a Sunday’s Dim Sum in downtown LA. I accepted. I was glad that I did. I enjoyed myself thoroughly in the company of Nick and his family, sharing the pleasantries of a Sunday morning get together.

Another time, when Nick learned that my wife and I would be eating solo on Thanksgiving, he invited us to dine at his brother’s rather than go to a restaurant alone for our holiday meal.

Over the three years I spent getting to know Nick, he and I, and our wives, would spend more time together, mostly eating and letting our pets, his dogs and my dog, play together.

Nick was a kind, warm, somewhat cynical, wit. I never heard him utter a bad word about anybody. He never raised his voice. Nick was a generous man, an overall nice guy.  I never left his presence feeling angry or hurt.

Nick’s brother gave a thoughtful eulogy. He ended with a simple phrase, “He was my brother and I am proud of him.”  I have found it rare for so few, loving words to describe so fully the enormity of a life.

I imagine that if you did the math, you’d find out that most of us spend about half of our lifetime with and  around others in the pursuit of making money.  The odds are that we spend more time in the “workplace” then we do at home. Yet how much do we really know about those others with whom we labor for a paycheck? And more importantly, how much do we really care about them? How much do we really want to care?

It’s been my feeling for a long time that when this plane ride we call life takes its final approach and we come in for the landing, in the final analysis, we’re all going to want the same thing--to be around those we care about and those that care about us. It’s the rare person that wants to die alone.

Thus, the paradox: few of us want to be alone, but in that place where we go to make a buck everyday, when it comes to the important things, the things that really matter, many of us are alone and, for the most part, are unknown. It doesn’t have to be this way. Nick Statman’s life was living proof that the workplace could be different. He saw that a relationship could easily go beyond the commercial. He understood community.

Nick Statman was my friend. I am proud of him and all he held dear. I will miss him in ways words cannot describe. May his soul rest in peace.

Q: What was the biggest fashion trend at CES?

A: Earphones

I just got back from two days at  CES 2012 prowling from exhibit to exhibit, looking at the newest and coolest technology that the Captains of Industry have put forth to fatten the coffers of the World Economy. I found pencil thin flat-screen TV's the size of my wall, 3D video cameras priced at consumer purchasing levels, home appliances united together in electronic, domestic bliss, cars that are more like computers on wheels than basic transportation and a plethora of other gizmos that will keep the factories of the planet churning out products of delight into the supply chains of every Big Box and online retailer from Sheboygan to Shanghai. (Whew.....)

I also saw a hell of a lot of earphones. I mean, a HELL of a lot of earphones. They were coming from everywhere. Go down an aisle, there's an earphone manufacturer. Given that I went down a lot of aisles in the 1 million plus square feet of the Las Vegas Convention Center, I am betting that there were enough earphones present in the facility to adorn the ears of every head in Nevada and Utah, maybe Arizona too.

So, it got me to me to wondering, why? I mean, the only time you put on the earphones during my pot-smoking adolescence was so that you could play Led Zep at the recommended volume and not incur the wrath of your parents yelling down from the top of the basement stairs to, "turn down that noise you call music." Otherwise, we'd get together, throw the vinyl on the turntable, light up the joint, turn up the volume and pass the album cover around for collective and individual ogling as music came roaring through the loudspeakers. We wanted to hear the music together.

A lot has changed since that time. First of all, I haven't smoked pot since the Reagan Administration, despite the fact that Ronnie provided more than enough reason for me to want to puff my way into a state of dream-like oblivion. The 12" vinyl disc, with accompanying album-art cover, has gone the way of the typewriter, leaving in its stead song based music downloads for playing on an iPod.

For all the good stuff about it, the iPod is not about listening to music together. I haven't heard of many occasions where the kids are sneaking downstairs, iPod in hand, to listen to the Devil's Music in a state of group euphoria. Parent's aren't yelling at the kids to, "turn it down", because there's nothing to turn down.Yeah, you can hook an iPod into your home theater. But the primary listening device for the iPod is not the loudspeaker. It's the earphone and its scrawny kid brother, the earbud.

Yeah, the entry point for it all is still the same two holes that Nature has put on either side of your head. But, it's fundamentally a solo experience. Yes, you can share your playlists online to those that are interested. But, sharing a playlist socially does not necessarily translate in to a society of listeners. After all, there is a certain bond to be enjoyed knowing that all your friend's parents are yelling at them too, to "turn down that noise you call music."

So, there were a lot of earphones at CES 2012, too many probably. Half of the manufacturers most likely won't be in business next year. The competition is too stiff and the product is becoming more subject to the dynamics of fashion, a dynamic in which, if you have "trendsetter" status, you get a two year ride; less, if you're marketing a style for women.

Or, maybe it hasn't changed. Maybe kids hang around the schoolyard and bond over the complaint that their parents keep coming into the basement yelling, "why don't you kids take those damn things off and listen to that noise you call music, together."

Q: What happens when you put 1 and 1 together?

A: Depends on the type of personality you ask.

To my observation there are two types of people in the world. The first type conceptualizes that when you put 1 and 1 together, you get 2. The second type accommodates the notion that put 1 and 1 together, you get 11.

If you bang around a Meyers-Briggs lexicon of psychological types, you'll find that the first type is called a Sensing type. These folks rely upon the concrete facts to understand the world. The second type is called an Intuitive personality. For them, reality is a Big Picture in which the given experience is but an episode in a larger story.

For Sensing folks the role of language is to describe the world. For Intuitive folks the role of language is to express the world. Sensing folks ask, "How does it work?" Intuitive folks ask "Does it work?" For the Sensing folks, the beauty is in the balance sheet. For the Intuitive type, the beauty is.

When it comes to commerce, the Sensing type understands a very important fact: A business that does not make money, ain't. If the dollar amount attached to Accounts Receivable is not greater than the dollar amount of Accounts Payable, even the mightiest of businesses has a problem. And, if the problem goes on long enough, the business has a Big Problem. Profitability is acutely describable, always has been, always will be. As any Captain of Industry will tell you, profitability is best handled by Sensing folks.

Still, there is a whole world out there that cannot be described with words: explaining the color, red to a blind man or the sound of a chirping bird to one who is deaf. The only description is the experience itself. This is the world of the Intuitive. One just knows what it is.

Intuitive people experience a thing as part of a past, present and future whole. These are the people that see a ski and motorcycle and imagine a snowmobile. They see two cans tied together by a string and imagine computers talking to one another via an Ethernet cable. Give an Intuitive person a good cup of coffee, put him or her in front of sales force and you have a chain of Starbucks.

Businesses that last need both Sensing and Intuitive folks. Yet, it seems that as a business matures, the power bias moves toward the Sensing. Profitability, while essential, becomes paramount. Success is known more as a measurable quantity rather than a state of experience. Creativity, the de-facto realm of the Intuitive folk and the indescribable soul of an enterprise, becomes a transient commodity purchased from Ad Agencies that have a proven track record of staying on budget while increasing sales and market share. The transformation is not one of nefarious premeditation intent on sucking the life blood of originality from the corporate environment. Rather, to paraphrase an often used term, "it's just Sensing being Sensing."

So what's the point here? My point is this: If you are a reader that happens to have hire and fire power in your enterprise and also happens to be very good at understanding and defining profitability, when you look about your direct reports, how many people do you see that can accept that when you put 1 and 1 together, 11 is a plausible, if not useful result? If it's not half, you might be in some very real trouble.

Q: How many people can you fit into an Athenian taxi?

A: As many as possible

I am forever amazed by the number of cars in the world that sit around doing absolutely nothing. Take my car, for instance. It sits in my carport all day Saturday and Sunday doing absolutely nothing. Monday to Friday it sits deep in the bowels of an office complex, three levels down, away from the elements, for 8 hours a day, doing absolutely nothing. It is just sitting there existing through another day in eternity, because eternity is how long the machine will last if left unfettered in its absolutely doing nothingness.

So let's do some math.

Let's say the typical consumer, non-commercial, car owner uses his or her vehicle for 2 hours a day. This translates in to a daily capacity usage of 8%. The unused capacity is a whopping 92%. Or to put it another way, 1 person is using a resource that has the capacity to service 11 other people. It's almost like owning a 2400 sq. ft. house, but only living in one of the bathrooms.

Sort of a waste, I'd say. Yeah, you could make the argument that by not using the excess capacity, you're not having to pay money to support the resource, heating the entire house or keeping the gas tank full for 24 hours of automotive operation. But, we'll let this go for now.

The fact is that, for the most part, having a lot of cars sitting around doing absolutely nothing might not be good in the Big Picture. But, we're Americans. We seem to have a historical sense of having unlimited resource. In Europe this is not the case. They used up all the forests a long time ago. Efficiency counts: don't heat water in big ass tanks that go cold; heat the water intensively as it comes through the pipe to the shower head, which is what I experienced in an old house at the foot of the Pyrenees. Or pack as many riders into a taxi as you can, as I experienced in Athens.

Taxi drivers in Athens can pick up passengers in transit. It's the way they do business, nothing shady about it.

I remember sitting in the back of a cab, on my way to the airport and at each stop light the driver negotiated with would-be fares for the available seating next to me. If the driver could make the deal, the street corner traveler jumped in. The driver was continually revising his route given the destinations in play, mindful of keeping his gasoline consumption down. It's sort of brilliant in a way.

So back to the 8%. Imagine what it would be like if any given non-commercial automobile was operating at say, 70% capacity. First off, parking in LA would be a lot easier because most of the cars would in use and not sitting on the side of the street doing absolutely nothing.

So, how would it work? Well, instead of owning your own car, you'd belong to a "Car Association". Membership would get you access to the car you need, when you need it. For example, when it's time to leave for work in the morning, I'd go to my smartphone, sign into my Car Association App, and get the photo and license number of the car that I'd use to get to work. The car would be sitting in the parking lot outside the local Ralphs down the street. When I get to the car, I'd use my smartphone again to photograph a bar code on the car's windshield to verify that I was indeed at the car and then punch in my password. The App would have the smarts to unlock the car via a service such as OnStar. The key would be under the driver's seat.

I'd drive the car to work, to a Ralph's nearby or to a parking lot underneath my office complex. I'd leave the key under seat. Once I exit the car, I'd contact the Car Association to have the car locked and make it available to anther driver as soon as possible.

I'd pay a monthly fee to belong the the Car Association. The fee would vary according to the class of car I want, figure Standard Plan (Corolla, no older than 5 years) to Triple Platinum (BMW 700 series, no older than 2 years). Maybe one month I'd want to go Standard. If my in-laws are coming to town around the Holidays and I want to impress them, I'd go Triple Platinum. I'd have status on demand!

Gas and upkeep? If I take an Association car to the car wash or gas it up, I'd receive a credit against my monthly fee. Or the Association would pay contractors to maintain the properties, just as my Home Owners Association pays a contractor the keep the building grounds well groomed.

Of course, for this to work, we'd need to have a lot of cars and a lot of people in the Car Association. So it will probably be an urban thing. But, what the hell? There's already a lot of cars out there. They just happen to be sitting around doing absolutely nothing. From my point of view it's just a matter of the ability to think and act differently, a small nudge on the collective automotive consciousness.

Here's some good news: It's happening as I write. There are a number of Rent-Your-Car-Out services on the rise. Think of the these service as first generation Car Associations. You have Spride and Getaround which allows you to rent the cars of others starting at $5 an hour. And the technology is already in place. OnStar has just cut a deal with Relay Rides, to unlock cars for renters.

If Car Associations take hold, maybe via Facebook or Google+, it's only a matter of time before Hertz, Enterprise and National will want a piece of the pie providing the vehicles. Having a car will be about... well... huh.... using a car.

To my understanding the dining policy in the US Army's Boot Camp is this: You can have all the food you want. Just make sure you eat what you take.

So, all we have to do is map the thinking onto automobiles: You can have all the car you want. Just make sure you use what you take.

Q: Is it possible to own a car in good conscience?

A: Might be, might be

I find it funny in a way that for the last 5 years I have made my living coding around the automobile industry. I don't particularly like driving and each time I get behind the wheel I think of myself as an ecological liability. Yet, there I go driving around solo like most of the other people in LA, sucking up gas and throwing out carbon monoxide in a mechanical conglomeration of metal, glass, plastic and hand-crafted leather that will last longer than my time on Earth.

Owning a car causes me no end of moral dilemma. Yeah, I try to reduce my ecological footprint. I drive about 8 miles a day. I own a car manufactured in 2003. I rarely drive on the weekend, walking whenever I can, which might be de rigueur on the island of Manhattan, but is quite the oddity here in Tinsel Town. Yet, at the end of the day, I own and maintain close to two tons of environmental terror.

But, last week I learned something that offered me a morsel of hope that those of us driving on the highways of the planet can actually do so without bringing Mother Nature to her knees.

There is a little doubt the the electric car is here to stay. Just about all the major manufacturers are pumping out battery powered vehicles. And, new manufacturers such as Tesla and Coda are completely dedicated to the electric car space. So for me, it is not very hard to image a future 10 years from now when there are 50 million electric vehicles traveling the highways and byways of the nation. And, each of these puppies is going to require charging everyday. That's a lot of demand to drop on the power generation capabilities of America. That's the bad news.

The good news is that our national power grid has gotten pretty good at figuring out how to provide large amounts of electricity on demand to consumers everywhere, without putting the entire country into a state of anticipatory anxiety due to episodic blackouts.

Right now many houses have two power meters. One for the usual household stuff and the other for the central air conditioner. The reason for this separation is that having the air conditioner on a separate meter allows the Power Company to effectively monitor and control the device. In other words, the Power Company knows your air conditioner, it knows how much electricity it is using, has used and is likely to use. Also, the Power Company has the ability to turn the air conditioner on and off. So, when it's noontime on a hot July day in Topeka, Kansas and every air conditioner in the area wants to go on, machine intelligence at the Power Company can assess the situation and say, "well, we have so much electricity to go around and all these air conditioners want a part of it. Let's see if we can turn some of the air conditioners off for 10 minutes and allow the ones left on to get the cooling going. Then, we'll turn off those that are running and turn on the dormant ones. The humans will never notice a thing."

Great technology. So what does this have to do with cars?

This is what it has to do with cars: Most modern cars are digitally identifiable. For example, if you have a car with OnStar installed, the company will send you an email each month that reports the state of your car--do you need an oil change? Do you need to inflate your tires, etc...? OnStar technology talks to your car's on-board computing system and sends the email.

When it comes to electric cars, it will be possible to pass certain information about you and the car onto the Power Company: What is your customer profile? Where is the car being charge? What type of car you have? What is your current battery state? When do you need the vehicle fully charged?

Based on this information, the Power Company can figure out demand in terms of the Big Picture: "We know we have to charge 50 million cars in the next 8 hours. Given our super duper computing capabilities, we can figure out the most efficient way to charge all these cars to the specification of all these customers."

It's the typical win-win: the consumer gets another day of driving bliss; Mother Nature remains chaste.

I gotta tell ya, for the longest time I felt as if we are driving our way to vehicular Armageddon, without a care in the world about the effect that our 300 horse-power, fuel-injected beasts have on others and our planet. But, when I learn about something like the ability to power all the cars in the world using smart supply, I sleep better at night. Maybe this technology thing works, just maybe.

Q: How much data is Big Data?

A: More than a single human being can type in a lifetime

(PS: This post is like a box of Cracker Jacks. There's a prize at the end.)

Recently I've developed more than a passing interest in working with Big Data. I made the obligatory pilgrimage to Hadoopsville: got a little bit of HDFS and MapReduce under my belt, taught a few intro classes. I even did an interview with Doug Cutting.

Then, about a month ago a colleague at the Day Job suggested that I take a look at MongoDB as a data store for a project I was imagining. So, I did.

As I began to piddle around the innards and start to work with the on-demand, distributed features of the product, I found myself in need of a lot of structured data, to do some advanced piddling, about 45 gigs worth.

(I wanted to make it so that I was performing operations that exceeded the memory capacity of any one machine, or any two machines for that matter.)

As I said before, I'm interested in Big Data. So allow me a moment to share with you just how big, big is. The sort of structured data I figured to use is the typical log file entry, about 100 characters long, which looks like this:

20204,2011-11-11 19:19:07.123,Leslee,Falleti,777-64-9738,625 Orange Terr.,Suite 87,MT,59430,16413,A

The structure of the line above is:

unique_id,datetime,firstname,lastname,ssn,address_1,address_2,state,zip,a_random_number,a_random_character

I figure a structured log entry like the above will provide a lot of flexibility to do some MongoDB indexing, run some Map-Reduce analysis and do some benchmarking comparison between Hadoop and MongoDB.

So let's take a look at what 45 gigs of log data looks like. Say I want to make a ~1 MB text file full of unique log entries. Such a file will contain about ~10,000 lines of text. Again, this is for ~1 MB file.

45 gigs translates into forty-five thousand 1 MB files.

Now, this is not the type of data that you keep around on a hard drive next to the pictures of your dog and grandkids. In fact, getting your hands on this sort of data is kinda hard.

So, I figured I'd make some.

I could type out 450,000,000 lines of structured data at a rate of a line every thirty seconds, which translates to 120 lines a hour, Thus, I can get out 2400 lines in a day, provided I forsake sleep and go for twenty hours without mistake. But, being the world's worst typist, we're more likely looking at a lot less than the optimal. Anyway, typing as required, I could do ~12,000 lines during a work week, which means that I could get my 45 gigs of data in say, 40 thousand days, which translates into ~123 years.

Not an option.

Or, I could create a website and get a grant from the Federal Government as a jobs initiative to get everybody in the US to go to a web site I've made and enter some data according to the structure I need. Actually, upon reflection I would need every man, woman and child to enter data over 150 times.

Again, not an option. Government cutbacks are rampant and few innocent bystanders have the patience to do a structured task 150 times.

So, I decided to automate.

I made a cute little project in Java under Maven. The program uses an address and name randomizer I made a few years ago. You can download a zip file containing the project--data generator and randomizer all-- here. (There is also a Post Office module included that is more relevant to the article that describes the randomizer.)

One of the modules, DataGen, kicks off files of unique log entries. You can set it to kick off a variable amount a files. You can kick off 10 files of 100 bytes of unique log entries. You can set the DataGen to kick off 1000 files of 1000 bytes of unique entries.

In terms of rate, DataGen presently kicks off a ~1 MB file of unique log entries of random names, addresses and US zip codes taken from some predefined lists loaded into memory, in about 2 minutes.

I configured DataGen to kick off 45,000, ~1 MB files of log entries. I started running the generation last night on my Ubuntu laptop. So in about 2 months I'll have the 45 gigs I need.

Sorta makes possible the notion that if you chained a chimp to a typewriter for eternity, eventually the beast would produce Hamlet.