I’m mad at GE.

You know, General Electric. The company that makes jet engines, dishwashers, locomotives, and medical scanners. The company that brought us Jack Welch and The GE Way (not to be confused with The HP Way).

I had a problem with my new GE dishwasher and needed it repaired. So I called the retail store where I bought it and told them my tale of woe. They said they don’t actually service the appliances they sell (no surprise there), but, since they already had all my information, they’d be happy to contact GE on my behalf and set up a service visit. With my permission, they passed along my name and address, model and serial number of the dishwasher, the date of sale, and various other tidbits of information, and promised that an official GE representative would call me to schedule an appointment.

Sweet. Even though the retail store didn’t do service, they did a nice job of handling my problem on the first call. They could have easily said, “Sorry, buddy, not our area of expertise. Youse gotta call GE yerself. Have a nice day.” So bonus points to the retailer. I’ll definitely use them again.

All is in readiness. I wait for the call from GE. Nothing. A week goes by, and I’ve pretty much forgotten about the problem.

Until 7:00 AM. My sleeping companion and I are abruptly awakened by the telephone. It’s an automated robo-call informing us that “The service technician you requested is on his way to your location. We were unable to reach you by telephone to schedule an appointment, so one has been scheduled for your convenience. The serviceman will be there shortly.” Click.

What, now?! It’s 7 o’clock in the effing morning! How shortly are we talking? And what’s this nonsense about being “unable to reach me by phone?” I’ve gotten no calls from GE in the past week and there are no messages on my answering machine or voicemail. And forgive me for noticing, but didn’t you just call me by phone right now? What is the matter with these people?

Panic ensues. Quick, quick, hurry, hurry. You know the drill—jump out of bed, throw on some clothes, hide the drugs, chain up the dogs, wipe down the blood, disarm the traps—the usual preparations for company. I was running around like Hugh Grant in the first minutes of Four Weddings and a Funeral.

Sure enough, the GE Repair truck pulls up in front of the house just as I’m stashing the last of the liquor bottles in the bathtub.

To their credit, it was a genuine GE truck, too, not “Clem’s Appliance Repair and Fishing Tackle,” like I’d expected. The repairman himself had on a clean uniform with his name and the GE logo embroidered on the chest. I know you’re supposed to ask for ID before letting people into the house, but after all, they did tell me he was coming, didn’t they?

Nice guy, too. Once I guided him to the dishwasher he got straight to work, a man on a mission. All competence and proficiency, he got down to business and fixed the dishwasher faster than a Formula One pit crew can change a set of tires. He even printed out a little receipt from his portable wireless printer (slick; now I want one) and handed it to me, pointing out that the entire service call was covered under warranty and would cost me nothing. Wipe down the counters; have a nice day, I’ll be going now. In short, the repairman was great. I might name my next kid after him.

Then came the follow-up call from Corporate.

Again it’s an automated robo-call, this time to tell me they’d love to hear my honest feedback regarding the recent service visit. Boy, I’ve been waiting for this. Now’s my chance to provide a little “constructive criticism.” Your repairman was great but your scheduling protocol sucks. “Dance 10, Looks 3.”

Except they don’t ask me that. Instead, they spit out exactly five canned questions that I’m supposed to rate on a scale from 1 to 10. How satisfied was I with the repair itself? Uh, 10. How satisfied was I with the courtesy and professionalism of the service technician? Well, 10 again. Wait, wait! You’re not asking me the right questions! Ask me about the nonexistent scheduling! Ask me about the early morning wake-up call! You bastards promised!

No joy. After answering, truthfully, that their repairman in his shiny white van was great, GE hung up on me, evidently satisfied they’d collected all the data they wanted. There was no opportunity for additional comments; no option for write-in remarks nor openings for spontaneous feedback. It was strict multiple-choice Q&A. They asked what they wanted to know, not what I wanted to tell them. A failure in a closed-loop system.

So—how close is your organization to GE? How does your department conduct research, and how does your support group quantify customer satisfaction? You may find that the foxes are guarding the henhouse, in the sense that the group responsible for providing a service may also be creating its own metrics. That’s a conflict of interest that should be obvious, but, sadly, it is pretty common.

As engineers, programmers, or developers, we like to have data. We like numbers; we want answers. There’s a creative side to engineering, too, but the numbers and research need to back it up. The insidious problem is when you don’t ask the right questions, as with GE. On a technical level, their automated scheduling software and voice synthesis were probably judged a complete success. It just didn’t work in any practical sense. The same goes for their follow-up survey. As far as they know, they got a perfect score, and some mid-level manager in GE’s service department will likely get his quarterly bonus, smugly satisfied that he’s done a good job. There’s a saying in the restaurant business that you always know why customers come back but you never know why they don’t.

The key to scientific research is in knowing how to test a hypothesis, and the trick to data-collection is in knowing what questions to ask. It’s trivially easy to put together an online survey, and almost as easy to get a thousand people to fill it out. But gleaning any data, knowledge, or (hardest of all) wisdom from that is another matter entirely. Armed with bogus data, you can blindly but confidently stride down the wrong path. Basketball player Yao Ming and I have an average height of 6’ 8”, but that’s a useless statistic no matter how accurate.

For gaining real insight, there’s no substitute for face-to-face conversations. Yes, it’s time-consuming and requires human interaction, but it’s far more useful than even a well-prepared survey. And most surveys are not well-prepared. There’s a real skill to developing accurate surveys and questionnaires, and most of us don’t have it. Neilson, Arbitron, Gartner… these companies know what they’re doing, and, frankly, we don’t.

The very nature of surveys constrains the range of possible answers, as with GE’s example. I couldn’t tell them what I really thought—what they needed to know—because they never offered the opportunity. I’m sure GE thought they were covering all their bases, but they really just asked the same question five different ways. Discovery and insight require leaving the door open for unexpected answers to questions you hadn’t thought of asking. Ranking satisfaction from 1 to 10 won’t tell you that your customers really prefer strawberry to vanilla, or that they’re about to abandon you for a competitor.

No amount of surveying would have sparked the development of the iPod or Linux or booger-flavored Jelly Belly candy. Those were wild shots in the dark, unencumbered by facts and statistics and survey data. Business guru Peter Drucker said, “Don’t solve problems; pursue opportunities.” He wasn’t working for GE at the time.