Beware the "Black Box"

Throughout the course of my career I have developed hundreds of analytical models that have been or are being used by many of the banks, credit issuers and direct marketing companies in the country. Statistical modeling is not a topic that I typically bring up at cocktail parties or in general conversation - as it it is the equivalent of administering a few doses of vicodin mixed witha shot of bourbon....zzzzzzzzzzzzzzzzz.

However - there is one topic of modeling that has interested me for many years and should be of interest to the general model consuming public. That topic is that of the "Black Box" approach to selling model based solutions.

Back in the 90's when modeling solutions were coming out of the woodwork, a variety of companies begin marketing their models and scores as their differentiator. The typical tagline was that their solutions were superior because their analytics and analytic teams were "better".

A prime example of this was the Falcon product from HNC which today still is the modeling platform for the majority of all card transactions world-wide. HNC brought neural networks to the industry and they positioned their solution as superior because of those deep analytics. And guess what - it worked! The positioning catapulted their solution to the top and they never looked back. However - if you ever asked HNC what was behind the modeling, they would tell you it was a "Black Box". That is, it was the secret sauce and could not be divulged.

A couple years later, I was presented with a similar situation. I was jointly developing a model with the old Fair Isaac (now FICO). While it was supposed to be a joint effort, when I asked to review the models and codes, their lead analyst informed me that he could not divulge. I asked why not, and he told me that it was a "Black Box", could not be divulged and I probably could not understand anyway. As a trained statistician, all I heard was - "We're smart and you are stupid".

That is when I began to become fixated on the implications of the "Black Box" approach to marketing scoring based solutions. At the time, FICO positioned themselves as the "Gold Standard" in modeling. However, their prospects, clients and competition were referring to them as the "Four Letter F Word". I do think that over time, this has led to their market erosion and loss of credibility.

Most of the people I have ever met from HNC or FICO are, indeed, very smart people. However, when you take the "Black Box" approach to positioning scoring solutions - I think you are jeopardizing your brand and doing a disfavor to your people. After repeatedly hearing "We're smart - you're stupid", eventually people grow tired of it. After all - I don't think anyone likes to be told that they're stupid or incapable of understanding. Especially when it is not true. There are tens of thousands of modelers out there who know exactly what I am talking about.

I have always been a firm believer in transparency in the modeling and scoring process. As solutions providers, we exist to solve problems and provide value. When we throw up the "Black box" it undercuts our value proposition and opportunity to be collaborators. That is why we always work closely with our clients and give them deep understanding of what it is that drives those models. This does not mean that I have to pull out the source code or hand over the actual model on a silver platter. We all need to protect our Intellectual Property. However, this transparency secures a position of collaborator and partner. It also leads to deeper understanding from our clients about what is driving their business.

Transparency in modeling and scoring is becoming more routine over time, as model developers and users understand that it is one piece of the solution. However, it still amazes me that there are some select companies today that are positioning the "Black Box". I welcome that competition. I have to say that I don't mind my competitors positioning themselves as "We're smart - you're stupid".

A.E>>>