Ahh, spring! Mud season in these parts, and as if on queue Nucleus Research has delivered a small monsoon of data about SAP’s customers and held up a mirror to their latest ad campaign.
The skinny: SAP claims in a global ad campaign that their customers average 32% better profitability than the companies that use Brand X for things like ERP, supply chain management, and CRM. Nucleus Research, which has been known to crash and spoil vendor self-congratulatory parties like this has said, not so fast. According to Nucleus, a population of reference customers from SAP’s Web site showed they were 20% less profitable than their industry peers.
If all you read is the headline you might conclude that Nucleus has provided us with a valuable service and taken the big corporation to task. The fine print might reveal something more – or less.
The issue is that Nucleus did its research on 82 customers bragged about on the SAP Web site, a minuscule portion of SAP’s 32,000 customers – about a quarter of 1% to be precise – and, of course there is also the issue of whether or not this is a representative sample. People who like to see the glass as half empty (and it was cheap whiskey too) will probably say, of course these customers aren’t representative. They ought to represent SAP’s best if they’re referenceable. On the other hand, people who say the glass is half full might point out that no one stays on top forever, life’s vicissitudes and all that.
Not to put too fine a point on all this, there are also some questionable circumstances such as, the research group that provided the data on which the ad campaign is based, Stratascope, is headed up by, Juergen Kuebler, a former – you guessed it – SAP executive. Moreover, the data is culled from a report that has so far not been shared with the public.
It’s hard to pick a favorite in this mess. SAP has played fast and loose with its customer data for years and Nucleus seems to have made a career out of planting a pie in the face of an unsuspecting high flying corporation. I have never been a fan of the idea of splitting the difference on something like this, of saying, “A pox on both your houses.” Maybe it makes me a bad negotiator or maybe it simply says that things should be run to ground, if possible, that we shouldn’t just give up.
Several memorable research reports by other firms like Credit Suisse First Boston (CSFB) have shown that in previous years SAP was so intent on capturing the pole position in the CRM market share race that it invented a novel way of counting deployments. Surprise! SAP became the market share leader. Maybe they really are today, I don’t know. I don’t keep close tabs on market share because I don’t think it measures anything other than sales. I think a lot of people feel that way at least in part because the definition got blurred just before the trophy got retired.
Nucleus, on the other hand, first came to prominence (at least to me) when they did the same kind of research on the Siebel customer base. Actually it was a handful of Siebel customers promoted on the Siebel Web site, and the metric then was customer satisfaction. At that time Siebel was promoting its customer satisfaction scores which were in the 90% range as proof of its wonderfulness. It was such a small sample that Nucleus researchers were able to interview the Siebel customers individually. I forget the actual results but they were substantially at variance with Siebel’s 90% claims and a small scandal ensued. At that point I asked Siebel for access to their customers to conduct a statistically valid survey which would, hopefully, get to the bottom of it all. Siebel complied and what I found was very interesting.
We got over 300 completed responses to our survey and the data proved Siebel’s assertion of 90% customer satisfaction. We also conducted a small number of direct interviews at random with Siebel customers. What we found in direct interviews was that although customers were satisfied with their purchases, they were not so enthusiastic when it came to all of the work involved in deploying a new version of what was then client-server software.
From that we saw the smoldering discontent with client-server software and the evolving importance of loyalty versus simple satisfaction. It was an eye-opening experience and it set the stage for a lot that has happened in the years since in areas like on-demand computing and the escalating importance of customer loyalty.
I hate to sound like I am splitting the difference but I don’t know who was right then and I don’t know who is right now. Certainly all the players know how to use statistical analysis and as we all know, you can often get statistics to support your thesis. That’s basically where Disraeli’s famous quote, “Lies, damn lies, and statistics,” comes from.
If there’s a lesson in all this I think it is that when a tool like ERP, Supply chain management, or CRM becomes ubiquitous, it ceases to have a major impact on competitive advantage – you either have it and continue to compete or you never adopted it and are probably out of business or heading that way.
The winners don’t look back because they are always looking for the next innovation.