Analyst ethics?

Quis custodiet ipsos custodes?

Who guards the guards? This is a question that my peers and I have been discussing for some time. We are in a quandary. Analysts and analyst relations live in a symbiotic relationship where we need each other to thrive – you could argue that we are each others PR team in that the success of one group inevitably helps the other.

Conversely, the effect of a ‘bad apple’ impinges on us all. And so the question remains – who polices the analyst industry?

I have often felt that it is self-policing and that the success of firms is bred on trust, integrity, respect and knowledge. Therefore what should we do if we feel that a firm has lost the respect of the AR field? This is a serious question and one that shouldn’t be taken lightly – if we make a post such as this, are we damning a business out in the open when this should be done behind closed doors?

In private, my peers and I regularly do not recommend working with a specific analyst firm (who for the purpose of this blog shall remain nameless, although those in the industry may be able to guess who I am speaking about). They are seen as guns for hire (note this past link refers to Aberdeen but the firm I talking about here is another one completely). In the past when this has happened with other firms, people simply stopped spending money with them. But now, I believe we are at a tipping point. Money for analysts and AR is scarce and justifying their worth and independence is something that I feel I should not have to do.

Let me give you a couple of scenarios and you can judge whether this firm is unethical or simply doing business.

 

 

Scenario 1 – the award

A client of mine was contacted out of the blue to inform them that they have won an award for being the best at their chosen field. Naturally they were excited, and despite the feeling that they felt they deserved it, were delighted as not many people realised in the mass market they were as good as they were. Call it ‘the worlds best secret’. However, they were then told that in order to receive their award they would have to pay a significant sum of money. When they refused to do this, the award was presented to somebody else who now appeared first.

The question I asked was if a company deserved to win the award, then surely they should still be first despite the fact that they have not paid “to pick it up”?

Scenario 2 – the review

In this scenario, the analyst firm was commissioned to survey two phone handsets.  In an independent evaluation, the handset that came out top was the one produced by the commissioning firm. Now I have no problem with the firm coming out higher, the issue I have is that even though the analyst firm did mention the report had been commissioned, it failed to say by whom. The sceptics amongst us may presume that it was worried that, given the result, everyone would think the report was biased.

We have two situations here that call ethics into question. Can a firm write positively about vendors only if they have been paid to do so and secondly, can a firm be commissioned to write independently about a vendor if they have been given money in advance?

The simple answer is that no firm should provide positive commentary because they have been paid. Advocacy is determined by value not by money. It is the very basics of integrity that is at the heart of this matter and unless this process is stopped, and vendors refuse to spend money on such obvious gimmicks, as the market perceived value of these reports is so low, then I fear this practice will continue.

The specific firm in question has been deliberately omitted. Indeed you can argue that this practice isn’t limited to them. Which analyst firms disclose the clients mentioned in a report? None of the large analyst firms do that. Can we really say that white papers and commissioned research are ethically so different?

Of course, this last statement was deliberately provocative. Any firm can produce independent commissioned research. Take Freeform Dynamics for example who have a business model comprising of patronage. Sure IBM, Microsoft and others have paid them money to write research – but they have never told FD what the conclusions should be, what methodology they should practice, what criteria they should take note of, what weightings to follow or what assumptions to make.

My thoughts aren’t alone on this. Merv Adrian and Curt Monash recently discussed ‘White Paper Sponsorship and Labelling’. It was clear that much of what is commissioned by analysts was used as marketing material which should by default be taken with a pinch of salt. However, the need for authenticity still remained. Indeed one of the telling comments in the post by Carl Olofson is good practice for us all:

My recommendation for analysts is that any sponsored documents should clearly show their sponsorship right up front with the title and author, and also to consider that you do have some influence over how it is displayed. At IDC, we have forced vendors to make changes to their references to our work when the references improperly imply that the documents in question explicitly endorse them or their product (IDC has a firm policy prohibiting the endorsement of vendors or products; we may endorse or affirm technologies, methodologies, or trends, but in a vendor-neutral way).

To prospective buyers, I would say this: assume that the analyst reports page, like the rest of the website, is designed to impress you. If you just skim the titles of documents on this page without looking at them, you are allowing the vendor to create cumulative impression that could be misleading, just like looking at the very impressive customer logos that all vendors show in their presentations as if they are all strategic customers, when really, many are not. Caveat emptor, my friends.

I strongly believe that any time a vendor gives money to an analyst firm for paid-for work it needs to be referenced and authentic. Our industry needs firms it can trust to help them make informed decisions – without them, and if it’s reputation is tarnished by the acts of a few who practice methods I don’t agree with then the whole market fails.

Who analyses the analysts?

We all do.

Vendors need to stop looking for the easy marketing win by using these firms and the analyst houses themselves need to be more transparent. As to the specific firm in question – sort it out!