Dealing with AR

By / / AR, CMS Watch, IIAR>

Alan Pelz-Sharpe In this article, Alan Pelz-Sharpe, Principal with CMS Watch and former VP North America for Ovum, shares some very interesting insights on AR from the analyst’s perspective.

Some tips on how not to deal with a critical ‘independent’ analyst

I have been an analyst and commentator for 10 years now – for most of that time I have written or contributed to detailed and critical evaluations of software technologies. My topic areas are the Content Technologies ranging from ECM and Document Management to e-mail Archiving – my audience is almost exclusively buyers and implementers of these technologies – and a typical deal size is in the high hundreds of thousands to the multiple millions. People read my research to create shortlist’s, and typically to ensure they have a better chance of selecting the right product. It’s a simple model really – much like a ‘Consumer Reports’ or ‘Which Guide’.

I play with the technologies, I see them in action, I talk to many users, I talk to channel partners, resellers and also consultants and integrators. I also talk to the vendors – but my use of vendors is more for fact checking, product demo’s and gaining insights on nit picky elements than anything else. I appreciate the help of vendors, but ultimately my research focuses on how products work and are sold in the real world, and the world of vendor marketing and sales is of peripheral interest. Granted that is an unusual research model, having been in the ‘industry’ for 10 years and having run research practices and undertaken extensive competitive intelligence, I am well aware that typically ‘research’ by analysts is heavily dependent, and in many cases almost entirely dependent on, ‘vendor briefings’. I am also aware that the vast majority of analyst firms are dependent on vendor funding of one form or another to pay the bills. Hence I go to great length with all those I write about to try and inform them of my requirements – and my methodology.

So to be clear, I am a realist – I know that most analysts make their money by selling ‘independent’ analysis to the very people they claim to be ‘independent’ of. It’s the way it is – whether I like it or not. I also understand that AR professionals have a very tough job to do. Frankly I do not envy your role – you have to try to keep everyone happy all the time, and that is an impossibility. I have deep admiration for many AR professionals, some of whom I am proud to call friends rather than contacts. I also have deep admiration for any vendor who stays in business more than six months, life is tough out there. Running a business or simply having responsibility for a P&L is always a challenge. At the same time, my job is to provide my customers with honest and critical evaluations of products. That means highlighting all the warts, along with spotlighting all the shiny positives. If anything my job is to focus on finding the warts. Because lets be honest, it is not hard for a buyer to find the positives. As they will be deluged by ‘White Papers’, Marketing Collateral and Sales Spin. Finding where the products sweet spot is or it’s drawbacks, is much harder. It’s my job to help them in that process, and by definition that is not going to make me popular at times.

It seems clear to me that some AR professionals simply don’t know how to deal with analysts like CMS Watch – and rather than continually lock horns, I thought I would jot down some thoughts to help the process – I am doing this as I am just about to publish a major report (major in the sense that it runs to over 300 pages) technical evaluation of 14 vendors. The frustrations and wounds of dealing with AR are very fresh! So here goes:-

1: Don’t assume the analyst is out to get you

You are not as important as you may think. The analyst is writing about many vendors, you are just one in a long list. You almost certainly have no context to judge their review of your product, in light of what they have said about your competitors – you may wish to consider slowing down before jumping to bias conclusions. In my most recent report, the AR group that had the biggest and nastiest hissy fit, ironically is the vendor that has received the best review of all in the report. They are also the vendor that had the biggest hissy fit last time they were reviewed (different product, different report, different analyst – again a great review). They are also the vendor that analysts from rival firms share AR horror stories about…. The firm has good technology, but a terrible reputation for bullying or attempting to ‘coerce’ analysts.

2: Do make an effort to understand the analysts research methodology

If the methodology is focused on talking to customers and partners and you have been asked to supply customer references. Respond in one of two ways – politely but immediately decline, or do your best to provide references. Ignoring the request for weeks or months is not a good policy. By that time customers and partners have been found by the analyst and interviewed. When critical views are captured from such interviews you cannot at the last minute claim “our customers love x or y or z” – we know they don’t and frankly you haven’t been able to supply any that do. Harsh as it sounds, we are not just going to take your word for it.

3: Don’t threaten analysts

If you don’t like what an analyst has written – try at least to be respectful and polite. You are far more likely to enter a dialogue that way. Provide facts to counter their critical assertions, if you cannot provide facts and instead rely on bluster you will only dig a deeper hole for yourself. Also remember that analysts are human, threats via nasty e-mails (the cowards way) or phone calls, hurt (no matter how long you have been in the industry) and they don’t get forgotten quickly. Using such a confrontational approach does not make the AR person look important or even imperious, it makes you look unprofessional.

4: Don’t quote your own press releases or other analysts reports as evidence

There is frankly nothing more silly than to tell an analyst that they must be wrong about your firm/product because “Forrester/Gartner/IDC…ranks us as a ‘leader’ etc”. The only thing that rivals this is to quote from your own press releases – trust me this has been done. Most of the time, this kind of response will simply result in an internal e-mail chain that shares the joke with other analysts. Bottom line, that kind of supporting evidence, looks desperate, patronizes the analyst, and suggests you have simply drunk too much of your own kool aid.

5: Never say “we provided an x% ROI…….to our client over six months etc etc.”

Its a silly thing to say, period – and its a daft thing to say to most buyers. It’s a little like Home Depot claiming that they dug my vegetable garden for me, when all they did was sell me a spade. You provide tools – people use the tools, the use of those tools provides business benefits (or doesn’t). And just like the spade I bought from Home Depot, most software likewise goes unused.

7: Understand the difference between a fact and an opinion

For every 10 vendors I evaluate there will be one or two that freak out – most work well with me and we agree to disagree, and where there are errors (I make many, and do my best to fix them) we work together to get them corrected. I never want my reports to contain factual errors, presumably nor do you. But my opinions are my opinions, I am paid to have opinions. To change my opinion requires a very different approach from AR. To change my opinion you need to understand why I have formed that opinion (see below) before attempting to ‘re-educate’ me. In addition, when you claim a report is full of factual inaccuracies, and then send an annotated Word document listing differences of opinions – and can quote no factual errors at all – expect your response to be ignored, and my respect for you to slip.

9: Understand that those that use and/or implement your systems have a very different perspective to share

Just as I will see your product or service differently to you – recognize that a sales person, a channel partner, a user, an implementer or a consultant will all have differing perspectives. When a report does not reflect your personal or corporately mandated vision, that does not mean it is wrong. Some vendors use my reviews of their products to change perceptions, in some ways they see my reports as free consulting – a fresh pair of eyes if you like. They recognize that the information and insights that I get are not usually available to them – they see criticism as potentially constructive. Some find out there are strengths to their product, that I have noted, that they had previously underestimated. Remember, if the only research you have read is from people you directly or indirectly pay – then it won’t be surprising if you find some kind of uniformity with your own viewpoint. True outside opinions will by definition differ from your own.

10: Don’t believe your own hype

We know it’s your job to be passionate about your company, about its product and its services. We understand it is your job to help sell this vision and to educate us all. But make the effort to really understand your competitors and your competitive landscape too. Work out who really influences your deals and those of your competitors – understand your competitors strengths in terms of product, sales focus, corporate culture etc – don’t live in a vacuum, analysts don’t. I applaud your enthusiasm, and I wish you and your colleagues the best of luck, I really do. But I wish all your competitors the same too. I am not passionate about your company, I am passionate about ensuring that buyers and users avoid costly and sometimes disastrous mistakes. That they pick the right product each time, and that they use it to its best advantage. We have different agendas, but they don’t need to be agendas in conflict.

Disclaimer: Alan is not a member of the IIAR and this post reproduce his own opinions, not those of the IIAR or its members.

 

Other recent analyst guest posts

 

3 thoughts on “Dealing with AR”

  1. Pingback: Dealing with AR | Product reviews

  2. Pingback: landscape depot

  3. Pingback: IIAR Analyst Survey and other musings « IT Industry Outsider

This site uses Akismet to reduce spam. Learn how your comment data is processed.

%d bloggers like this: