The IIAR “Tragic Quadrant”

Last year, as part of the 2014 IIAR Analyst of The Year Survey, we invited analyst relations professionals to rate their favourite industry analyst individuals and the firms they worked for. More than 60 individual organisations responded to our survey. We were interested to see if we could do further analysis on the data that was collected.

When we set out to do the IIAR Analyst of the Year (with Helen Chantry), we always had envisioned doing a Magic Quadrant of analyst firms. This year the survey provided us with further information which we have been able to breakdown and analyse to provide a more detailed understanding of how analyst relations professionals perceive the relevance, impact and reachability of industry analyst firms. We are not claiming that this is an exhaustive study. Rather it simply opens a new (slightly cheeky – hence the notion of “Tragic Quadrant”) window onto the analyst landscape, where we attempt to rank industry analyst firms by impact, relevance and ease to do business with.

The IIAR survey asked respondents to assess individual industry analysts on a range of issues broadly grouped according to the following criteria:

  • ‘Impact’ (to what extent does the industry analyst impact the technology purchase decision);
  • ‘Relevance’ (is the particular industry analyst relevant to the technology purchase decision, do they understand the marketplace);
  • ‘Interaction’ (how easy is the industry analyst to reach and to interact with).

We are able to represent the top 13 industry analyst firms according to this new analysis. We have produced three Tragic Quadrants according to i) overall industry analyst firms, ii) global industry analyst firms, and finally iii) independent industry analyst firms.

  • The Y axis depicts the ‘Impact’ of the industry analyst firm on the purchase decision. This also relates to their perceived credibility and capability to provide an objective opinion.
  • The X axis marks their ‘Relevance’ for the purchase decision. This means their capability to cover the market and their specific geographical allocation. It also includes public recognition of their presence in the market (e.g. as an expert).
  • The size of the bubble is ‘Interaction’. This relates to issues of communication (e.g. how easy is it to get to them and to talk to them).

Overall Industry Analyst Firms Quadrant

The IIAR Tragic Quadrant 2015 (Overall) featuring Gartner, IDC, Forrester, Ovum, HfS, Constellation, 451 Research, Celent, Pac, ESG, Digital Clarity Group, Ventana, SMB Group

The IIAR Tragic Quadrant 2015 (overall) featuring Gartner, IDC, Forrester, Ovum, HfS, Constellation, 451 Research, Celent, Pac, ESG, Digital Clarity Group, Ventana, SMB Group

In terms of the Overall Industry Analyst Firm Quadrant, Gartner, Forrester and IDC came out ahead of rivals. In terms of their specific ranking according to our analysis, Gartner is 1st, Forrester 2nd, and IDC 3rd. Gartner is higher on impact and relevance than its immediate rivals, but did less well in terms of interaction. IDC by contrast scores someway ahead of Gartner in terms of interaction but is perceived to be slightly less impactful and relevant. By contrast, although some way behind the leaders in terms of impact and relevance, Digital Clarity Group, Ventana and SMB Group did particularly well on interaction.

Global Industry Analyst Firms Quadrant

2015 IIAR Tragic Quadrant - Global Firms featuring Gartner, Forrester, IDC, HfS Research, Ovum, 451 Research

2015 IIAR Tragic Quadrant – Global Firms featuring Gartner, Forrester, IDC, HfS Research, Ovum, 451 Research

In terms of the Global Industry Analyst Firms Quadrant, Gartner stood out as 1st. This was followed by, in 2nd place, Forrester, and IDC in 3rd place. As mentioned above, Gartner were successful with high levels of relevance but performed less well on interaction. IDC performed particularly well on impact and interaction. In terms of the other firms, HfS performed well across all three categories but scored particularly well on impact. 451 Research and Ovum did well based on interaction.

Independent Industry Analyst Firms Quadrant

2015 IIAR Tragic Quadrant - Independent analyst firms featuring Constellation, HfS Research, Ventana, SMB Group

2015 IIAR Tragic Quadrant – Independent analyst firms featuring Constellation, HfS Research, Ventana, SMB Group

In terms of the Independent Industry Analyst Firms Quadrant, Constellation were in 1st place, HfS in 2nd place, followed by SMB Group in 3rd place and Ventana in 4th. Constellation scored particularly well on impact, whilst HfS performed well on impact and interaction but less well on relevance. SMB Group scored highly in terms of interaction. Ventana did well in terms of impact but slightly lower in terms of interaction.

In case you were wondering the Tragic Quadrant is by no mean meant to be a normative or scientific depiction of the industry analysis industry. It offers but one window onto this world that was partially inspired by Juvenal’s ancient dictum Quis custodiet ipsos custodies? (which, roughly translated, is who assesses the assessors? or, in our case, who analyses the analysts?). Whilst it was initially intended as a bit of fun we do nevertheless think there are some interesting aspects for AR professionals and analyst houses to take from it. We see at least two takeaways emerging for different audiences:

  • AR professionals may use this as a subjective analysis tool to look at their target audience engagement strategies. They should balance ‘ease to do business with’ against ‘relevance’ and ‘impact’. Or, in other words, they shouldn’t brief analysts just because they’re easy to deal with (or conversely they should look at analysts which are less of a pain depending on the type of impact the AR professional is looking for (see the AR SOSM model);
  • analyst firms should monitor the ‘transactional tax’ they impose on AR people: if they raise the ‘interaction barrier’ too high while not providing sufficient coverage and not showing impact, their vendor information source might soon provide them only a partial view of the market (raising exhaustivity and fairness issues) or their vendor revenues might suffer too.


By Neil Pollock (LinkedIn@neilpollock), Yulia Sidorova (LinkedIn) and Ludovic Leforestier (LinkedIn@lludovic)


Related posts:

, , , , , , , , , , , , , , , ,

32 Responses to The IIAR “Tragic Quadrant”

  1. Stephen England Tuesday 9th June 2015 at 03:55 #

    We and our clients have almost zero interest in which firms vendors find easy to deal with. What we want to understand is which firms the end users find easy to deal with and look to for advice.

    KCG’s Mystical Box Charts plot precisely that, and our annual revenue analysis and landscape backs that up with the numbers that matter – what end users spend on advice from the firms – advice on what to buy, who to buy it from and what to pay for it…

    Our analysis does not align with this at all except for Gartner being number 1. Gartner takes well over 70 cents of EVERY SINGLE end user dollar spend on RAS.

  2. Michael Rennett Tuesday 9th June 2015 at 10:37 #

    Does the IIAR have a breakdown of the type of AR professionals that answered these questions? It would be interesting to see if there was a particular skew based on coverage area – ie. if the vast majority of respondents worked in IT Services AR, or if there were no AR professionals that covered the mobile industry.

    Realise this is all subjective but it could give some good insight on the above.

  3. Stephen England Tuesday 9th June 2015 at 19:47 #

    Gartner has 7 times the end user business of Forrester, 30 times that of IDC (Inightsand over 100 times that of almost every other firm.

  4. Duncan Chapple Wednesday 10th June 2015 at 16:44 #

    The strength and the weakness of this quadrant is that it shows how vendors over-estimate a few analyst firms. I’ve just posted an article to explain why here:

    Stephen is right that it’s influence on purchasing that counts, but it’s been ten years since you could equate the buyers of research with the buyers of IT and telecoms solutions. There’s hard data that shows that, and I’ve written about that too here: Given the impact of freemium research and vendor-sponsored analysts, I just cannot say that 1000 Gartner analysts have 100 times more influence than every other time. That means that a representative group of 10 Gartner analysts have more influence on end-users than the 6,500 non-Gartner analysts. That’s just silly.

    The Mystical Box, like Stephen’s market sizing of the analyst industry, have a lot of necessary guesswork behind them. Whatever the weaknesses, is has the redeeming effort on trying to focus on something which is helpful to AR people: focussing them on where they can made a business impact. Surely the logic of highlighting firms that are easy to work with would be that you’d spend more time on them, and less time on the analysts who are more influential but harder to work with.

    • Ludovic Leforestier Wednesday 10th June 2015 at 17:12 #

      You’re missing the point Duncan: AR is a sales channels for analyst firms. Think about this for a minute…

  5. Stephen England Wednesday 10th June 2015 at 17:18 #

    “Surely the logic of highlighting firms that are easy to work with would be that you’d spend more time on them, and less time on the analysts who are more influential but harder to work with.” said Duncan

    That’s the entire point – focus on influence not ease of interactions.

    And, if you judge influence by where end users actually spend money all that has changed in the last ten years really is the level of noise.

  6. Duncan Chapple Wednesday 10th June 2015 at 23:09 #

    Ludovic, honestly AR is not a major sale channel for most analyst firms. As you and I both say, if analyst firms want to develop business relationships with AR people then they should pay attention to how easy or hard it is to work with them. But not every analyst firm is going to go soft on vendors for that reason, especially if the analysts are disconnected from business development.

    Stephen, I totally agree: focus on the analysts that matter, and accept that some interactions will be hard. But, indeed, the reason why you see freemium and vendor-sponsored research as noise is because you mistakenly equate analyst influence with end-users spending on analysts. Even in organisations that do subscribe to Gartner and Forrester, there are often many more people using freemium and vendor-sponsored research. The days are over when AR people can ignore the long tail.

    • Ludovic Leforestier Friday 12th June 2015 at 11:37 #

      Duncan, do you have facts to back that up? I’ve worked in about 10 companies over the last 15 years and I can assure you that AR is a key influencer and/or buyer for research.

      Stephen, Duncan: the long tail is important for some AR folks, maybe not for others. It’s a question of alignment, I find strange that so few AR folks get this. See also the AR Compass >

    • Stephen England Friday 12th June 2015 at 17:27 #

      Our CIO clients strongly disagree. The small firms are interesting but there is no proof they influence the Fortune “10,000” at all in our opinion. SMB is changing for sure though.

      They would love to have us believe otherwise.

      • Ludovic Leforestier Friday 19th June 2015 at 10:14 #

        Stephen, I’m not sure what your CIO clients disagree with. Influence is proteiform, not unidimensional.

  7. Laurie McCabe Thursday 11th June 2015 at 16:02 #

    Couple of thoughts here. First, as a very small firm, we’re honored just to be on the radar! Second, I believe we’re the only ones that focus exclusively on SMB. As we all know, most SMBs do not buy analyst services directly. Our main focus at SMB Group is to provide vendor clients with fact-based perspectives, consulting, etc. (based on our quantitative and qualitative research studies) to help them better understand, sell to and serve SMBs.

    That said we know “influence” is also key and are ramping up our efforts to increase distribution of our perspective to SMBs through more social channels and outlets. We welcome your thoughts on how we can improve and add more value!

  8. Pranay Juyal Friday 12th June 2015 at 09:00 #

    IIAR Team, thanks for this research. is there a view on Everest Research Group for this evaluation?

    • Ludovic Leforestier Friday 19th June 2015 at 10:15 #

      Sorry, the respondents did not comment specifically on Everest. Feel free to contact us to let our members know what you guys are up to.

  9. Chuck Van Court Friday 12th June 2015 at 17:15 #

    Any technology buyer who gives more than 5 to 10 percent weighting to any analyst firm’s evaluation is delinquent in their responsibilities to shareholders, team members and the folks who will be actually using the technology.

    Other than possibly being an effective CYA strategy for protecting purchasers from accountability for bad technology decisions in the eyes of naive board members, I have yet to see any analyst firm that does not suffer from two deadly sins to justify more than a 5-10 % weighting: They earn too much revenue from the firms they are writing about to be considered objective and unbiased and the level they actually use and understand the technology being reviewed lacks the necessary depth and practical usage.

    I ask you, should a person buying a car give much weight to a car reviewer who has at best only driven the car in a parking lot and who earns a large percentage of their income from the car manufacturers they are reviewing?

    Mind you, I have found most of the people in this business to be smart with generally good intentions, but in my mind they should stick to macro-level analysis and helping vendors understand how to best sell their offerings. Notice that I said they should help vendors understand how best to sell and not actually help them sell. Who does not really believe all the contests, matrices and watchlists are more about generating revenue from technology vendors and less about providing technology buyers with quality data points used in purchasing decisions?

    The best place to get quality data points used in evaluating technology is from hands-on usage and from in-depth reviews by actual users of the technology who are not financially or otherwise motivated to write favorable reviews, or at least keep their negative comments tempered toward the hand that feeds it. I have been told off-the-record by several analysts that technology purchasers often give too much weighting to their analysis, but the revenue is certainly too compelling to say that out loud.

    The problem is not with the people in this space, it’s with the business model. People inherently follow the money and who can really blame them?

    • Ludovic Leforestier Friday 19th June 2015 at 10:17 #

      Chuck, analysts are not just about research and opinion.

      Did you know that in many large deals, industry analysts actually write the RFI / RFP and help evaluating submissions? Or that sourcing advisors actually take a cut on some deals?

      • Chuck Van Court Friday 19th June 2015 at 17:53 #

        Hello Ludovic: I fully realize that, which only further reinforces the importance for analyst firms to fully disclose any benefits they receive from technology vendors. My god, the FTC requires more disclosures from an unknown blogger regarding potential conflicts of interest than they do from Gartner and the like. Do you think that is right? All should be disclosed and the buyers should be left to decide if it matters to them or not.

        • Ludovic Leforestier Tuesday 12th January 2016 at 14:39 #

          Possibly, but I wonder if it’s enforceable. Personally, I find the US regulation authorities very peculiar in how they impose a heavy regulatory burden on innocents while letting lobbies get away with crime (applies to FDA, FTC, etc.)

  10. Stephen England Friday 12th June 2015 at 17:29 #

    Absolutely terrifying that this group has analyst firm employees as members in my opinion – they are specifically banned from KCG Connects on LinkedIn so that our members can have safe open discussions. Please change the policy here.

    • Michael Rennett Friday 12th June 2015 at 17:33 #

      The IIAR has member only discussions on the Huddle site, but as this is posted publicly on a blog then surely it’s within everyone’s rights to comment on it. The KCG Connects group serves a purpose but it can be beneficial to have analysts and AR professionals join the same debate. Disagree with your stance here Stephen.

      • Stephen England Friday 12th June 2015 at 18:04 #

        Got it, we are expected to pay for privacy with the IIAR to join the Huddle.

        At least post a health warning on the blog then?

        • Michael Rennett Monday 15th June 2015 at 08:41 #

          It would be up to the IIAR to edit the site, but if something is on a public website then I would expect it to be visible to everyone 🙂

        • Ludovic Leforestier Friday 19th June 2015 at 10:12 #

          Stephen, a blog is public almost by definition. And yes, our members can discuss on our Huddle Extranet.

          • Stephen England Friday 19th June 2015 at 23:31 #

            I know blogs are public, that’s why we at KCG use a secure LinkedIn group. I do not disagree with your right to allow analyst firm employees but I strongly disagree with the fact that you do not WARN users that they should never discuss anything here that they don’t want the firms (lots of them) to read.

    • Ludovic Leforestier Friday 19th June 2015 at 10:13 #

      We’re a broad church but no need to be terrified. The IIAR has been set up precisely to promote best practices and analysts too. Sales folks are welcome, readers will able to be gauge them on what they write.

  11. Chuck Van Court Saturday 13th June 2015 at 18:08 #

    Greetings: What is the hesitation in approving my comments (made Friday 12th June 2015 at 17:15) in this public blog? My comments are likely contrary to the opinions of many of your readers, but they are still respectful, detailed and touch on an important topic that warrants open discourse for the benefit of technology buyers. Thank you. Chuck


  1. Cottrill Research » The IIAR Tragic Quadrant - Wednesday 10th June 2015

    […] here to read the entire post for more details and view the three […]

  2. HfS hits bullseye of the IIAR Tragic Quadrant for IT analyst firms - Enterprise Irregulars - Friday 12th June 2015

    […] are in the mix with the Gartners, Forresters and IDCs of the world in The IIAR’s now infamous “Tragic Quadrant,” which IIAR just released on the impact and relevance of IT analyst firms. As you can see, […]

  3. IIAR’s Tragic Quadrant Spotlights Overrated Analysts | Kea Company's Influencer Insights - Thursday 18th June 2015

    […] June 8th, the IIAR published its first Tragic Quadrant. The TQ shows the perception, by around 60 vendors’ analyst relations professionals, of the […]

  4. Constellation and the curse of the quadrant | Institute of Industry Analyst Relations (IIAR) - Friday 1st June 2018

    […] Decision Matrices, Chartis issues the RiskTech Quadrants, etc. Even the IIAR proudly boast the Tragic Quadrant. Apart from PAC who went full circle with their RADAR and MetaGroup who tried ellipses, they all […]

  5. The IIAR Tragic Quadrant for 2017 | Institute of Industry Analyst Relations (IIAR) - Friday 14th September 2018

    […] years ago, in 2015, we produced the first IIAR Tragic Quadrant. It was met with much enthusiasm and comment, thus we have decided to repeat the exercise once […]

  6. Who will be the IIAR> Analyst and Analyst Firm of the Year 2020? | Institute of Industry Analyst Relations (IIAR) - Saturday 17th October 2020

    […] The IIAR “Tragic Quadrant” […]

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.