Constellation and the curse of the (not so) magic quadrant

Industry analysts, more specifically the buy-side “prescribers” exist to help technology buyers (often referred to as end-users) select the best vendors and providers. They gather insights through public and private sources such as (semi-)private vendor briefings and conversations (inquiries) with their end-user subscribers. Some analysts take hundreds of briefings and inquiries in a year, allowing them to gather unique insights on the market segments they cover. This accumulated knowledge allow them to monetise this information asymmetry as reports, consulting sessions, speaking engagements, etc.

One of the first report format aiming specifically at helping technology buyers to select vendors/providers was of course the Gartner Magic Quadrant. Based on a two-by-two matrix (as popularised by BCG, which is ironic considering Gartner is ran by a McKinsey alumni), the “MQ” quickly became irreplaceable. The X axis is for how good a product/service is (completeness of vision), the Y axis is for market traction (ability to execute) and the quadrant is in fact four quadrants, from leader to niche with visionary and challenger in between. Gartner now publishes a staggering 149 MQ’s, enough to employ a small army of analyst relations (AR) managers at each megavendor. Those type of landmark vendor ratings not only impact sales by pre-selecting vendors in long or short lists but also shape markets altogether as my fellow IIAR board member Neil Pollock studied. No wonder it’s been abundantly blogged about. See our own IIAR Best Practice Paper on Managing the Magic Quadrant.

 

As with most industries, digital disrupts industry analysis firms, for instance crowdsourcing may challenge the information asymmetry position they enjoy. We’re now in an attention economy and firms are competing for attention with free content. The sentiment that everything can be found on Google seems universally accepted by Generation Y, which doesn’t bode well for detailed research. Quadrants cut through that clutter with their neat, apparently scientific, visuals. Everyone likes a list and wants in, buyers can zero in on prospective suppliers, analyst firms can sell reprints (I mean webrights), so what’s not to like?

And thus each and every firm now needs to have a vendor evaluation landmark report that is based on the seminal quadrant: Forrester surfs on Waves, IDC churns out MarketScapes, HfS paints Blueprints, G2 crowdsources TrustGrids, Ovum publishes the Decision Matrices, Chartis issues the RiskTech Quadrants, etc. Even the IIAR proudly boast the Tragic Quadrant. Apart from PAC who went full circle with their RADAR and MetaGroup who tried ellipses, they all look like Jeff Mann’s spoof above, though some introduced a third dimension with bubble sizes.  Ping me a comment for the ones I forget and I’ll build a list.

 

Over time, under vendor pressure and to respond to client queries, methodologies have incrementally become more robust and now routinely include detailed spreadsheet-based surveys, mandatory client references checks, independent surveys, vendor briefings, etc. In my own estimates, one MQ will cost ICT vendors and service providers between 100 and 150 man-hours -more if they got consultants on the bench to keep busy. On the analyst side, they’re likely to disappear in a dark room for one to three months, before escalations. This makes quadrant expensive and too slow to produce -sometimes missing refresh cycles on some markets. One fix is to automate and get users to do the legwork. G2 pioneered the crowdsourcing model and Gartner is gradually introducing more Peer Insights into their MQ’s. Another drawback of industrialised quadrants is that the analyst opinions and insights are either concealed behind a methodology black-box or diluted.

Constellation saw that and think they’ve got a fix. They’ve just introduced ShortLists which are just that: short lists.  Here’s an example of the graphic below.

As you see, it’s a deboned quadrant: there’s no axis, not even a ranking as vendors are in alphabetical order. Call me a cynic if you like but I find that refreshing in its simplicity and also in the approach: all of it is free. You can download ShortLists for free, cite them in press release for no fee, use the graphic for zero Dollars (and even less Brexit Pounds), schedule an inquiry for nothing and even brief the analyst gratis. Look away awards sweatshops. Their methodology isn’t detailed but I’ll take the contrarian viewpoint that methodologies might have gone a little too much in the way of opinions at large firms.

Of course, Constellation hopes to increase their market traction, convert some prospects into paid subscribers but as for me I like that freemium model.

Yet, landmark vendor evaluations are such a pull for research firms that they’re unlikely to drop those signature reports any time soon. They’re usually the most downloaded research formats with case studies and vendor fuel the hype with a flurry of press releases each time a favourable evaluation comes out (but who would miss out?) They also buy webrights (reprints in industry parlance), a nice little earner for many firms -and a business model in itself for some.

Quadrants are imperfect but it seems there for the long haul. Personally, I would love to see them not only augmented with social ratings but also with informed opinions.