Gartner Magic Quadrant shortcomings or how not to use the gartner magic quadrant

The Gartner Magic Quadrant methodology has become quite good now but it has two major shortcomings.

1. The taxonomy, ie. how analysts categorise vendors and products in a market. This in turns influences inclusion criteria. This has several impacts.

Sometimes, inclusion criteria are fallacious (for instance analysts making an artificial distinction on whether a solution is offered as a software or a managed service without having the ability to check contract terms), inconsistent, or hard to evaluate (for instance gauging service firms on FTE is a surefire way to get it wrong).

• 2. Analysts also try to artificially limit MQ participation to 16–18, mostly for bandwidth reasons. In less mature markets, it can be actually misleading.

The decision to include some features but not others or how analysts make the cut between stand-alone features is extremely subjective.

MQ’s are two dimensional. Whilst everyone can figure out a two by two matrix has only two axes, it’s harder to work out the impact:

In services for instance, a 2D MQ can’t accurately describe geographies.

Nor can they depict sizes, in this respect the Forrester Wave is better.

Nor can they cater for difference use cases, so buyers should use the associated Critical Capabilities IF there’s a capability matching their use case. In less mature Gartner offerings such as Gartner for Marketing Leaders, Critical capabilities are often not published.

3. And finally, Gartner collects many variables for an MQ. They then apply non-transparent weightings (though they mention which are high/medium/low) to combine those apple and pears. This is the greatest and most contested flaw in the MQ model.