Will research crowdsourcing finally move analyst firms to an experience business model?

G2 Crowd Grid for Help desk - IIAR website
G2 Crowd Grid for Help desk

Good piece by Tony Bradley on TechSpective.net (via Rob Enderle and Stephen England) on whether crowdsourced analysis could displace Gartner, Forrester, IDC, etc.

I’ve been watching analysts for a long time and think this is fascinating -I was waiting for such a “JD Power of Tech” for a long time. The piece talks about Chaordix and Whale Path, and I’ve known for a while about G2 Crowd. Another well known so called Peer Review Site (PRS) is TrustRadius.

If they get it right, it will finally change the analyst business. Analysts are not going to disappear anytime -they may just be called something else: you’ll still need curators, pundits, showmen and showwomen, futurists and other shamans. But if the data collection could be automated instead of those more or less bad quadrants, they could focus on experience. It remain to be seen whether firms can be weaned off subscriptions? We, vendors, all know that moment when your rep suddenly wakes up and invites you to lunch… They’re all hooked onto subs: it’s relatively easy to renew, they can sneak in a 7-15% uplift tax (I did not say racket, did I?) every year, they produce the research once and sell it many times (Gene Hall calls this leverage and it certainly worked so far for Gartner).

This reminds me of a debate between the transition from on premise and SaaS: in the industry analysis industry, moving to an experience model would be a similar business model shift.

The other reason why analyst firms find moving towards being an experience business (despite some talking ad nauseam about the age of the customer) is psychology. Analyst firms are geared up to create technology myths and own the truth. To them, crowdsourcing is abdicating. The larger the firm, the harder it is: some smaller firms are very inclusive in bringing folks together (I was really impressed by a recent Constellation event for instance) while on the other hand I always thought that networking at Gartner Symposium was hard work. Both Gartner and Forrester market services aimed at specific communities, for instance respectively EXP and the Councils but it remains analyst-centric (although I haven’t paid the required 200k to try the EXP CIO service, so my guess is as good as anyone’s here).

One final thought. What everyone forgets (except maybe Le CXP) is the role of the smaller (and not so small) consultants and system integrators. Large firms have the scale to have research desk and even publish research and thought leadership (that’s actually my day job) but smaller ones don’t have that luxury. What they have however is hands-on experience of implementing solutions, deep relationships with their end-user clients and often an extremely good knowledge of a functional or technical niche.

In other words, consultants have insights that is complementary to analysts and are not competing with them. With a mechanism such as G2crowd, analysts could leverage them. Consultants on the other hand could get access to detailed benchmarks which the analysts currently struggle to produce. That would be Uber for research and I think the concept sounds cool.

What do you all think?


Read also:



10 thoughts on “Will research crowdsourcing finally move analyst firms to an experience business model?”

  1. To be honest I see this as nothing more than a novel data gathering mechanism. They also need to disclose way more info about the participants for it to have any value.

    Gartner could add some crowd sourced score cards to their methodology and crush this in a heartbeat – the quality of interpretation / moderation and – more importantly – the expertise of the moderators/commentators will remain the key here and G2 just does not have the bench strength to gain CIO trust.

    Thes publishing style models will continue but they’ll never rival RAS analysts in influence.

  2. Thanks for including us in your article, Ludovic!

    Many enterprise-level software purchasers already use our data and reports in conjunction with reports and articles from analysts and publications. In fact, a number of analysts and reporters ALREADY use our data in conjunction with their own expertise. What we do is definitely complementary to analyst research, and in some cases can provide insights that only users can produce, based on their own experiences implementing and using different products.

    In terms of our methodology, we vet not only every review but every reviewER as well, to ensure they’re not written by company employees, competitors, resellers, etc. We’re also fully transparent in terms of how products are scored on our Grids. (https://www.g2crowd.com/static/g2_grid_scores)

    Our goal is to bring more openness and transparency to the software/SaaS purchase process. Fortunately, it seems we’re doing just that.

  3. How many is “many” Daniel – compared to Gartner’s 10K end user clients? How many people at G2 are employed to sell your services to end users (not vendors?). What is the average contract value compared to Gartner’s $30K+ minimum?

    And exactly who is completing the surveys, not how do you use their scores to position the dots, is the question I was asking.

    End Users? Job Titles? Purchasing authority?

    How do you know they actually have the product reviewed? Do you interview them or is it everything online?

  4. It is difficult to say how many is “many,” because the majority of the users consume the data for free anonymously (although premium reports can be purchased for $599). According to our site analytics, we receive several hundred thousand new users each month. 70% of the users’ seniority level is manager or above.

    We get the level of experience and role for each reviewer based on their LinkedIn profile. This can be seen on the individual review and in the reports. (See an example of comparison data here: https://www.g2crowd.com/compare/salesforce-crm-vs-microsoft-dynamics-crm?tab=compare-group-21.)

    We require each reviewer to authenticate with LinkedIn; from there, each review and reviewer are vetted by a dedicated member of our QA team. They check each review manually to ensure they were submitted by a real person that is not a competitor, employee or reseller. In addition, users submit screenshots of the product actually in use, showing their user creds, to confirm they use it.

    Hope this answers your questions!

  5. Thanks Daniel. The user numbers are high but obviously most are very low level in comparison to Gartner. Users rather than decision makers would be my thought.

    We also value free usage much lower than paid relationships in the analyst world but as a Publisher I would be proud of your “circulation”.

    What stuns me – to be honest – is that anyone would give you (or any third party) their end user credentials for a large piece of software from within their employer’s organization. That would breach so many rules of logic and potentially of contracts that it confounds me.

  6. Pingback: BREAKING: Gartner gobbles brand benchmarking agency L2 | Institute of Industry Analyst Relations (IIAR)

  7. Pingback: IIAR Webinar – Why Peer Review Sites Matter: Influence and the digital buyer | Institute of Industry Analyst Relations (IIAR)

  8. Pingback: Constellation and the curse of the (not so) magic quadrant | Institute of Industry Analyst Relations (IIAR)

What do you think?

This site uses Akismet to reduce spam. Learn how your comment data is processed.

%d bloggers like this: