Thursday, July 20, 2006
Should Industry Analysts get paid to write research reports?
In a previous blog entry there were two comments left that need to be explored...
First, AG wanted to know my opinion on the MS funded Gartner reports or the notorious "Get the Facts" campaign. For the record, I am OK with software companies paying analyst firms to conduct real research on certain subject domains. I would say that I wouldn't call the particular work you mentioned though as research but more of a marketing piece. Of course, it is easy to concoct a scenario in which one vendor will look better than another and publish the results. Research is much deeper than that and should outline the probability that one would be in that situation along with why folks got there in the first place.
I think that most folks would have thought of this more as research if it also included other operating systems in the mix along with a little bit more transparency on how the analysis was conducted. I would also say that Gartner is not alone in the field of taking money from software vendors to create research. The likes of Radicati, Yankee, IDC and others also have a similar practice. Maybe the real rub with some in the community is not the money but that they didn't on the front page of the report clearly show in terms of a disclaimer who exactly was paying for it, how much they paid for it and what was out of scope. If this were included, I suspect you would have less issue with it.
The second comment in the blog that was equally intriguing was with Neil Macehiter and his comment on European analyst firms and how being across the pond makes it more difficult to be face-to-face. At some level I agree that there are certain circumstances in which research could occur over the phone but I am sticking to the point that face-to-face is much better. I guess the way this gap gets closed is by acknowledging that industry analysts tend to frequent many conferences here in the United States. The odds are good that many conferences will be in the Big Apple (NYC). If you are close to NYC then you will be close to us which makes it easier to close the gap and have a face-to-face conversation. Now the more difficult part is in getting analysts to publicly display their travel schedule months (instead of the day before in their blog) so that this can be orchestrated.
In the blogosphere, I tend to have a lot of interactions with folks over at Redmonk and periodically they visit my state to engage in a dialog with IBM. Sadly though we have never managed to coordinate a face-to-face meeting (Stephen and I have met at the OSBC conference though.
The more interesting comment was from James Bach who asked the question of who is going to pay for a publicly published study of their own screw up?. He further states I am simply not allowed to publicly criticize, in detail, anyone's specific situation or practices. I would like to respond with the following considerations:
Anyway, in thinking about AGs question of paying for research reports. I would like to say on the record that not only are there situations in which software vendors should pay analysts to create them but there are scenarios in which us enterprise folks should also consider the same.
One of the problems that enterprise folks face is that there is a lot of innovation going on in the marketplace which usually occurs outside of the walled gardens of our incumbent vendors. Venture Capitalists such as Ed Sims and others frequently create innovative startups to address a need. Herein is the start of the problem space. Imagine if analyst firms were to instead of writing up wonderful grids showing how products compare to each other instead focused on the current needs of enterprises and levelset those for vendors to consume.
We have been busy circulating a specification for a problem space we are labelling as data masking whereby we believe that in order to protect from loss of personally identifiable information additional constructs need to be built directly into the database engine. We have sent this specification to Microsoft, IBM, Oracle, ANTS, MySQL, Ingres, PostGres and others. Likewise, we have sent it to enterprise architects for other Fortune enterprises seeking their support. The problem space is that we would like to see certain problem spaces we all collectively face become commotitized out of the gate but don't have enough bandwidth ourselves to get others to assist in this undertaking.
Now imagine if I could pay an industry analyst such as James Governor of Redmonk to actually write a whitepaper on the pervasive problems we face. Likewise, imagine if this same whitepaper contained statistics on how many other Fortune enterprises also face the same problem so that my incumbent vendors when hearing of my requirement will not immediately think this problem is somehow unique to me nor put it in the queue until their other customers have the foresight to articulate it to them. Would it be a better investment for me to say pay an analyst firm $20K to write this whitepaper with the goal of commotitizing something that if left to the venture capital community would result in yet another vendor to work with and the large six figure purchase price for this solution.
| | View blog reactionsFirst, AG wanted to know my opinion on the MS funded Gartner reports or the notorious "Get the Facts" campaign. For the record, I am OK with software companies paying analyst firms to conduct real research on certain subject domains. I would say that I wouldn't call the particular work you mentioned though as research but more of a marketing piece. Of course, it is easy to concoct a scenario in which one vendor will look better than another and publish the results. Research is much deeper than that and should outline the probability that one would be in that situation along with why folks got there in the first place.
I think that most folks would have thought of this more as research if it also included other operating systems in the mix along with a little bit more transparency on how the analysis was conducted. I would also say that Gartner is not alone in the field of taking money from software vendors to create research. The likes of Radicati, Yankee, IDC and others also have a similar practice. Maybe the real rub with some in the community is not the money but that they didn't on the front page of the report clearly show in terms of a disclaimer who exactly was paying for it, how much they paid for it and what was out of scope. If this were included, I suspect you would have less issue with it.
The second comment in the blog that was equally intriguing was with Neil Macehiter and his comment on European analyst firms and how being across the pond makes it more difficult to be face-to-face. At some level I agree that there are certain circumstances in which research could occur over the phone but I am sticking to the point that face-to-face is much better. I guess the way this gap gets closed is by acknowledging that industry analysts tend to frequent many conferences here in the United States. The odds are good that many conferences will be in the Big Apple (NYC). If you are close to NYC then you will be close to us which makes it easier to close the gap and have a face-to-face conversation. Now the more difficult part is in getting analysts to publicly display their travel schedule months (instead of the day before in their blog) so that this can be orchestrated.
In the blogosphere, I tend to have a lot of interactions with folks over at Redmonk and periodically they visit my state to engage in a dialog with IBM. Sadly though we have never managed to coordinate a face-to-face meeting (Stephen and I have met at the OSBC conference though.
The more interesting comment was from James Bach who asked the question of who is going to pay for a publicly published study of their own screw up?. He further states I am simply not allowed to publicly criticize, in detail, anyone's specific situation or practices. I would like to respond with the following considerations:
- When industry analyst firms do case studies on enterprises, they do them to gain their own insight as well as to share with others. The enterprise doesn't pay anything for their creation.
- To a certain extent, a case study on an enterprise is a little different than a software vendor. For example, my enterprise doesn't sell technology so by sharing information with analyst firms in hopes of favorable press, it will never result in a better bottom line. It does however benefit us to tell our competitors that we are better than they are at technology.
- No enterprise would volunteer to be publicly trashed regardless of the analysts perspective. The key thing to understand is that an enterprise of course will tell you all the wonderful things they have accomplished. The harder part that actually makes a case study more compelling is in getting an enterprise to also share with the analyst firm things that they would do differently in hindsight. This is distinct from trashing. Analyst research should provide this form of insight which is useful to lots of folks.
- Its OK to admit you are human in public. I do it all the time. If you want to know what I am doing well, ask. Likewise, if you want to know what I would do differently, ask. There is no magic to this. I suspect you run across a lot of folks in other enterprises who lack integrity and want to have one sided conversations. You will not find anything other than the highest levels of integrity amongst the circles I travel in.
Anyway, in thinking about AGs question of paying for research reports. I would like to say on the record that not only are there situations in which software vendors should pay analysts to create them but there are scenarios in which us enterprise folks should also consider the same.
One of the problems that enterprise folks face is that there is a lot of innovation going on in the marketplace which usually occurs outside of the walled gardens of our incumbent vendors. Venture Capitalists such as Ed Sims and others frequently create innovative startups to address a need. Herein is the start of the problem space. Imagine if analyst firms were to instead of writing up wonderful grids showing how products compare to each other instead focused on the current needs of enterprises and levelset those for vendors to consume.
We have been busy circulating a specification for a problem space we are labelling as data masking whereby we believe that in order to protect from loss of personally identifiable information additional constructs need to be built directly into the database engine. We have sent this specification to Microsoft, IBM, Oracle, ANTS, MySQL, Ingres, PostGres and others. Likewise, we have sent it to enterprise architects for other Fortune enterprises seeking their support. The problem space is that we would like to see certain problem spaces we all collectively face become commotitized out of the gate but don't have enough bandwidth ourselves to get others to assist in this undertaking.
Now imagine if I could pay an industry analyst such as James Governor of Redmonk to actually write a whitepaper on the pervasive problems we face. Likewise, imagine if this same whitepaper contained statistics on how many other Fortune enterprises also face the same problem so that my incumbent vendors when hearing of my requirement will not immediately think this problem is somehow unique to me nor put it in the queue until their other customers have the foresight to articulate it to them. Would it be a better investment for me to say pay an analyst firm $20K to write this whitepaper with the goal of commotitizing something that if left to the venture capital community would result in yet another vendor to work with and the large six figure purchase price for this solution.
