Saturday, February 14, 2009
Gartner releases paper on Static Analysis
Figured I would share thoughts on the quality of Gartner research in the static analysis space...
Gartner analyst's Joseph Feiman and Neil MacDonald have created the Magic Quadrant for Static Application Security Testing for the fine price of $1,995. Of course they focused on productecture and provided interesting insights into particular vendors such as Ounce Labs, Fortify Software, Coverity, etc but not in a way that is actionable.
For example, they mentioned that Fortify software has a heavy sales model that has annoyed customers but didn't say why this existed. Is it because of the way that products are licensed where folks need to have ongoing interaction with sales folks over the long haul or is it because they have too many sales people that don't have nothing better to do or is it because at some level enterprisey types like the attention and being annoyed is actually a good thing.
Some of my enterprise architect peers love the vendor sales guys. After all, they have an expense account and can take you to fancy restaurants. There is a secret relationship between sales people and enterprise architects that is rarely discussed in public. Some sales guys will allow the enterprise architect to outsource the sales pitch, the dog and pony show the enterprise architect themselves should be doing. So, if you work for one of those shops where you are an enterprise architect but otherwise are clueless, then the commentary of Gartner may not be a negative.
Is security based on superficial things you can count such as number of languages or could it be based on the ability for information to be distilled to non-developer demographics. It is important to acknowledge that static analysis is very useful for helping developers write more secure code but it has a side effect in that it can be used for management purposes as well. Consider a scenario where I may have two Indian outsourcing firms where your otherwise non-technical CIO may want to know whether Cognizant writes more secure code than say Infosys or Wipro. The reporting capabilities may be more important in some shops than its actual ability to help developers.
I find it fascinating that Gartner was notoriously silent on mentioning ways to evaluate static analysis tools. Most folks know that the OWASP WebGoat project serves as a test harness for many of these tools. I have always wondered whether it has anything to do with the fact that other analyst firms have beat Gartner in terms of participation in OWASP? For example, Chenxi Wang of Forrester has participated in user groups and conferences and the same thing can be said of the 451 Group, Security Curve, Nemertes and pretty much every other analyst firm on the planet. Gartner's ability to exercise their right to remain silent regarding OWASP is interesting but also potentially damaging to their clients.
I can say that both Ounce Labs and Fortify Software also participate in the open source community. It would have been great if they had acknowledged that many of the open source projects that you use in your production infrastructure are using these tools. When you pay attention to open source participation, the vendors start to separate even more.
Anyway, I do hope that Burton Group and Forrester will produce their own documents covering the static analysis space. I know that Chenxi and Blakley respectively will do a lot better job than our friends at Gartner...
| | View blog reactionsGartner analyst's Joseph Feiman and Neil MacDonald have created the Magic Quadrant for Static Application Security Testing for the fine price of $1,995. Of course they focused on productecture and provided interesting insights into particular vendors such as Ounce Labs, Fortify Software, Coverity, etc but not in a way that is actionable.
For example, they mentioned that Fortify software has a heavy sales model that has annoyed customers but didn't say why this existed. Is it because of the way that products are licensed where folks need to have ongoing interaction with sales folks over the long haul or is it because they have too many sales people that don't have nothing better to do or is it because at some level enterprisey types like the attention and being annoyed is actually a good thing.
Some of my enterprise architect peers love the vendor sales guys. After all, they have an expense account and can take you to fancy restaurants. There is a secret relationship between sales people and enterprise architects that is rarely discussed in public. Some sales guys will allow the enterprise architect to outsource the sales pitch, the dog and pony show the enterprise architect themselves should be doing. So, if you work for one of those shops where you are an enterprise architect but otherwise are clueless, then the commentary of Gartner may not be a negative.
Is security based on superficial things you can count such as number of languages or could it be based on the ability for information to be distilled to non-developer demographics. It is important to acknowledge that static analysis is very useful for helping developers write more secure code but it has a side effect in that it can be used for management purposes as well. Consider a scenario where I may have two Indian outsourcing firms where your otherwise non-technical CIO may want to know whether Cognizant writes more secure code than say Infosys or Wipro. The reporting capabilities may be more important in some shops than its actual ability to help developers.
I find it fascinating that Gartner was notoriously silent on mentioning ways to evaluate static analysis tools. Most folks know that the OWASP WebGoat project serves as a test harness for many of these tools. I have always wondered whether it has anything to do with the fact that other analyst firms have beat Gartner in terms of participation in OWASP? For example, Chenxi Wang of Forrester has participated in user groups and conferences and the same thing can be said of the 451 Group, Security Curve, Nemertes and pretty much every other analyst firm on the planet. Gartner's ability to exercise their right to remain silent regarding OWASP is interesting but also potentially damaging to their clients.
I can say that both Ounce Labs and Fortify Software also participate in the open source community. It would have been great if they had acknowledged that many of the open source projects that you use in your production infrastructure are using these tools. When you pay attention to open source participation, the vendors start to separate even more.
Anyway, I do hope that Burton Group and Forrester will produce their own documents covering the static analysis space. I know that Chenxi and Blakley respectively will do a lot better job than our friends at Gartner...
