I have a confession to make: I’m not a whiz-bang, controversial, rock star security researcher (go ahead, GASP!). I’ve revealed a few bugs here and there, created some proofs-of-concept, but I’ve never dropped a crazy leet 0-day, had someone sick a legal team on me (yet), or dealt with miles and miles of red tape. I do, however, happen to know many fine folks that have been put through the ringer for disclosure, responsible or otherwise, and it sucks to see/hear/read about.
Lawsuits, gag orders, controversy…just some of the negative things I’ve witnessed my contemporaries and friends undergo after approaching a software vendor about some bug they uncovered. This type of reaction can be very discouraging for a security researcher, possibly resulting in them eschewing communication with the vendor in favor of disclosing it outright or selling the details on the black market.
Now, the “responsible disclosure” debate is one that’s been run dry; reconstituted; run over by a tank; re-assembled; annihilated by a nuclear bomb; and then somehow turned into a zombie mutant that just won’t die; so, with that in mind, I will spare you the usual rigmarole.
I think that it’s important to mention the EFF’s “Coders’ Rights Project Vulnerability Reporting FAQ“, as this is not only a great guide but also serves as somewhat of a basis for my proposal (also, I’ve got a huge crush on the EFF). I’ll give you a moment to go ahead and peruse the aforementioned FAQ. *waits patiently* Okay, let’s move on.
Note the last section in the EFF vuln. reporting FAQ. It summarizes the points that a researcher should consider when embarking on the potentially perilous journey that is vulnerability disclosure. I’d venture to guess that I’m not alone in saying that these points are essential, but it would be naive to think that following them to a ‘T’ will result in a happy, grateful vendor. While vendors and researchers alike have made great strides in strengthening their relationships, many (on both sides) remain opposed to any semblance of responsible disclosure (for various reasons).
After all that, on to my point: I would like to propose the creation of a site that lets security researchers share their experience(s) when going through the vulnerability disclosure process with particular vendors. This would allow other researchers to get a sense of how certain vendors might react, the timelines they might deal with, and other points involved in the disclosure process. While this type of discussion already happens on mailing lists and disparate forums, I know of no central location for this info.
Of course, this isn’t without its challenges. Just to cite a few:
1. Authenticity of a researcher’s claim about their experience. How do we verify that a researcher’s interaction with a given vendor is legitimate?
2. Utility and attractiveness (not aesthetically speaking) of the site. How do we get people to want to use this without having it become just another outlet for frustrated researchers? They should want others to benefit from their experiences, and hopefully avoid the same pitfalls or tribulations.
3. The “(Security) Assassination Market.” How do we minimize any negative influence this site may have (e.g. researchers going after soft targets because of good/bad experiences)?
I’m going to cut it right there and see who bites and who bitches (plus, I’m tired). 😉
UPDATE (2009-07-17 08:50): After a few responses from people (off the blog), I realized I failed to list the challenge of protecting the identity of whomever posts a story to the site. Naturally, this is something that would need to be balanced against the authenticity piece.
(CC licensed image from twenty_questions)
I recently conducted an assessment which was shut down prematurely because, apparently, the vendor didn’t realize I might successfully compromise the product. In subsequent meetings, their only concern was that the information be buried and I sign a non-disclosure agreement. The vulnerability is still there in dozens, maybe hundreds of systems used by thousands of people around the world. Going into any further detail whatsoever, even naming the company, would violate the NDA.
I don’t know if my experience is typical – I received the vague shadow of a legal threat, but nothing more, once they were sure their idiot mistakes wouldn’t be revealed to the world by me. Point is, a site like that could be cool, but I wonder if it would be filled with tales of companies who cannot be named for legal reasons.
[somewhat tongue-in-cheek] Besides, Blackhat/Defcon provide us with fairly public, yearly horror stories – HID, Halvar’s deportation (ok, maybe this wasn’t about disclosure), Massachusetts transit authority, this year’s ATM thing. Maybe that’s enough to keep the disclosure issue on everyone’s mind.
Brad: Bummer of a situation to be put in, though precisely the type of thing one could share for the benefit of others (though being respectful of the NDA…at least during its lifetime). As for the disclosure debate being kept fresh in people’s minds: I totally agree. I think it’s healthy for the community and industry to keep talking about it, it’s just that we keep spinning our collective wheels. Thanks for the comment.
Oh Zach, you’ll always bee a rock star security researcher to me…
*sighs dreamily*
Not a bad idea by the way. This could be expanded into ISPs too regarding phishing/botnet/etc takedowns.
Authenticity: 100% accuracy would only be attainable through acknowledgment by a vendor. For items “in the works” or “waiting to hear back from vendor” I’m struggling to think of a valid way to identify authenticity. Maybe a scoring system should be in place. “Yes, these are 100% valid look at the vendor information and what the submitter said[1]”. “These are probably real as we have XYZ information so we rate them 70% accurate/real/authentic”. “These are unverifiable at this point. They may be moved to another category if more information is garnered. This issue is rated 20%”. Or something like that. Items that have a valid CVE/OSVDB ID and the like can be linked an vetted quickly. Items that are still being addressed are going to be very hard to validate as authentic.
Utility: It isn’t a point of making the site useful, it either will or it will not be. If hundreds of security researchers need the information, it will be useful. If a handful of researchers need the information, it will not stay afloat for long. To me, it is a sound idea. You say “not aesthetically”, but if the site gets started, that really is one of the key elements that needs to be focused on. Presentation of data. You don’t have to create a market. Right now you’ll find out if it’s a viable idea or not. (Depending on if you get feedback from the blog or not and how many researchers read this blog and take the time to comment.) If the information is easily searchable and the site is simplistic enough to both review and post information then you will not be shooting yourself in the foot.
There are some key elements that probably should be included in the information posted so it doesn’t become something like a site of just stories. The vendor should be linked, maybe have some details on contacts for the vendor, do they have a standard SDL that includes security. Track records of vendor (for 100% verified disclosures). Maybe include severity of issue or installation base of affected application. The more data available the more conclusions can be made. Whatever the process is for researchers to get information uploaded, it has to be simple and fast. And the display of the uploaded content should be created with ease of use as one of the top priorities.
Assassination: No idea. Good question. After some time a poster may develop a presence that would increase the belief that their submissions were accurate and true. First time poster, not likely to be true. 100 post poster verified by vendor, likely true. Then again, that assumes there is some kind of requirement for a profile/account and I don’t know how that would impact researchers wanting to post.
Anonymity: I can’t imagine how well this will work with anonymity. Successful disclosures would probably detail the identity of the researcher. At first blush it would seem that only successful disclosures would be able to be verified as 100% accurate, so maybe assigning some type of accuracy value would be less helpful than I initially thought.
More coffee needed.
[1] If the goal is to keep researchers names out of light I don’t know of a way to be able to accurately verify a claimants authenticity of the incident, unless the vendor wholly acknowledges the incident.
PS Wow, way more coffee needed, but I’ll hit submit anyway. Maybe this will spark more discussion.
SecurityPerson: Excellent comment; thank you. The first time I thought about this, after a chat with my pal Oday, it was just going to be a catalog of vendors, service providers, and their respective security contacts and disclosure policies. After kicking it around a bit, and jumping into a lively discussion on disclosure at July’s BeanSec, I tossed this idea out there. Since posting, I’ve talked with a few other people who are interested; they’ve expressed concerns or notions similar to yours, so I’m really pleased to see that we’re all discussing and kind of working toward something.
As for “attractiveness-but-not-of-the-aesthetic-variety”, I meant that more as a “form follows function” sort of thing. I’m reminded of how the redesign of the OSVDB brought not only a more pleasing interface, but one that was also far more efficient to work with when retrieving information or mangling vuln entries.
OSF has discussed enhancing the vendor dictionary (http://osvdb.org/vendors) to support this type of dialog. Where researchers can not only update contact information, but rate a vendor, add good comments or horror stories, etc. Ideally, it would also be nice to link to advisories with the horrific disclosure time lines (e.g., some recent CORE advisories).
The question of anonymity isn’t so much a problem for us. The problem of people providing false reports of horror stories to undermine the competition or otherwise hurt a vendor’s reputation is a concern. Without some publicly documented version of the time line and a method for the vendor to dispute the claims, it turns into a mess.
I like the first part (vendor contact and disclosure policy). I initially thought of that after the first sentences of your blog post. I do believe that something like that was done before. But maybe it was only talked about. Either way, there isn’t anything that I’m aware of that exists in that form currently. OSVDB tried, is trying, or did, gather that type of information for whatever vendors they could. But I’ve not participated really and I’m not sure what the status for that data is currently.
I think that once established, the site would certainly help those researchers that have only dealt with a particular vendor or have only disclosed a few issues if any. I believe that the greatest benefit would come only after a substantial amount of data has been entered for display/searching/reviewing. That’s when even those who disclose on a regular basis would be able to benefit.
It would be great to be able to automatically grab and insert the disclosures by as those are usually standard-ish forms sent to various email lists. If we could get a back story on each of those, or at the very least parse the timelines that are included at the end of the disclosures, they could be added and would offer more value than just the disclosure verbiage itself sitting in random mail spools that may or may not be reviewed, ever.
I was also contemplating if it would be a good idea to set up something like a proxy service for disclosures. Information is disclosed to website, website contacts vendor and starts disclosure process on behalf of anonymous submitter. But, I guess that would be a replication of the current pay for vuln process already in place. However, maybe a researcher isn’t motivated by money, or feels it would be unethical (illegal? violation of the NDA?) for them to seek money for an issue they found while employed by a consultant shop doing a third party assessment. Maybe those items could be run through a pay for vuln service and the money gained goes directly back into site costs (domain name, server parts, colocation costs, etc). Complete transparency on the latter would help that work without fear of that money going to something/somewhere else.
Integration, details, simple APIs exposed. Those are keys to success for something like this. I think that a lot of valuable information could be gathered and correlated based on something like this. Especially in conjunction with data from something like OSVDB.
And I concur, “form follows function” is a good thing. But it’s best not to design something completely obtuse that would put people off at the onset.
I’m not sure about relevancy in the next 10 years, but right now, I think this is a great idea and would certainly be handy for a number of individuals who get down and dirty with disclosures now.
(Update: I didn’t do a refresh since the initial response to my post by Zach. I started to type a response, then decided I should get my work done first and wait for lunch to finish. So although my post time is after Jericho’s consider mine typed up before his and a direct response to Zach. But, I’m happy I was correct in thinking OSVDB had/did/was doing something of that nature. After reading Jericho’s post, I still think this a viable thing to do and perhaps something that could be done outside of OSVDB but certainly in cooperation with them. And someone should sit down with a few beers and see what else they’ve uncovered in their attempt (was it really a conscious effort or a byproduct type of thing?) at a similar concept.
And stop stalking me Jericho.)
Jericho: by now I think CJI should have contacted you. After his recommendations (on my *personal* blog, where I posted this as well), and the comments from SecurityPerson, I think it would be a real boon to the community (and a real treat for me and for others) to look into taking these same ideas and apply them against what the OSF has already done.
OSVDB had plans for years to do a 3-rd party disclosure system, where we would handle the process on behalf of the researcher. The original idea was to provide a relatively standard system and timeline, allow the researcher to keep researching (rather then get bogged down in the time consuming process of disclosure in some cases) and to give vendors a point of contact that would not back down and ignore bogus legal threats. I’ll have to look at the status, but I think our dev came up with an automated system for handling all of that. Just not sure if it was unfinished, needs testing or what.
CJI pinged me yes. Please mail moderators{at}osvdb.org with discussion. You can keep it brief with “read this thread [url]” to catch the others up. As with many of our enhancements, it will take longer to discuss and come up with the appropriate fields and safeguards to deal with the hurdles brought up in this thread, than it will take to actually code and implement it.
On a side note, I have also been personally tracking vendors that use legal threats against researchers in any capacity. That page (when I finish the chart and complete my rant) will be public on attrition.org. Ideally that information would be housed on OSVDB and those vendors would be flagged so that they appear in big flashing letters with a “DO NOT DEAL WITH $VENDOR” type warning. =)
Sorry for spam, but my failing memory..
I did finish off the vendor threats page and posted it a while back! =) As always, if anyone recalls other incidents, please submit them to errata{at}attrition.org.
http://attrition.org/errata/legal_threats/