There is no spoon.

I just read this article from TechTarget, quoting Gartner research vice president Ramon Krikken as saying:

[...] it may be time to ask whether it’s faster, cheaper and ultimately just as effective to use a device like a WAF to shield an application from a security flaw than face the unending cycle of developing, testing and implementing software patches. [...] I have an increasing number of customers starting to question whether putting a Web application firewall in front of an application to fix something is all that much worse than fixing the code.

And I agree that some apps can't be remediated in a short enough time span, others can't ever be fixed, and so on -- for those exigiencies where there's no other choice, a WAF is better than nothing (assuming it's actually in blocking mode and properly configured).

However, I would strongly caution anyone against deciding that the wave of the future is simply to rely on the WAF or any other network-based security device for application security, because THERE IS NO FRONT.

You can't put a WAF in your production DMZ and declare it done.  An attacker who bypasses that DMZ and gets into your internal network will have a field day inside the soft and chewy interior.  Dev and test applications, often with production data in them, will be vulnerable.  They may also have back doors -- excuse me, I mean "test harnesses" -- that don't get stripped out until the production build. 

So maybe you'll put a WAF on board every web application server on your network.  Are you ready to manage all those rules, as new security vulnerabilities are found?  (You may well have different versions of the same application throughout your enterprise.) And wouldn't it be tempting just to start loading any other functional fixes that you can onto the WAF instead of having to fix, test and release the code?

There's another problem, though:  portability of code.  You may not ever plan to make it available outside of your organization, but can you imagine giving it to a business partner, or selling it, and saying, "Oh yeah, you'll need to get a WAF and about 80 rules because we never fixed the code"?  That's just not an option for anyone who is shipping, releasing or sharing their apps.

And finally, don't forget that pesky cloud.  As applications become distributed among different sites and providers (not to mention mobile devices), there isn't going to be a choke point for all application traffic to go through in order to enforce policies and detect attacks.  As hard as it sounds, trust boundaries need to be built into the applications themselves so that they correctly handle and protect their own architecture components and processes.  Otherwise your app is going to be wandering around the Internet with its hair creepily following two steps behind.

A WAF is a Band-aid, not a cure.  And even though it can be very useful for defense, additional validation or data transformation, it still doesn't provide 360-degree protection for an app; only the app can do that.  Don't slip into becoming part of a popular blog.

Slide rules.

In my time as a CISO, and now as an analyst, I've seen more vendor presentations than I can possibly count.  Over time, I evolved a set of rules that you may want to know about if you're going to share a slide deck with me.

Here are the required elements:

1. First off, there must be a slide talking about The Problem We All Face, and indicate that it’s a scary, scary world out there.  Otherwise I would forget why we’re all here. 
2. Next, there must be a conceptual  slide that includes icons of people, the cloudernet, and either monitors or CPUs.  Extra points for locks, or creatively drawn bad guys.
3. Add a chart of your company’s growth with the arrow pointing skywards on the right-hand side.  Don't include any numbers or units on the axes; those details are irrelevant.
4. There must be at least one circle-shaped process flow, indicating that the customer will never be finished using your product.
5. Don't forget the obligatory page full of customer logos (whether they approved the use or not).
6. And tiny screenshots of your product, which I cannot possibly read.
7. Compliance.  The word compliance has to be on there; otherwise I’m not reading it.  APT is not a one-for-one substitute, although it’s close.
8. You must show your boxes replacing your competitor’s boxes in an abstracted network diagram.  If your product is only software, you should still use boxes.  Virtualized appliances should be depicted by cloudy boxes.
9. Please include some fancy transitions or build sequences so that I can watch them break, or miss them altogether, during an online presentation.
10. And finally:  I cannot take your presentation seriously without military references, a fortress metaphor, or an onion metaphor (depicting defense in depth).

Now, if you're feeling especially ambitious and would like bonus points, I would love to see:

1. The classic "risk = vulnerability x impact" equation.  I just can't get enough of that one.
2. Carefully chosen quotes from a couple of bank customers saying how wonderful your product is.  Because I hadn't been planning to buy until I saw those. Banks always know what they're doing.
3. A description of your bad-ass threat researchers, whose continuous stream of published vulnerabilities and exploits makes my job as CISO so much easier.
4. Add a percentage figure to your "low false positive" rate.  Better yet, make it zero; that saves us all time.
5. A reference to Kevin Mitnick is just the cherry on top.

Thanks for tuning in, and I look forward to the next 24-megabyte PowerPoint file in my inbox.

Conferring about conferences.

There's a great discussion going on right now on Twitter about what's wrong with security conferences:  do we have too many?  Are they focusing on the wrong things? 

Josh Corman threw out the figure that more than 60% of conference paper submissions these days were on Android security issues.  This sounds pretty excessive when you consider all the other security topics out there.  However, let's not forget that there are many different audiences for security talks, just as there are different sub-communities within the security industry.  For "breakers," Android security is a hot topic these days, and you would expect to see a lot of talks on mobile security in general at conferences "by breakers, for breakers."  And because that's a hot topic among breakers, you'll see defenders and builders eyeing it as well, because in the security ecosystem, what's getting targeted the most is what everyone will tend to focus on.

That's not to say that security conferences are homogeneous.  There is a very different culture and flavor at work at a conference for defense-related security (law enforcement and military, and to some extent critical infrastructure), as opposed to a meeting of financial services CISOs, or civilian government, or academia, or "hacker ethos" tribal gatherings.  Even if the hot topics are nominally the same, the perspectives and timbre of discussions will be very different.  And a conference that features roundtable discussions will bring out information exchanges that aren't as readily forthcoming at classic "stand up and present" functions (even if you count the hallway track).

So even though the sheer number of security conferences these days is dizzying, I think the variety is healthy.  We need the grass-roots B-Sides just as much as the vendor-oriented RSA, or the raucous Shmoocon, or the Chatham House Rules-driven CISO roundtable.  If anything needs to be changed or tweaked, I simply think that we need to make sure that the same speakers aren't touting the same perspectives at all of these different venues.  Everyone wants to hear a sexy war story about mobile every so often, but I really admire the efforts to bring in first-time and local speakers to certain events as well.  The "democratization" of security conferences is a trend that I'd like to see continue.

Too many questions.

As an analyst, I have too many things I'd love to research and can't.  I'm in a target-rich environment (then again, so was Custer).  It doesn't stop me from coming up with questions, though, and hoping someone else will want to answer them.

Take the discussion I just had on Twitter with @jeremiahg, @chriseng, @attritionorg, @dakami, @rybolov and others.  I objected to the claim that everyone in the Fortune 500 is hacked, in the absence of two things:

1. A clear definition of "hacked," and
2. Some data supporting the assertion that everyone in the F500 fit that definition.

So we got to talking about what data would constitute proof, and I suggested that having one host in your IP range detected as being a member of a botnet could qualify as "hacked."  This could theoretically be straightforward to determine, if you had access to enough threat intelligence feeds and/or had enough sensors to compile a list yourself.  Now, there are some open source feeds, but for the most part companies that create their own feeds want to monetize them. (One laudable exception is Microsoft, which has been testing a feed that it would offer free of charge to law enforcement, CERTs, foreign governments and private corporations.)  If you have one machine on a botnet at some point in time, that could designate you as hacked, at least until you scrubbed it. 

But is it the tip of the iceberg?  Does having a bot automatically mean that more nefarious things are going on besides just selling V1agr4 or perhaps DDoSing the Anonymous target of the week?  This is the risk calculation that we need more data to perform, and it's one that the C-suite would really appreciate.

So I'd love for someone to comb through their incident response data and present statistics on what, if anything, followed after an initial malware infection.  If you could say that (for example) 70% of the time, it was simply used to grab CPU without necessarily trying to grab passwords or data, and 20% of the time it led to password compromise for financial theft, and 10% of the time it led directly to IP theft, those would let us infer probability.  It would depict in a more concrete way just why being part of a botnet is a symptom of something more dangerous.

By association, any company that found itself with membership in a botnet could reasonably suspect that it was even more compromised than that.  It might take the time to look further.  (There are plenty of enterprises that just wipe the affected machine, re-image it, and go back to work.)

The other question is whether membership in a botnet should be considered public data.  If anyone on the Internet can discover it, you could argue that it's the kind of compromise that anyone can report.  The fact of an enterprise's system interacting with another host on the Internet isn't confidential; it (like a public posting) is just assumed to go unnoticed.  Would a company have grounds to complain if its membership in a botnet were revealed, based entirely on publicly available information outside of its private network?  I am not a lawyer, but sometimes I want to ask lawyerly questions like this.

Following this chain of thought, anyone could set up sensors, collect data on botnet membership, and publish it widely.  Someone could collect statistics on just how many of a company's systems were in a botnet at any given time.  In the absence of any other data, could this be used as a poor man's Compromise Index?  It would be like someone noting how many broken windows you could see in a building: one indication of a breach, but without any way to know what, if anything, happened or was taken after the windows were broken.

And armed with that data, someone could actually make a substantiated claim that the whole Fortune 500 is hacked, without hearing the clackety-clack sound of thousands of eyes rolling.

After that comes the question, "So what?"  Would this kind of naming and shaming prompt any additional diligence on the part of these organizations?  Would it make regulators sit up and take notice?  Call me a skeptic, but I suspect that botnet membership is so widespread that people would assume it happens to everybody -- just like ant invasions -- and it wouldn't be condemned except within the security echo chamber.  I could be wrong.  Either way, I'd love to find out.

[DISCLAIMER: I am not encouraging anyone to compromise any systems themselves without the permission of the affected organizations.  I am not suggesting that anyone collect data that can only be gathered directly from those systems.  I am certainly not recommending that anyone leak confidential data, even if it's with the best of intentions.  Do not try this at home.  Ask your parents before calling.  And so on.]

For great justice. I mean security.

The Verizon Data Breach Investigations Report (available here) was basically another year of "all your POS are belong to us."  Which is depressing, but not at all surprising.  As you know, I talk a lot about what I call the Security Poverty Line, and how smaller organizations that are IT-poor tend also to be security-poor.  Moreover, because security and IT are so often separate, security becomes optional, a luxury and an omission for the small business that doesn't know it has something to lose -- or even if it does, it hasn't got the faintest idea of how to go about addressing it.

Enter the DBIR, and what I think is one of the most helpful steps ever taken to address this security-poor population.  On page 62, the redoubtable Verizon Risk Team has created a cutout sheet that you can hand out to your favorite retail, hospitality and food establishments.

Greetings. You were given this card because someone likes your establishment. They wanted to help protect your business as well as their payment and personal information.
It may be easy to think “that’ll never happen to me” when it comes to hackers stealing your information. But you might be surprised to know that most attacks are directed against small companies and most can be prevented with a few small and relatively easy steps.

 And the cutout doesn't get too fancy or preachy; it basically recommends two main things:  change your default passwords, and make sure you have a firewall.  And if you're not the one who is in charge of these things, make sure your vendor does them.

The beautiful simplicity of this is hard to overstate.  The cutout doesn't invoke FUD; it just says, "Hey, we've seen a lot of this and you might want to be careful."  The language makes it accessible to someone who is busy running a business, and who doesn't have time to delve into arcane IT concepts.  It tells them the most important things they need to do, and puts it in a digestible format.

I hope people will go to the trouble of making copies of this cutout and giving them to as many franchises and local businesses as possible.  It would also help to have a simple and cheap answer to the question, "How do I find out more about this?" if the business owner should ask.  I know of at least one security professional who makes a point of going to speak about security at chamber of commerce meetings, and we need more of this kind of outreach.

For the security-poor organizations, the best thing we can start with is to arm them with information -- the kind of information that is useful to them.  If we made a concerted effort to reach out to this underserved population, I'm hoping the DBIR numbers would get smaller over time.

Going back to the stack.

In the spirit of trying to suggest solutions, here are a couple of thoughts about what an enterprise can do first off to make security a little better.

It's bothered me that infrastructure is being administered more horizontally than vertically these days.  Everyone specializes in a different layer: network, OS, utilities (such as Exchange), middleware, applications, etc.  And this gets worse when you outsource one or two layers to "the cloud" (think IaaS, PaaS and SaaS), so that you have to coordinate with a third party to troubleshoot something.

Back in the Pleistocene era, system administrators knew their systems like they were their babies.  They knew everything that was running on them, how they were configured, who they talked to, and they knew when something was "off."  I know sysadmins that would regularly help a developer debug code, and they were often better at it than the developer, because they also understood the underlying environment better.  They could troubleshoot all the way up and down the stack, and you went to one source to do it instead of having to get a conference call together with 3rd level engineers from four different companies.  (Seriously, I know of a data center that had five different networks owned by five separate entities.  Think you could figure out what happened to a packet?  Think again.)

So one thing that enterprises can do is simply to get control of their layers as much as possible.  Know what you have, know where it is, and be able to cause changes to it when you want to.  That sounds so obvious as to be not worth saying, but I don't know of any admins who know more than about 500 hostnames by heart, and many times the environment is so dynamic that boxes come and go without any centralized tracking keeping up with it.  (And I'm not even talking about VMs.)

If you already have parts of your infrastructure outsourced, go over your contracts and strengthen your relationships with your providers.  You want them to be able to give you logs, for example, within a few minutes of the request.  You also need to have the right technical level support people on call without having to fight your way through first-level script-readers.

And finally, go back to designating "stack admins," who are generalists rather than specialists in one particular technology.  It should be their job to know as much as possible about any given system.  You can fit this into DevOps if the developers truly know the lower layers.  A stack admin is your best hope for knowing what normal operation is, and for alerting you when something doesn't smell right; they're also the best at understanding the implications of any given planned change (such as changing the ports an application uses without creating the corresponding firewall rules).

Start with knowledge, and then work your way to control.  Notice we haven't really touched on security yet; that'll come later.  But knowledge and control are basic building blocks of security.

In 50 gigabytes, turn left: data-driven security.

I love Scott Crawford's research into data-driven security.  I agree with him that IT operations and development can both benefit from the right security data -- where "right" means at the appropriate level and relevant to what they're doing.  It also has to be in the right mode:  an alert should be based on a conclusion drawn from the analysis of data (20 failed logins per second = someone is using automation to try to break in), based on an event or confluence of certain events.  Once someone in IT needs to perform an investigation, the need changes to looking at more atomic data (exactly which logins are being targeted, whether they're active or disabled, etc.).  In other words, the details need to be available on demand, but they shouldn't be shoved at the IT staff in lieu of useful alerts.

Another kind of data that is useful is situational data:  how things are configured and what is happening during "normal" operation.  Viewing all the responses from a database is too much to ask of a developer -- but the developer would benefit a lot by knowing that some queries are taking 25 minutes to return (do you suppose that would have some effect on application performance?).  This is the sort of data that is incredibly useful, but setting up every possible abnormal situation to trigger an alert is way beyond the scope of an overworked operations team.  Sometimes you just have to sit down and do some exploring every so often, to find out these sorts of operational problems.  Packet captures can teach you things you can't learn any other way -- if you have the time and skills to read them.

Because detection is expensive.  It requires the luxury of having staff both knowledgeable in the technology and in the context of those particular systems, and having them devote a lot of their time just to sitting and looking at things, sorting out what's normal from what's not.  Those are the kind of costly eyeballs that have been transferred so frequently to managed security service providers.  It's the kind of thing you pay consultants to do, because if your staff weren't completely occupied with keeping the infrastructure running, you wouldn't be allowed to keep them.  Data analysis today is expensive, and it's a one-off deal unless you can find economies of scale somewhere.

Yes, automation is getting better, but it's not there yet.  There are still too many alerts taking up too much time to sort through (particularly in the tuning phase).  IT staff get hundreds of emails a day; they can't handle more than two or three alerts that require real investigation.  (By the way, this is why operations often can't respond to something until it's down -- it's the most severe and least frequent kind of alert that they receive all day, and they don't have time to chase down anything lower-level, like a warning message that hasn't resulted in badness yet.)

If you break security events down, you're generally looking for two kinds of things:  normal activities that are being done by the wrong people (as in, from a Tor exit node through your administration console), or abnormal activities that are being done by the "right" people (internal fraud, or someone has taken over an authorized account).  And by "people," of course, I also mean "systems," but at first glance it's sometimes hard to tell the difference. 

This determination of "wrong" and "right" is a security activity, and for the reasons I listed above, operations people may not care that much until it makes something happen that they have to fix.  If someone wipes a database, they'll care a whole lot, but if there's some unusual encrypted traffic leaving the enterprise on port 80, not so much.  A fully leveraged (i.e. overworked) ops team doesn't have time to analyze alerts at that level.

"Wrong" and "right" to the business is on a completely different stratum, and it's one that's hard for automation to reach today.  Executives care when it gets to the level where they have to do something about it, like fire someone for looking at patient data, or talk to the press about a breach.  They care when an event starts to present the risk of legal liability or increased cost.  But you can't bring them alerts like that until you have digested everything at a lower level and put together enough evidence to reveal a business issue.

And finally, historical data can be extremely useful in determining what works in security and operations and what doesn't.  But that kind of data has to be analyzed in a different way from real-time operational data or situational data.  It requires a different model that caters to the requirements of risk analysis -- and that, too, is expensive, even assuming you know how to do it today.  (Hi, Chris.)

My point here is to say that data-driven security is where we need to go, absolutely.  But there is no single path to take with the data we have; there are a number of divergent paths that are all needed in the enterprise.  We also need to be able to drive the data in the right delivery directions -- which means that we need a really good data navigation system.