Avoiding Snake Oil: SaaS Challenges

It can be awkward to talk to other security geeks between when you accept a job at a startup and when you have worked there long enough to figure out what is actually going on:

Other Geek: What do they do?
Me: Client security using a SaaS model.
Other Geek: How does that work?
Me: They filter the traffic between the client and whatever it's talking to on the Net. It's cool because the customer configures stuff once for all their sites (clicky, clicky) instead of having to install & configure a bunch of boxes at each site. Plus it can protect people working from coffee shops.
Repeat Indefinitely:
Other Geek: Completely reasonable questions about successively more detailed engineering and business concerns, all of which boil down to "How could that possibly work?"
Me: Increasingly vague hand waving, as the questions proceed further and further away from my limited knowledge of what my soon-to-be employer actually does.

The only plus side of conversations during this period is, you know you aren't revealing anything secret, because you don't know anything yet.

As I finish my first week of work at Zscaler, I have a lot more idea of what is going on, and I have acquired my first self-assigned mission: making sure that what we are selling is not, and never becomes, snake oil, whether or not all other available solutions are unctuous or reptilian. Because of our line of work I'd say we have some risk in this area (as many of my friends and acquaintances were keen to point out), and I'm not too worried about ensuring we are the best available solution, since plenty of my coworkers are focused on that.

I see 3 keys to avoiding snake oil, from a vendor's perspective:

Accurate customer expectations
The customer must understand what to expect, and their understanding must match what is actually going to happen. Since the vendor knows what is actually going to happen, the vendor has to make sure the customer has the right impression. This is tricky for any security product. As security people know but other people would usually rather not hear, 100% protection is not possible. To make things worse, 90% protection doesn't mean anything, since even within the security industry there is no simple, well-understood metric for describing how much protection a particular product provides. Security people would normally describe specific kinds of protection provided and the strategy for implementing each, but non-security people don't have the background to understand what this implies for them. And because each situation is different, the security people need more information about each customer to be able to explain the implications.

Real benefit
The product must actually do something to protect the customer in ways that matter. Since real benefit can be in the eye of the beholder, security vendors need to consider benefit from the viewpoint of a hypothetical security-knowledgeable customer; a representative such person may or may not exist. Some things are clear: it is valuable to protect against attacks that are actually occurring and would have worked if the product hadn't stopped them. It is less clear whether protecting against attacks that are still theoretical is worthwhile. It is also apparent that the scope of protection must cover gaps in other protections the customer is planning to use. This is a challenge for security as a service today, because a SaaS model implies a new way to partition the problem, so existing products may not mesh well. On the deeply unclear side: for non-security people, it isn't about viruses, botnets or cross-site scripting, it's about somebody erasing important data, clogging the network, stealing intellectual property, and so on. But protection tends to be organized by security geeks, i.e. by class of attack it defends against. Whether protecting against viruses, but not botnets, is enough to be valuable depends on customer expectations and scope considerations. Finally, the lack of metrics applies here, too: currently it is very difficult to quantify how much protection is enough.

Verifiable benefit
The customer must be able to verify that they are actually being protected in the way they expect to be protected. This can be tricky for the same reasons setting accurate customer expectations is tricky, but for an in-the-cloud security service, it is even harder. After all, in the ideal world, a customer is protected but not inconvenienced, i.e. the customer never notices the service. In most cases, all a customer has to go on is the vendor's word that protection has taken place. Certainly security in the cloud is blacker than black box: a would-be tester doesn't even have a box to play with directly, and the box is intended to change regularly without warning or customer intervention. Few companies are paranoid and wealthy enough to employ skilled security folks to set up an ongoing independent monitoring system. In this situation, it will be tricky but even more important for vendors to supply an answer beyond the infamous "trust us, we know what we're doing".

Until now, I've always been on the other side of the fence, poking at products and determining whether they live up to my exacting standards. The view is a little different from over here, but I feel confident. Keep watching and we'll all find out how Zscaler meets these challenges.

-- Brenda