In December 2019, we published the Top 11 Habits of Effective CISOs that provided our perspective on those habits that resulted in effective cybersecurity programs. One of our identified habits was Seeking Security Alpha.

In cybersecurity, it has long been assumed that the attacker has the advantage and that defenders must deploy a disproportionate amount of resources (time, money, etc.) to even try and maintain some parity.

In the financial industry, there is a term called “seeking alpha” for those investment managers looking to exceed standard performance on a risk-adjusted basis. Recent work by the New York Cyber Task Force implies that CISOs can seek security alpha as well – that is spend a dollar on defense that causes an attacker to spend a disproportionate amount on offense.

In seeking security alpha you should be deploying strategies and solutions that increase the cost to the attacker and provide you with maximum security return-on-investment for the threats and risks your organization faces.

In this piece, we’ve conducted interviews with two successful CISOs to provide insight into how they view security alpha issues. Mark Weatherford is a highly experienced and successful CISO who has worked in the public sector at both the state and federal level and also as a CISO for multi-billion dollar commercial organizations. Our Global FS CISO currently works as the Global CISO at one of the largest financial services firms in the world and has 25 years of experience working on cybersecurity and risk management issues.

Their responses provide direct insight into how they work to improve the ROI of their program and increase attacker cost.

OODA: Are there any particular strategies that you’ve used to increase your ROI for all or parts of your cybersecurity spend?

Global FS CISO:  Conceptually, working for a bank that’s a leader in digital transformation of financial services, the ROI proposition of cybersecurity is that it preserves and extends the value of the digital product and service. beyond what it would be without protection from cyber risk. But that’s a rather theoretical concept. More tangibly, we use a framework that starts and ends with the threat. What is the threat, what are they after, why, how, etc., based on a very strong cyber threat intelligence capability. Then, using frameworks like the MITRE ATT&CK framework, we map out, what capabilities are adversaries likely to deploy in pursuit of the what? why? how?, and how would those present themselves on our environment? Using a defense-in-depth approach, where we try to see how far up the kill chain we’re likely to be able to prevent/detect/respond (yes, NIST CSF), we map out defense capabilities to face off against threat capabilities. This approach allows us to understand and prioritize “bang for the buck.” We try to affirmatively measure readiness as well – how faster this month than last month for patching, purging phishing emails out of inboxes, detecting the red team, etc. – from our investments, how more effectively did we enhance our defense capabilities against specific active threat capabilities, and how better are we getting at detect/respond/contain?

This is the best we can do in the concept of return on investment. When you invest in security properly, nothing returns. When security works, nothing happens, which makes it difficult to “prove the negative” i.e., demonstrate ROI and causality of “because I did this, nothing happened” which is vulnerable to the counter “nothing was going to happen anyway.” So generally, I try to steer conversations about a risk function away from the concept of ROI – which is pretty easy to do in a bank, where the executives inherently grasp the difference between measuring risk and measuring ROI (even if they don’t necessarily have expertise in this particular specialized risk).

Mark Weatherford:  Since my CSO/CISO experience spans both public and private sector organizations, including critical infrastructure, a security software vendor, and travel/hospitality, the single theme that seems to resonate with leadership as ROI is a clear understanding of risk across the organization.

To understand risk, you need to understand what is valuable and important to the organization so you can focus your resources. Asset management, including data management, is the single most important constant. If you don’t know what you have, you cannot protect it. This simple issue still seems to be the one thing organizations continue to struggle with. By focusing on identifying your hardware and software assets, where data resides, how data is being moved around, what data is encrypted, and who has access to the data, you can target your resource allocation to what reduces risk the most. I think we often try to make security harder than it is by looking at the candy store full of cool security tools when in reality, fundamentals like asset management, patch management, endpoint protection, and security training can get us to the 80% security mark.

You may remember Wendy Nather and Andy Ellis coined the term “Security Poverty Line” back in 2013 (I think) and I still think this is one of the most important concepts for security leaders to grasp. If I know where my security poverty line is, I can make very rational decisions about my best ROI bang for the buck.

OODA: What are the most important metrics you collect to determine the success of your security program?

Global FS CISO:  From a residual risk perspective, I collect various metrics on basically four different areas: the threat activity, hygiene and controls effectiveness in the infrastructure, controls effectiveness and compliance with cybersecurity policies and practices among the business divisions, and risk profile presented by third parties who have access either to our infrastructure or our sensitive data. We unpack each of these four categories into 3-4 metrics each, we report up to execs, e.g., spearphish victimology (threat), external-facing host patching (infrastructure), affirmation of least-priv/need-to-know on access rights (business divisions), # of known or aging high-sev risk issues among top tier vendors (third-parties), etc.

Mark Weatherford:  A baseline of assets is the key metric that allows you to manage changes in your CMDB, manage safe and unsafe uses of data, and identify gaps in the overall asset landscape.

Another key metric is understanding the threat environment from the perspective of what makes me a target and who is interested in taking advantage of that target.

OODA: How do non-tangible or unmeasurable elements play into the equation?

Global FS CISO:  Depending on the “non-tangible” it can provide context to the fact that residual cyber risk will always be high relative to other operational risks. The purpose of these metrics is to serve two audiences: the cybersecurity staff across the firm (by identifying areas where trends are deteriorating, defects are growing or hygiene is slipping), and governance (by illustrating whether the metrics and underlying residual risk point to broken defense capabilities, a lack of due care, or a situation that makes us outliers among our peers). Even with the non-tangible elements, the important question is, is the trend going in the right direction, and if not – why, and what’s the plan to reverse?

Mark Weatherford:  I have always worried most about the insider threat as the most concerning unmeasurable risk. If that insider is part of the security organization the risk is amplified exponentially. I can spend a lot of money on tools and services but a dedicated malicious insider will find a way to circumvent and/or cloak their activities.

OODA: What types of technologies have traditionally provided the highest security ROI?

Global FS CISO:  Training and awareness, mainly because the cost is relatively low, but the “return” in a more security-conscious and aware workforce is massive compared to the investment. Otherwise, analytics (AI, ML etc) are going to be the next big ROI space, in my opinion.

Mark Weatherford:  Asset management and security awareness training.

Asset management – The simple act of identifying all of the physical and virtual products in an IT environment creates confidence that you can then effectively manage and secure the environment. Of course, this is much more simple than the reality of just knowing what you have and where it is (and getting harder in the cloud era) but even a periodic baselining provides some assurance that you at least have a starting point.

Security awareness training – Easily the most important and least expensive thing an organization can do to decrease security risk. I think we often forget, because we live in the security world 24/7, that most employees in our organizations don’t know anything more about security that what the latest headline is, and they don’t even understand that! Every time I conducted employee training, which often consists of nothing more than telling stories about security events, I am amazed at how little most people understand about security. Constantly re-emphasizing security hygiene and security manners is one of the most efficient and effective ROI indicators.

OODA: Which technologies have worked best to increase attacker costs and disrupt attacker ROI?

Global FS CISO:  Same as above – i think defender and attacker ROIs are inversely proportional

Mark Weatherford:  Security awareness training is certainly one of the best ways to increase hacker costs by raising the bar across the board. I think it’s sector/business dependent, but in my last job where we had thousands of developers writing the public facing code our company depended on for practically 100% of our revenue, the money we spent training our developers to understand, think, and act with real security knowledge was the best money I spent by far. And we were able to measure that success through savings of time and money in the software development lifecycle.

Additional Reading:

11 Habits of Highly Effective CISOs

Cyber Sensemaking

Deception Needs to be an Essential Element of Your Cyber Defense Strategy

OODA Releases a Traveling Executive’s Guide to Cybersecurity

For Executive Protection, Physical and Cyber Security Have Fully Converged

10 Red Teaming Lessons Learned Over 20 Years