How to Track a Security Program

When you're running a security program for a whole company, like I was at Watershed, there are a lot of things to keep track of and a lot of things to do.

I found, though, that these weren't just work tasks like you might track with Jira or Linear, and they weren't quite OKRs, and it wasn't just audit-related tasks that might show in in Vanta.

When you zoom out, an information security program is really about Things You Want To Be True. What are the necessary steps to achieve those states of the world? How can you tell if you're making progress? How do you know when you're done? And how do you make sure those things remain true once you've hit your target?

However you track these things, you want your system to be flexible enough to capture your full range of goals, so you can see the big picture.

One of my deep convictions in this area is that you can have rigor and specificity early on, even in small companies, without needing a bunch of SaaS products or integrations to do it.

Let me show you what I mean.

Let's say I'm working with a company that just has a handful of people, and they don't yet have a way to track employee laptops. Maybe I'm following the Cyber Defense Matrix and looking for a way to identify devices, or maybe I just intuitively know that's a useful thing to do.

So I have a goal:

Keep an up to date inventory of all company laptops, including the type (e.g. Mac, Windows), serial number, and current user.

And I have a couple of things I want to be true to support that goal:

• Establish a centralized way to track workstation inventory
• Each employee has at least one assigned device

Let's write that up in Markdown, along with a goal header, and identifiers for each requirement.

# SEC-DEVICES-IDENTIFY-100

Keep an up to date inventory of all company laptops, including the type (e.g. Mac, Windows), serial number, and current user.

Requirements:
- 10: Establish a centralized way to track workstation inventory
- 20: Each employee has at least one assigned device

I've written a little program that will take that and display it in a nice way. It also knows that each goal needs an owner, so it puts that in as the first requirement.

In my system, I have simple text files that provide data in the TOML format. This is how we start showing progress against our goals.

I'll add a user, and show that they have responsibility for this goal.

[[users]]
email = "annie@example.com"
name = "Annie Clark"

responsibilities = ["SEC-DEVICES-IDENTIFY-100"]

Now my web view looks like this:

We're going to use this tracker as our system of record, so I'll add data capturing that:

[[applications]]

name = "Monorail Program Tracker"
supportsRequirements = ["SEC-DEVICES-IDENTIFY-100-R10"]

Right, now it's time to actually track the devices. Let's add a data check.

[SEC-DEVICES-IDENTIFY-100-R20]

asset = "users"
filter = """
    (user) => !user.groups?.includes("Contractors")
"""

check = """
user => user.devices?.length > 0
"""

This is pretty simple. We want to consider all users who aren't contractors, and we want each person to have at least one device.

Let's add a few more user records to make this more interesting. Now, our user list is:

[[users]]
email = "annie@example.com"
name = "Annie Clark"

responsibilities = ["SEC-DEVICES-IDENTIFY-100"]

[[users]]
email = "billie@example.com"
name = "Billie Eyelash"

[[users]]
email = "fred@example.com"
name = "Fred Brooks"

Now our web view looks like this:

Actually, Fred is a contractor, so let's add that.

[[users]]
email = "fred@example.com"
name = "Fred Brooks"
groups = ["Contractors"]

Great, he's off the list now. But we're still at 0%, because he's not included in the population. Let's add devices for Annie and Billie.

[[users]]
email = "annie@example.com"
name = "Annie Clark"

responsibilities = ["SEC-DEVICES-IDENTIFY-100"]

[[users.devices]]
type = "Mac"
serial = "abc123"

[[users]]
email = "billie@example.com"
name = "Billie Eyelash"

[[users.devices]]
type = "Mac"
serial = "abc125"

This approach also extends well to deadlines. Let's say a company has a web app and a mobile app, and they both need to be pen tested at least once a year.

With this application data:

[[applications]]
name = "ExampleCo web app"
tags = ["prod"]

[[applications.penTests]]
startDate = "2025-01-14"

[[applications]]
name = "ExampleCo mobile app"
tags = ["prod"]

[[applications.penTests]]
startDate = "2024-05-08"

And this check:

[SEC-APPLICATIONS-PROTECT-100-R10]

asset = "applications"

filter = """
    (app) => app.tags?.includes("prod")
"""

check = """
    (app, { addTime, today }) => {
        if (!app.penTests) {
            return { pass: false, message: "No prior pen tests" }
        } else {
            const latestPenTestDate = app.penTests.map(p => p.startDate).sort().slice(-1)
            const deadline = addTime(latestPenTestDate, 1, "year")
            return {
                pass: deadline > today,
                deadline,
                message: deadline < today ? `Deadline was ${deadline}`:''
            }
        }
    }
"""

We get a view like this, where everything is passing, but we have a visual indication that one of our deadlines is approaching:

You could imagine having a whole set of these goal cards that track an entire security program, including requirements that are in scope for audits, tracking upcoming goals and roadmap, while also making sure that the things you've completed stay green, all with very minimal tooling.

I suspect, too, that this kind of structure is useful outside of security, as well. I could see this being useful for any situation where you are interesting in ongoing operational excellence, and leveling up the overall program. There are parallels in HR, IT, engineering operations, sales, program management, customer support, and more.

All the status checks are data driven, so you're never relying on somebody just deciding something is done–it has to be supported by some level of evidence. And the computational checks mean you have to have the right level of rigor to really tell if you're done or not.

In a more complex system, you'd probably want to automate pieces of this–nobody is going to want to enter serial numbers by hand when you have 1000 employees–but you don't have to wait for scale to achieve rigor, and you don't need complex systems to achieve rigor.

And if you want some help building up a well-structured, rigorous security program that gets green and stays green, get in touch! That's one of the things I do as a vCISO and security consultant.