Security Confidential S8 E7 Ted Harrington

Manoj Tandon
This is your host, Manoj Tandon, and welcome to another episode of Dark Rhino Security Confidential. Today, we have an awesome guest. He’s one who really doesn’t need much introduction, but we’ll do it for the heck of it anyway. He’s Ted Harrington. Ted, for those of you who may not have seen the book, is the author of Hackable: How to Do Application Security Right. He is also an executive partner at Independent Security Evaluators, a very well-known ethical hacking firm that has hacked cars, medical devices, and password managers, to just name a few. But you know, his clients are folks like Google, Netflix, and Amazon—you name it.

He’s really the real deal, and we’re honored to have him on the show. Thank you so much, Ted, for joining us today.

Ted Harrington
Yeah, I’m excited to be here. Thanks for having me.

Manoj Tandon
Absolutely, absolutely. There are so many questions that we have and we have such little time, so we’re going to try and make the most of it.

So, you know, there’s something while we were doing a little bit of homework on you that you said that I’m going to paraphrase, and if I butcher it, please correct me. But I really want to get your enlightened feedback on it. Okay, so I think the statement was to the effect that many companies base cybersecurity on the notions of frameworks and processes of today that are based on the experiences of yesterday, and what really needs to be done is the use of ingenuity to solve these problems. And I’m paraphrasing this, but that statement is a powerful one to me, and much is said in that.

Much is said in that, so please expand on it a little bit and enlighten the audience.

Ted Harrington
Sure, yeah. So here’s the problem I think that we’ve identified in this topic that you brought up: the way that a lot of organizations think about security is—and I’ll simplify the language of what we’re describing—they say, “Give me the checklist. Give me the list of things I’m going to do; I will go do those things.”

And inherently, that’s actually not a terrible way of thinking about life in general. Like, I don’t know, I just got a new exercise bike delivered yesterday and I was like, “Where’s the list of stuff I have to do to set it up?” Literally, that’s the way we’re wired. We’re like, “Give me the list, I’ll follow the list.”

But security doesn’t really work that way because, inherently, what it requires to compromise a system to arrive at a security breach requires creative thinking, and requires new ways of doing things, and requires literally expecting the unexpected.

And so, if our entire approach to security is “Show me the list of things that are known to be problematic and I’ll just do that list,” we’re automatically excluding creativity from the process. Hacking is, by definition, a creative process, and that’s a really significant disconnect.

So if we’re trying to defend against an attacker, an attacker is creative, and if we’re removing creativity, what have we actually achieved? Really nothing. And we’ve made ourselves maybe feel good; we’ve made progress relative to doing nothing, but we haven’t effectively solved the problem.

Manoj Tandon
You’re absolutely correct. But most security teams and departments aren’t structured that way.

Most requirements aren’t structured that way. A lot of security is driven by procurement, really, if we simplify it.

Ted Harrington
Absolutely right. Procurement, unfortunately. When I was writing the book, that was actually kind of an interesting moment for me as I realized that. I was thinking about our customers who are featured throughout the book—not by name, but their stories are featured throughout—and I was really asking myself, “So why do they invest in security?”

My initial reaction was, as an optimistic guy, “Oh, because it’s the right thing to do. Security matters; it’s important.” And then I realized it’s like, well, yes, they all do believe that—all of my customers believe that—but they also realize that this is going to help them build trust, earn customers, get contracts faster, stuff like that, and differentiate from competitors.

So procurement is driving this, and when procurement needs a way to think about, “Well, how do we know it’s secure?” that’s where we arrive at this idea of frameworks, checklists, and compliance.

That’s really where the problem is coming from: there has to be some sort of shared language. And really, the difference here is between the companies who get security right and the companies who don’t. The difference between the two of them is the ones who are satisfied with just checking boxes versus the ones who say, “I actually want to think like a hacker thinks, and I want to understand my system in that way, and that’s how I’m going to improve the system.”

Manoj Tandon
Well, if those categories were swimming pools, one is an Olympic-sized pool and the other is a little kiddie blow-up thing. I’m trying to—there’s really not that many.

Ted Harrington
But there’s a hose between them. The little kiddie pool is getting filled up slowly, and the progressive companies are in the little kiddie pool and they’re winning. The progressive companies are winning in the kiddie pool right now, while everyone else is in this disgusting, German festival pool. It’s gross. You want to get out of that pool and get in this—it’s not a kiddie pool, it’s like the Beverly Hills mansion pool, but no one’s in it. Go get in that pool.

Manoj Tandon
So which brings us to the question: is cybersecurity, for the vast majority of organizations, just consciously irrelevant?

Ted Harrington
I mean, no. And I’m saying no to the way you phrased the question. Maybe if you’re asking the question in a slightly different way, I might say yes to it. But the two parts to that I say no to in particular are “consciously so” and then “irrelevant.” Maybe that one is where we could say, “Maybe.”

Manoj Tandon
Let me give you some context on it because you’re a smart guy and you’re making some great observations. So, what is meant by that question is—when we, at least in our limited experience, are engaging with a variety of clients which are primarily small-to-medium businesses, that’s where we focus—we see a lot of times that attitude of exactly what you’re saying: “Let’s procure a technology, let’s get a checklist, and let’s say we did something.”

And we can go back to our executive team in charge, or to the people that are responsible for our liability insurance and our legal team, and say, “Look, these are all the protections we’ve put in place; we’re good to go.” So they’ve made a conscious choice of doing what might be perceived as the right thing, but it’s not really the right thing, if you will.

Ted Harrington
That’s a “yes” on that. I would primarily agree with that, and that is the difference. Because the original phrasing was that they’re consciously doing the wrong thing. They’re consciously doing what they believe to be the right thing, and I think that’s an important distinction.

Because if someone’s listening to the show and they’re saying, “Well, you know, I already do this, this, and this,” but if this, this, and this aren’t the right thing, that’s really the problem. It’s not that they’re sitting around—I think most companies are not sitting around doing nothing. I think a lot of companies are doing something; it’s just not the right thing.

And actually, that was one of the big motivations to write the book. It kind of broke my heart in a way when I saw all these companies actively investing time, effort, money, and resources, but in the wrong thing.

It’s like if I wanted to—I’m coming up with a metaphor on the spot here—if I wanted to run a road race, and it’s a 5K, and my goal is I want to run it faster than 20 minutes, so I’m like, “All right, I’ve got to do a bunch of stuff that’s going to make me faster.”

And the stuff that I’m doing is like… I read some article online and the article is like, “Well, here’s how you run faster: you do a bunch of bicep curls.” And that’s it. And I’m like, “Fine, I’m going to do all the bicep curls in the world,” but that doesn’t make me any faster as a runner. And so it’ll be stuff like that; they’re looking at the wrong list.

Manoj Tandon
Very much so. And that’s not a bad analogy, although that might increase your conditioning a little bit. I mean, I used to be a runner at one time; I didn’t look like this always. But I mean, everyone wants to pick up arms. That’s a separate, totally different topic there.

But when we look at the way in which we started this conversation—of when you look at a cybersecurity program and how they’re built and how budgets are procured—you often have a group of people going to people who are going to write a check, trying to explain what specific pieces they’re going to buy and how that might develop an ROI.

That is much easier to do when you put finite boxes around things, versus if you try and go to them with the concept of ingenuity, which by definition doesn’t have a box around it. It can’t be in a box. If it’s genuinely something new, how do you actually, practically procure a budget to do what needs to be done?

Ted Harrington
Yeah, and that is actually maybe one of the biggest challenges for any company to approach their security mission. If we think about it more abstractly, you’re talking about measuring the absence of something.

So like, let’s say you want to do something where you can measure the presence of something, like “I want to invest $100,000 in this marketing campaign, and as a result, we expect it to produce a million dollars.” Either it does or it doesn’t. And if it doesn’t, you’re like, “Well, that didn’t work. We move on.”

With security, you’re like, “I want to invest $100,000 and, as a result, we won’t get hacked.” And you’re like, “Well, is it just because we were lucky, or was it because it worked?” And you really can’t measure that.

So what I find is an effective way for organizations to think about that is: how can you correlate the investment to the reduction of attack surface or reduction of vulnerability? Can you actually measure that you have discovered vulnerabilities and eliminated them?

One of the things I talk about in the book is this idea that we did a whole analysis of our own—it was about 13 years of data of different companies we’d worked with—and we could actually correlate level of effort to vulnerabilities discovered on average. Obviously, there’s a distribution, and you can of course assign a price to level of effort.

So now you can look at it and say, “Okay, well, you can reduce your price to some level, but then your output becomes significantly lower as well.” And so the question becomes: what’s the sweet spot? There’s this idea of an S-curve, right? Where it’s too little, just right, and then too much. We want to be in that middle area. And so I give some guidelines in the book about how to think about that so that people can do exactly this: invest appropriately.

Manoj Tandon
Yeah. I think developing the language… that’s one part of the equation that I’m going to blame some of our fellow cybersecurity professionals for: they often need to develop their skills in the language of business, and they need to understand the business that they’re trying to protect—what drives that business—so that they can frame things in the context of the business.

Ted Harrington
Yeah, I agree with that. I had two interesting stories I can tell that sort of draw that contrast.

Manoj Tandon
Please.

Ted Harrington
The first story was—and both of these stories are about two different individuals who were very senior security analysts at our company. The first story was when this first analyst came to me with this really amazing set of vulnerabilities that he discovered in small-office/home-office routers.

As he’s describing it to me, he’s talking about, “We did this, we did this, we did this, and we got root.” And I said to him, “Awesome. Why does that matter?” And he’s like, “Ted, what are you talking about? We got root!” And I’m like, “No, I know, but why does my grandmother care?” And he’s like, “Ted, it’s root!” And he just walked away.

Of course, after we figured it out, after that initial conversation, he wasn’t speaking in the language of why someone would care. He figured out how to communicate very, very well after that—that was a learning point for him.

But the second story was a few years later with a different analyst. He came to me with a different piece of research; it was around cryptocurrency wallets. He said, “Ted, I was able to successfully predict the keys.” And I was like, “Awesome.”

I already knew, just like with the root story, I knew why it mattered. But I wanted to push him; I was like, “So why does that matter? Why would my grandmother care?” And he didn’t even hesitate. He just tells me this metaphor, and I’m like, “I’m putting that in the book!”

He’s like, “Well, it’s like this: imagine you go to the beach and you pick up a grain of sand, and then I go to the same beach and pick up any grain of sand. How likely is it that I pick up your grain of sand?” And I was like, “Oh, well, that’s impossible.” And he goes, “Right. Now multiply that by every beach on Earth, and multiply that by like a gazillion planet Earths. That’s how unlikely it is that I would guess your key. And I just did it 732 times.”

I was like, “Wow!” Now I understand the impact of what this finding is. Because he can predict the key, that means for 732 wallets, he could go steal their money. That was effectively the impact.

These are the kind of things that, in the security community, we need to be able to do: speak in metaphors, speak in stories, speak in business terms, speak in money, and not so much speak in the technical details. Because, frankly, the people we’re telling them to don’t care. They’re like, “I don’t care how you did it; just tell me what it means.”

Manoj Tandon
Exactly. I couldn’t agree with that more, and it’s an endemic problem. We see it all the time. I hope I’m not saying it—you’re saying it, and maybe now it’ll get a broader reach and people will listen a little bit better.

So, one thing that I wanted to circle back to was: your background started in psychology, correct?

Ted Harrington
Yeah.

Manoj Tandon
Isn’t that your origin? Why do a lot of programs—in fact, no programs in cybersecurity, I’m just going to state it—not have the psychological aspects of the hacker’s mindset incorporated into the training of a defender?

Ted Harrington
I don’t know why they don’t. Probably just because it’s a relatively new field in academics. I mean, it can be measured in years or maybe decades, whereas how long have we been teaching philosophy, right? Since the dawn of academia.

And so that’s probably the primary reason.

My understanding, though, is that a lot of these programs do, to some extent, maybe even unintentionally, inject some of this psychological perspective. Because you really can’t study security—sorry—if you don’t think about motivation.

And when I was studying—I studied psychology in undergrad—I didn’t exactly, at the time, know how I was going to apply it. I wasn’t, at the time, like, “Someday I’m going to lead a team of ethical hackers.” It just didn’t cross my mind at the time. But I was really fascinated in how people think. Why do people do what they do?

But then also, in particular, why do the bad people do what they do? I studied psychopaths and sociopaths and stuff like that. I’m like, “Why can someone kill 50 people?” I don’t even want to step on a bug or something. How are we the same human beings?

I think understanding the way other people think—and understanding that it’s different from the way we think, including maybe things that we are morally opposed to—is critically important for anyone in any part of the security field, but especially the hacking field.

Manoj Tandon
So walk us through a little bit about the mindset of the hacker. Can you help categorize some of them? Because I know some of them are not sociopaths or psychopaths; they’re extremely intelligent people like you and me trying to—in their mind—do the right thing. They might be working for a state-sponsored agency of some kind, or what have you.

Ted Harrington
I’d very much agree with that. But first, before we talk about the bad types of hackers, we should recognize that “hacker” is, in fact, a neutral term. A hacker’s not good or bad; a hacker is a problem solver. They’re creative, they just look at something and say, “Hey, can it behave differently?” They’re contrarian and they’re committed.

That’s what a hacker is. The difference is motivation. Those motivated to find vulnerabilities to fix systems—those are ethical hackers. That’s the world I come from.

Those motivated by something else are generally the attackers, and those fall into a bunch of different categories based on how their group aligns their motivation. Some of the common ones are… everyone typically thinks malicious hackers are motivated by profit, which is true for some of them, like organized criminals, definitely.

But nation-states—they’re organized to gain a political or geopolitical advantage. Hacktivists are motivated to be able to make a statement. Individual or small-group hackers—they’re motivated to prove they can do it. They like the notoriety; they like the challenge.

Corporate espionage—they’re motivated to gain a competitive edge in their given marketplace. So it really depends on the different types of attackers what their motivation would be. It’s incorrect to paint all “hackers” with the same brush.

Manoj Tandon
I’m really glad you said that. So that comes to the next question: of the groups that you just described, is there a difference in effectiveness between them? Does motivation drive effectiveness?

Ted Harrington
I haven’t thought of the question in that way. So let me think—reason it out loud.

Manoj Tandon
Please. Sorry, I didn’t mean to put you on the spot there.

Ted Harrington
I love it, this is why I’m here. Are there differences in effectiveness between those groups? Let me answer that part of the question first: definitely yes.

So, for example, nation-states and organized crime—they are very, very effective. They’re more effective than, say, an individual or small-group hacker. Not to say that small groups are not effective, but the difference between the two generally has to do with resources.

A nation-state has unlimited time and unlimited computational power. In some cases, they might even have access to the backbone of the internet. Whereas an individual is like, “Hey, I also have my day job, and I can only devote so much time to this.”

The effectiveness is driven by resources. I mean, you see the same in Major League Baseball. Big-market teams generally win the World Series because they have more money to pay players than the small-market teams. That’s why the New York Yankees have so many World Series wins.

But your question was: is it motivation that impacts effectiveness? I don’t know if I can definitively say that someone’s motivation is stronger or weaker or leads to better outcomes, but I think that because the motivation is so aligned with the type of organization that has available resources, you could indirectly say that some motivations are more effective.

But it’s not the motivation; it’s the access to resources. I’ll have to think about that, but let’s make that my draft position on this question.

Manoj Tandon
Okay, I think you’re on solid ground there. This is just my personal opinion—that people who are determined to do something are going to find a way to do it. So if that person or individual or set of individuals is on a mission for something, even if they have limited resources, there’s a very good chance that they could probably pull it off.

Ted Harrington
Yeah, I would probably agree with that. So maybe the question then is the degree of motivation, right? Like a small individual hacker—how much do they want to get this notoriety? And if it is life-defining for them, if it has to do with survival for their family, then they probably will do whatever it takes.

Manoj Tandon
Yeah, exactly.

When you look at a lot of the ways that systems are compromised—and there’s a host of them, and I know we don’t want this to turn into a technical discussion on SQL injection, or request smuggling, or various approaches to phishing—there are a thousand things that we could talk about on that alone.

But on the defender side—whether you’re a red team, blue team, in the SOC, a threat hunter, or an application developer or an OEM that’s created something—there’s often a failure of imagination on the creator’s side that doesn’t take into consideration a design point that a system could be used in an alternative manner.

And I know in the world, at least of defense equipment development, that’s actually a design point. There’s a design point put in place for that: “If we have malicious use or inadvertent use, how do we defend against it?” That’s often not taken into consideration on the application development or equipment development side.

And that’s a mindset. Do you have any suggestions on how we change that, or how we get people to start thinking about those things at the inception of a system or a product?

Ted Harrington
Well, I think the best way to successfully do that is to partner and pair those mindsets. Because they are very different mindsets.

If we think about—I’m big on metaphors—if we think about the way that a skyscraper is built: there’s the person who’s an absolute expert—I don’t know who this person is, but somewhere out there, there’s an absolute expert on building skyscrapers.

And if you go to this person whose expertise is building buildings and you say to them, “All right, I now need you to be the demolition expert on a building,” they’d be like, “I guess I kind of know. I could reason my way through that.” And they’d maybe be better than the average person off the street, no doubt. But who’s going to be better at demoing the building than a demo expert?

And so that’s why you sort of pair the two. This is why, as organizations are building things—like building the next application, whether that’s an internal application or something you’re commercializing and selling licenses to businesses or to consumers—you want to work with somebody who has that malicious mindset, who can look at the way something’s designed and say, “Well, what if…?” because they’re really different mindsets.

As much as we might want to invest time and effort to train developers to think like hackers, I think we can get some progress there, but think about this: people whose profession is hacking, they spend 100% of their time doing that.

Manoj Tandon
Yes.

Ted Harrington
It is an impossible ask to turn to someone who’s a developer and say, “I want you to be as good as someone who only spends their time on this.”

I have a new metaphor for this, actually, from just last night. I’m going to work this one in.

Manoj Tandon
Please, right here on Security Confidential.

Ted Harrington
You heard it here first. I saw this documentary last night—or maybe the night before—about the world of solving Rubik’s Cubes.

And yes, there’s a subculture—there’s a world championship for this—and it follows the Tom Brady/Peyton Manning rivalry of these two “best ever” in the history of the sport. They’re these intense competitors but also love each other; it’s actually a really heartwarming documentary.

But one of the storylines they talk about is that people tend to age out of this. Because at a certain point in your life, when you’re a kid or teenager and you don’t have responsibilities, you can dedicate the time necessary to be able to solve a Rubik’s Cube in like seven seconds.

But once you start having a mortgage, a job, and kids, and these other responsibilities, you just don’t have as much available time. And so once that life phase happens for these champions, they wind up not being relevant anymore.

You can see it—they track it; they start falling down the leaderboard. And the reason is they just can’t dedicate the time to hone their skills as well as everyone else.

So if we draw that metaphor to “Hey, you’re a software developer. I want you to be as good as the hackers who work for the nation-state of China, but you still got to be a developer first.” It’s impossible.

So you have to pair the two. It would be like saying, “In my spare time, I’m going to be a Rubik’s Cube champion,” working one hour a day against a kid who does it like 12 to 15 hours a day. It’s just not going to happen.

Manoj Tandon
So, when one starts on a new development effort, how do you set an expectation with the investors regarding designing for security? Because the timeframe here changes.

Ted Harrington
Not necessarily.

Manoj Tandon
Really?

Ted Harrington
I actually would say that the way you frame it is that it’s more effective, less expensive, and generally, when you look at the entire project life, it’s going to be faster to build security in rather than to think about it later. I know that sounds counterintuitive, but I can explain why.

Manoj Tandon
Yeah, please do. Because I can tell you there are a lot of people listening who are going to say, “You don’t understand how…”

Ted Harrington
People right now are either very much listening or they’re like, “Screw this, I’m out.” They’ve already stopped listening to the episode. But let me explain why.

Okay, so I said there were sort of three parts to that: more effective, less expensive, and the overall project lifecycle takes less time. I won’t spend really any time talking about it being more effective; I think we would all agree that the more you think about security and the more it’s baked in, the better it’s going to be.

If anyone disputes that, we can revisit that, but that’s probably not the point people are curious about right now.

The “less expensive” thing is, I think, surprising to a lot of people. The reason the earlier you build security in, the less expensive it is, is for two reasons. The smaller of the two reasons is the expense that you pay for your security partner.

If you’re working with a group of ethical hackers, the fees you have to pay them are a little bit less because when security is built into the process, there are fewer vulnerabilities for them to have to look at because they’ve kind of preempted them.

I mean, “Don’t make that connect to that; make it connect in this way,” and then you won’t have a problem there. That took 13 seconds rather than five hours of testing or something. So you’re able to make fundamental design choices that later on would become very difficult to do.

Manoj Tandon
100%.

Ted Harrington
The second piece of why it’s less expensive—the big piece—is your time and remediation.

The ethical hackers come back to you and they’re like, “Look, here are all the issues; here’s how you fix them.” In addition to your development effort, you now need to spend development time on that, and there’s a cost to that.

That doesn’t show up as well on a profit-and-loss statement because you’re already paying the developers, but the productivity hit is what you feel.

And this third point, that it slows you down or takes longer—now we have to think about the whole project life. This is probably where people will stumble the most.

People want to say, “This is our release date. Our release date is September 1st; we’re going to launch it September 1st.” But they don’t think about, after September 1st, all that remediation effort that they have to do from all the vulnerabilities they injected that were now discovered.

And so really, the product—in its more secure state—maybe it’s the following September. Maybe it takes a full year.

And so what would make an investor interested in this? Of course, the investor needs to care about security, and the product needs to… security needs to matter for the product. If you don’t have those two preconditions, this argument is dead.

But assuming those preconditions, the right investor is going to say, “Wow, okay, the resources I’m investing are going to go farther because the expense goes down, it’s more secure, and we don’t have that problem where we’re putting a vulnerable product in the marketplace and then, some amount of time later, we finally get it in a more secure state that costs us more to do.”

Any intelligent investor is going to understand that logic; it’s just whether or not we can make the case for them.

Manoj Tandon
Well, let me ask you this another way. I’m just thinking out loud here, Ted: would it make sense to embed ethical hackers—or let’s call them maybe not even hackers, but expert systems users—in the QA cycle and let them go to town?

Because the way QA typically is done today is that the QA people are given a sheet and they’re like, “Check this, check this, check this, check this.” Or there’s automated tools that do a variety of things.

What if we started embedding people in there that can just go play and say, “Okay, go ahead, try to break it,” during the QA cycle?

Ted Harrington
Two parts to answer that question. One is: yes, we should embed these people, but they need to have the breaker mindset. The person who’s just like the system user—they’re not necessarily looking at it to break it; they’re going to be looking at it to see how easy or not easy this is to use.

You want to look at someone who’s like, “Hey, this is an input field. What if I didn’t put anything in? Or what if I put a command in there?” That’s the mindset that you need.

But the second part of the question is: we shouldn’t limit this to QA. Every step of the development process has a security aspect, and we should do it at that stage.

For example, when you’re in the requirements gathering stage, you’re now defining what problem this system solves and who it is for. Because you are answering those questions, you’re now going to be able to determine what assets this system will need to provide access to.

And if we can understand the assets, we can understand what types of functionality we’re going to need to design around. Once we understand the assets that this system will protect and sort of that attack surface that we’re going to be designing around, it helps us think about: who’s going to want to attack this system?

That’s your threat model. A threat model is three pieces: what do we want to protect, who do we want to defend against, and where will we be attacked? You’ve got to do that in the design stage, in the requirement stage.

And then, as you’re in design, now you can be thinking about defense in depth and how that threat model plays in, and all that kind of stuff. So by the time you get to QA, you’ve already eliminated a lot of unnecessary attack surface or broken functionality, etc.

Manoj Tandon
Well, you brought up defense in depth, so I want to—I can’t not go down that rabbit hole, although that’s an episode in itself, more than likely.

Ted Harrington
I like this rabbit hole.

Manoj Tandon
So, the problem—in any development effort of any kind, no one is starting from scratch. You’re always using libraries or pre-built assets that are out there, whether that be Java or Ruby on Rails—that’s a platform or language—or you’re using somebody else’s pre-built library functions for whatever it may be.

You don’t know what they’ve done on the security side or not done, and you incorporate them into your product. So you might be inheriting a lot of vulnerabilities that you’re not aware of, and even they’re not aware of.

Ted Harrington
Yep.

Manoj Tandon
So what do we do here?

Ted Harrington
Well, the first thing is that we should assume that we are inheriting vulnerabilities. Our mindset shouldn’t be, “Hey, maybe not, hopefully not.” It should be, “All right, there are vulnerabilities in this thing that we’re integrating. What do we now do about this?”

That’s an important distinction because, when you start from that point, it makes us think differently than if we’re saying, “Hopefully not.” Because if you’re “hopefully not,” then you’re like, “All right, we’re just going to operate as if they’re not there.”

But if you’re operating as if there are vulnerabilities, now you can say, “All right, well, how would we mitigate the likelihood that this particular thing…” which we’re not going to rebuild, that library, like you said. We’re going to use this other component, or maybe it’s even something like working with another company, right?

You’re like, “We don’t want to do payment processing, so we’re going to integrate this payment processing platform.” Okay, well, you think about the way that integration is with that third party and where the delineation point is between where your system is and their system.

That’s where defense in depth really helps, because if we can operate under the assumption that there might be a way we can be compromised through these integrations and through these third parties, then it helps us think differently about how to stop that.

The classic metaphor is, of course, a castle, right? A castle has a moat and a drawbridge, and then there are the dudes on the turrets with the hot oil, and you’ve got the archers, and then you get inside and there’s… I’ve used that one so many times.

And that’s the reason: because if all I had to do was swim across a moat and hope I don’t get eaten by an alligator—once I get to the other side of the moat, that’s it? No, you would never do that. That’s why castles have these layers, and that’s why we should have layers and systems that integrate.

Manoj Tandon
That is defense in depth.

Ted Harrington
That is defense in depth.

Manoj Tandon
That is defense in depth. It’s a military strategy, actually, that we’ve adopted into the cybersecurity world, but that’s where it came from. And that’s exactly right. I mean, it’s a multi-layered defense.

But the execution of that, Ted, is also non-trivial. I mean, you have to consider—you have to make certain assumptions about what that attack vector may look like.

Ted Harrington
I agree, and that’s the benefit of doing a threat modeling exercise early in the process. A threat model helps you determine what level of effort you are going to put into this.

If you get in that early stage—say we’re going to build this app that serves up recipes—you’re like, “You know what, let’s not worry about defense in depth here.”

But when you’re like, “We’re going to create an app that provides the means for how we collaborate on the next major blockbuster film before it comes out and generates two billion dollars in box office revenue,” we’re going to want to invest in different types of security in that type of system.

So that’s where threat modeling helps you think differently about what it is you have to protect and why someone might want to attack to get it.

Manoj Tandon
I’ve got a story and a question, actually. And then we’ll probably run out of time. When you brought up movies—we actually had a customer, who will remain unnamed, but I am sure the audience has definitely seen their productions 100%.

And they have layers upon layers in their organizational structure, multiple divisional CIOs, multiple CISOs—a very large organization. But in the end, when they completed production of the movie and it ended up on the cutting-room floor for editing, the database where that movie was being placed was completely accessible to the entire planet.

So when you look at a two billion dollar or a hundred million dollar production… if you knew where to look—it’s been since rectified, so you can’t find it anymore—but if you knew where to look, you could actually have gotten that movie six months prior to it ever hitting any kind of a theater and had your own private showing.

Ted Harrington
I’m not surprised. That’s why the profession exists: because stories like that happen. It’s real; you can’t make it up.

Manoj Tandon
Yeah, it’s a real thing. But when you brought up vulnerability modeling—stepping back to a basic level, there’s this notion out there, as we walk up the entire pyramid of pain, that we can address things with vulnerability scanning or with penetration testing.

And I think that’s a little bit of a false god. I don’t know what your opinion of that is, because to me, you really can’t automate some of this threat modeling; you need a human element to it. That’s just my personal view. Feel free to say that’s BS.

Ted Harrington
You’re preaching to the choir right now. Automation is awesome, right? It does so many powerful things in really any field, but in security in particular, what it does is it helps us do very manual, time-intensive, routine, repeatable types of processes—we can do them very quickly.

It also helps us check for known vulnerabilities very quickly. So we know there’s this whole database of different types of attacks or vulnerabilities that an attacker might be able to reference; they’re going to run some automated way to check for them, and we should check for that too. So that’s great.

The problem is when organizations want to rely too much on that and they’re like, “Oh, automation’s going to solve everything, the machines know.” And they don’t. Humans need to be able to connect dots.

Here’s a perfect story, a perfect example: there’s a project we were working on recently, and this system had two particular problems. The first one wasn’t that big of a deal—you don’t want it to happen, but it was what’s called information leakage.

Basically, it meant that the system was giving up information it shouldn’t. In this case, it was giving up the user identifier. So any user could identify any other user. You can’t even directly exploit it, but you don’t really want it to happen.

The second problem was where the authorization model was broken. That means that the way you actually changed credentials—like if you want to change your password—it didn’t work correctly.

Manoj Tandon
Awesome. This is awesome. I can make use of this. Go in there and change everybody’s password is what I’d be doing, but that’s separate.

Ted Harrington
Well, that is actually where the story goes. Because the way changing passwords worked was you had to supply information. For anyone who’s ever changed a password…

Manoj Tandon
So you only needed an ID? That’s it?

Ted Harrington
Well, the broken authorization said if you have your ID, you can change the password. And so, with the information leakage, you combine the two. And so now you can do it for everyone in the whole system.

Yeah, so now we can change anybody’s password, including the admins. And so it basically meant that all someone needed to do to completely take over the whole system was have an account. You have an account? Done.

And that’s the kind of thing where there’s no tool that will do that. There’s no automated script that would say, “Look for this, look for this, connect.” No, you need that sort of creative problem-solving that we’ve talked about. You need that contrarian thinking, the committed aspect that is hackers, and the creativity.

And so, yes, just running scans alone doesn’t get it done. You need that higher-level manual effort.

Manoj Tandon
There are a lot of things being sold out there in the name of a vulnerability assessment which are really more like penetration tests, or maybe a vulnerability scan. There’s a lot of stuff like that going on out there, at least that we’ve seen.

And I don’t think anyone is ill-intentioned. I think some of those terms are just not well understood by the people who are buying what they really mean.

Can you do a quick one-minute differentiation between a pen test versus a vulnerability scan versus a real vulnerability assessment which requires human interaction?

Ted Harrington
As with all things, I will use a metaphor to describe.

Manoj Tandon
Please. I love the stories. Everyone listens in their car, so they’re probably going to enjoy this one.

Ted Harrington
Yes. Oh, and it’s a car metaphor, so perfect.

The first thing we have to realize is what’s happening right now. What’s happening is that people are asking for penetration tests. They’re being sold vulnerability scans instead, as if it’s a penetration test. But what they really need is something else entirely: they actually need vulnerability assessments.

Those are three different things. They require different investments of time, effort, and money, and they deliver different outcomes.

And the reason that this matters is that, if an enterprise is saying to their vendor, “Give me a pen test,” the vendor’s like, “All right, I guess I’ve got a pen test.” So they Google “pen test” and they don’t get the thing that the enterprise is looking for. That’s bad for the enterprise; it’s bad for the vendor; it’s bad for everybody. So that’s why it’s a problem.

The metaphor is this: what penetration testing actually is, is kind of like when a carmaker wants to understand—maybe people aren’t going to like this if they’re driving their cars right now, because it has to do with crashing cars…

Manoj Tandon
Sorry, but we’re in it now. Slow down, this thing’s going to crash!

Ted Harrington
So, what are we crashing? Hands at 10 and 2. Let’s pay attention to the road, people!

Okay, so when the carmaker wants to know how the vehicle will perform in a particular scenario, such as a head-on collision, what do they do? They literally crash the car into the wall to see what will happen.

That’s kind of what a pen test is like. An actual penetration test is when you take a built system—it’s already been through a lot of testing, like the car is built, and it’s already been tested in all these different structural ways—and you know what you’re testing for, right? You know exactly what you’re testing for. You’re looking to see if a particular system or set of systems is going to perform in a certain way.

Manoj Tandon
Exactly.

Ted Harrington
And that’s actually what a pen test does: can someone escalate privileges in order to do X? So it’s narrowly scoped; it’s not just open-ended. That’s what a pen test actually is.

But when most people buy it, what most companies are selling is not that. Most people are actually selling scans. If you were to Google “penetration test” right now, probably 98% of the results you’re going to get from that Google search are going to be scans.

And what a scan does… a scan is more like when the check engine light comes on in your car. You stick that little thing under the dash, it queries and interrogates the computer, and the computer says, “Here’s why it turned on.” It’s only looking for known issues. It’s simple, it’s quick, but it’s certainly pretty different from “How will this car perform in a crash scenario?”

And that’s problematic, right? When people are like, “I’m going to crash the car,” but they just get the “Replace the oil” alert.

But the third thing is actually what most people want, and that’s where the automotive safety department comes in. What do they do? They look at how all the systems work together. They’re like, “All right, how do the side-impact beams work with the airbags, and work with the lane-departure technology?” and all that kind of stuff.

Because we want to think about this as a holistic system, and we want to know the unique ways that maybe our car might perform differently than, say, another manufacturer’s car. We care about this particular car and how—the unique ways—this car might not perform.

That’s what people really want. When an enterprise is saying to their vendor or supplier, “I need a pen test,” that’s what they’re asking for. They’re like, “Make it better.” They’re not saying, “Do the minimum.” And they’re not saying, “Look at a very narrowly scoped, specific scenario.”

They’re saying, “Make it better; think about it holistically; prove to me that you’ve thought about it in all these different ways.” So they’re looking for unique outcomes, and that’s going to require a vulnerability assessment, which requires human involvement. It can’t be done automated, which is why the people who really do them right do cost a little bit of money.

They cost a lot more—I mean, pen tests cost a lot more too. Scanning is like… you can get a scan, frankly, you can get a scan for free. You can get a license for Nessus for free.

But most scans… you kind of can identify if you’re getting a scan-based “penetration test”—and I’m putting that in air quotes—if it’s like $5,000, or $10,000, or $15,000. That level of effort is a scan.

It gets to five, ten, maybe even 20 times that price when you’re talking about proper security testing that has that manual emphasis. And that scares people, right? When they’re like, “Well, this one’s 15 and it says pen test; this one says pen test and it’s 80. What’s the difference?”

Manoj Tandon
Yeah, and then you go to your board to get the money and they’re like, “Well, this costs eight and this costs a hundred. What’s the difference?” And it becomes—we’re back to procurement again.

Ted Harrington
Full circle.

Manoj Tandon
The circle of life.

Ted Harrington
The circle of life.

Manoj Tandon
Well, we’re at the end here, but we wanted to give you a few minutes to talk about whatever it is that you want to put out there. You always have stuff going on, so what would you like our audience to know, or check out?

Ted Harrington
I do always have stuff going on. We already talked about everything I want to talk about; I want to nerd out on these security problems! But I guess I would just leave the audience with this: if anything that we talked about today is stimulating your thinking about a problem that you’re facing or questions that you have, use me as a resource.

If you want to talk about booking me to keynote an event you have, or you want to talk about hiring our company to help you with security testing, or you just want to bounce an idea off somebody—just go to tedharrington.com and everything’s there.

You can find how to follow me on socials, how to contact me, whatever you need. And I’m super, super responsive, especially on LinkedIn. Less so on Twitter, but on LinkedIn, I respond most of the time. So, I just want to be a resource, so let me be that.

Manoj Tandon
That’s fantastic. Is there any way to get a signed copy of your book?

Ted Harrington
I’ve got you, absolutely. I’ll send it to you.

Manoj Tandon
Oh, I appreciate that. And we might want to reach out to you and maybe get several of them and give them away to people who ask really good questions.

Ted Harrington
Absolutely, we’d love to do that. 100%, consider that a “yes.” Done.

Manoj Tandon
That would be awesome. Because that’s, again, our mission here: education, and anything that we can do to help people become better at cyber. Hey, and maybe just better people. We’ve got bigger dreams sometimes.

Ted Harrington
I love it, I love it.

Manoj Tandon
Thank you so much, Ted. It’s been a real pleasure. I’ve enjoyed it tremendously and let’s stay in touch.