DevOps Chats: Open Source Security, With WhiteSource

Transcript

Alan Shimel: Hey, everyone, it’s Alan Shimel, DevOps.com, Security Boulevard. You’re listening to another DevOps Chats. Joining us in this DevOps Chats is Rhys Arkins, and Rhys is a director of Product at WhiteSource. Rhys, is that correct?

Rhys Arkins: That’s right, Alan. Hi.

Shimel: Welcome, thank you, and yes—good, thank you. So, Rhys, our audience is pretty familiar with WhiteSource. We work quite a bit with them, and of course, they have kind of become one of the bigger plays in what’s become known as the SCA or software composition analysis market and, you know, that’s the fancy name, but it’s a lot about open source securing open source components. Because sometimes when you say open source security, people think we’re talking about security tools which are open source. [Cross talk]

Arkins: Yeah, that’s true, yeah.

Shimel: Right? You know what I’m saying?

Arkins: Exactly.

Shimel: The [Cross talk]. So, Rhys, I don’t know, maybe there are people who aren’t, though—do you wanna give a quick background on WhiteSource for maybe people who aren’t familiar?

Arkins: Yeah, sure. So, look, in general, the keywords are open source and security, like you said. But, you know, that casts a pretty wide net. As you already mentioned, the technical term, if you’re into the kind of Forrester or Gartner approach is software composition analysis. You know, our kind of tagline on our website, I actually really like it, and it says, “Use open source freely and fearlessly.” And that’s the way we try to distill it down because, you know, ultimately, that’s our goal.

Obviously, there’s massive benefits in using open source. There are some, I wouldn’t say downsides, but you know, it’s not a completely free lunch. And so, what we seek to do is to help our customers to use it freely and fearlessly, meaning that they don’t have to hold back and they can realize all the benefits of open source and avoid some of the security pitfalls that are potentially there.

Shimel: Absolutely, absolutely. So, Rhys, WhiteSource has recently published their annual report, the State of Open Source Security and Vulnerabilities. It’s available for the download on the WhiteSource site yet, or will be soon?

Arkins: Yeah, that’s right. Yeah. So, definitely, by the time anybody’s listening now, it is available. It’s a 12-page .pdf, and yeah, it gives a summary of both the state of affairs as well as, we try to look into what has kinda changed, what are the trends.

Shimel: Okay. So, I guess you opened it with that. Talk to us—what are the trends here we should be looking at?

Arkins: Okay. Well, you know, the first, the headline number is really the total and the growth. It’s probably no surprise that 2019 saw a record number of reported vulnerabilities in open source. Ballpark, it’s around a 50 percent increase, just short of it. So, we’re seeing—

Shimel: Fifty? Five zero?

Arkins: Five zero, yeah, compared to 2018.

Shimel: So, let’s dig in on that. Is it because we’re using more open source components so the pie is bigger, if you will, and so when we look at a bigger pie, of course, there’s gonna be, the slices got bigger in the bigger pie, or is it—you know, not the opposite, but another situation where we’re just seeing more vulnerabilities per 1,000 components or whatever? So, while the pie may have grown a little bit, the slices have actually gotten much bigger or heavier?

Arkins: Yeah.

Shimel: You understand what I’m saying?

Arkins: Yeah, there’s certainly an aspect of both. What we think it is is one of the things you said and one that you didn’t mention. So, it is definitely the case, you know, the pie is getting bigger. There is more open source being produced than ever before. It’s not as much about how much pie we’re eating but how much pie we’re making, right? So, there is more being—more open source being put out than ever. That means not just more packages, but packages are being released frequently as well. There’s a lot of code added to it.

So, No. 1, we’ve got the pie growing. No. 2 is not as obvious, and that is that there are a lot more eyeballs on open source security. So, it’s more a matter that we are finding more problems. Definitely more a case of that than we are creating more per thousand packages or something. It’s really that we are getting better at detecting vulnerabilities. So, in that sense, that’s a positive spin on the fact that the number of vulnerabilities is going up.

Shimel: Very cool, very cool. You know, it’s a positive spin, [Laughter] but nevertheless, vulnerabilities have jumped up 50 percent and that, you know, it’s hard to—

Arkins: Yeah, and you know, there’s like a metaphor that I like using about open source vulnerabilities because one of the kinda misconceptions about open source vulnerabilities is that they’re all kinda like new, you know? And kinda like we both just mentioned then, like—oh, people are making more, or we’re finding more.

So, it’s—the key point is that we’re finding them not necessarily when they’re being made. So, quite a lot of the ones that were found in 2019 weren’t necessarily created in 2019, it’s just that those eyeballs and improved methods that we have in the sort of security research industry that we are better at finding them now.

And the metaphor I use is that, you know, to an open source user, when you find out that there is a vulnerability in one of your third party components, you know, it might feel like a flash bang grenade or something, but it is a lot closer to an unexploded World War II bomb. I mean, these problems in the code can sometimes be sitting for many, many years, and then one day they are discovered—not because they’re new, but because they were discovered. And similarly, the first thing people need to do is to work out just how dangerous they are.

Shimel: Absolutely. You know, I’m of a mixed mindset on these, Rhys. You know, I’ve been in the vulnerability space since 2002. One of the companies I founded, StillSecure, came out with Vulnerability Manager back then.

Arkins: Yeah.

Shimel: And I’ve seen, you know, we’ve seen the pattern, open source does present its own unique sort of circumstances, but quite frankly, that was always one of the supposed pluses of open source is that there were more eyes on the code, right?

Arkins: Yeah.

Shimel: And that’s why, inherently, it would be more secure. And of course, as open source use has blossomed, we found that, you know, inherently, it’s not more secure, it’s not inherently less secure, either, though.

Arkins: Yeah.

Shimel: You know? All points—

Arkins: I like that question, because I have an opinion on that one. [Laughter]

Shimel: Let’s hear it.

Arkins: So, you know, the first part is, you know, this concept of eyeballs, and I’ve said it myself, already. In a way, it can be like a misunderstanding. You know, the original statement was, you know, with enough eyeballs, all bugs are shallow. And that kinda got translated into this idea that open source is inherently secure, because there’s many eyeballs looking at it.

The reality is, there are many, many lines of open source code. The total count of eyeballs that have looked at it is, like, two. [Laughter] So, that is a misunderstanding. But the idea that open source is inherently more secure, it’s close. There—one of the Product Directors at GitHub recently said that open source is not more inherently secure, but it is inherently more securable. And I think that’s a really great way of putting it, because—

Shimel: I like that.

Arkins: Yeah. So, when it’s all out there, and as we improve the tools, it is inherently more securable. It’s not that automatically. We’ve got to work to get there.

Shimel: Huh. Interesting. Alright, let’s look at—I think the other thing people have to also keep in mind when reading these reports and looking at open source components is, this isn’t some other company that’s using open source, or it doesn’t only apply to small companies that are using open source, right? Ninety-nine percent—and if I’m wrong about that number, tell me, but you know, a high 90 percent of all organizations use open source.

Arkins: Right, right.

Shimel: And that includes the Fortune 500.

Arkins: That’s the approximate figure, yeah. I mean, you can say the number of projects that use it is lower, but in terms of companies, it’s around 99 percent, because pretty much, if you have a website, you’re probably using some open source components in it.

Shimel: Yep. So, you know, we look—I mean, the only reason to look with what came before is to help us look ahead, right?

Arkins: Right.

Shimel: And though things are a little cloudy right now with all this Corona and COVID stuff, but you know, when we look at 2020, Rhys, what can we learn from this report of 2019, and what can we take with us going into the new year?

Arkins: Yeah. So, we’re definitely seeing—the really big change is a real step up in the research and reporting of vulnerabilities. It’s been a real shift, and I think that the data from 2018 to 2019, which we’re looking at now, and then from 2019 to 2020, I think we’re gonna see some really interesting changes just because, like, we’re looking a lot more. And in particular, in the last 12 months, we’ve seen a real realization of automated security scanning.

So, you know, there’s a few companies in this space, but for example, GitHub have one a great job with their Security Lab and what they call CodeQL. And that lets security researchers, you know, look for a single type of mistake across thousands and thousands of projects. So, we’re gonna see a real ramp up of that to some extent that could potentially be the low hanging fruit, but it’s still good that these are getting caught and getting picked up.

The other thing that we’re seeing a trend in, and this also actually is another contributor to why the numbers are going up, is that in the past, in open source, we haven’t been the best as a community at correctly documenting and responsibly disclosing vulnerabilities when they’re created, alright? So, this concept that you should inform the authors, owners, or maintainers of code, you know, and give them 30 to 90 days to fix, have the patches already available and then release them—that hasn’t been how open source has rolled a lot. That’s partly because the people that might find the aren’t familiar with that, but this is getting much better and I think that’ll be one of the big changes you see in 2020 will start to make a bit more sense of the chaos and see improved kinda like formal reporting rights compared to previous years.

Shimel: Absolutely. Rhys, I’m going through the report a bit as we’re talking here. One of the things, one of my pet peeves with open source use is, as the report indicates, we’ve gotten better at finding open source vulnerabilities. We’ve gotten better at releasing new versions of open source components that patch or fix vulnerabilities.

What we haven’t gotten better at is stopping people from still using the old versions, and even continuing to download these old versions of open source components.

Arkins: Yeah.

Shimel: Make sense?

Arkins: Yeah, look, this is a great question. It’s not covered in our report, but it’s absolutely related. You know, so, there is this pretty great company called Cloudflare. I was gonna say they’re a CDN company, but they do many things. They actually put out a blog post in January of this year that said people are not upgrading outdated packages. And the reason they know that is because they are the hosts of one of the most popular web CDNs, CDNJS.

So, a lot of people, when they use a library like React or Lodash or jQuery in their website, they reference to CDNJS. And so, people load it off the CDN. But based on that, they have the ability to monitor what people use. And they noticed that, you know, once a version like 2.1 has reached a certain level of use, that level of use only gradually tapers off over years, even when, you know, 2.3.0 or 3.0.0 or 4.0.0 are released. And the conclusion from that is true, that people are not updating once newer versions available. You know, if they create a new website, they use the latest, but they don’t go and update their old ones.

So, at WhiteSource, we actually released a tool that detects CDNJS references in HTML pages. I mean, it detects a lot of dependencies, everywhere, but we added support for CDNJS as part of an effort to try to help people automate that process. Because,  you know, if you’re on outdated dependencies, it’s pretty much guaranteed that you’re gonna get hit by vulnerabilities at some point.

Shimel: Agreed, agreed. So, here’s another one. While we’re at this, let me just throw the rest of my pet peeves around.

Arkins: Yeah, go ahead. [Laughter]

Shimel: With vulnerabilities and open source—let’s discuss ‘em. So, the problem—and this was a problem in 2002, in 2005, ’10, ’15, and ’20, as well. The problem is, is that, quite frankly, we get desensitized and have a hard time prioritizing, especially as we’ve moved security left, right, into DevSecOps and shifting to the left.

Arkins: Yeah, yeah.

Shimel: So, oftentimes, it may not even be a security person, it’s a developer or a DevOps team member who gets this long list of vulnerabilities, you know, and they may have some CVE priority critical, you know, not high critical, whatever.

Arkins: Yeah.

Shimel: But it may not even be something that is germane or applicable to this particular application or whatever.

Arkins: Yeah.

Shimel: And so, helping not only the security folks now, but we gotta help the DevOps and the developer people prioritize how to get the most bang for their buck, how to make these applications as secure as possible without totally impacting time to release and so forth. Because what we’ve learned is, if we give them too many vulnerabilities to fix that it jeopardizes releases and stuff, vulnerabilities just don’t get fixed.

Arkins: Yeah, yeah. No, that’s another great question. We’ve seen that a lot. As they increase, you know, the better we do our jobs at WhiteSource, the more alerts people are getting and the potential for them to be desensitized.

Now, the secret to overcoming that—I shouldn’t say secret—the way to overcome that is by prioritization and triaging. And there’s multiple ways that you can do it and, you know, ideally, the tools you use automate that as much as possible.

But like, for example, a lot of companies have internal tools, and if you’ve got even a high or critical severity on an internal-facing application that requires a malicious user, that’s a really big difference compared to if that’s on a public-facing application. So, first of all, just bouncing between internal versus external facing, that’s almost like a static rule you can use. The next phase is differentiating between dependencies that are used in production versus dependencies that are used in development or as part of the build and release toolchain.

It’s very rare—because remembering, when we talk about vulnerabilities, it’s generally talking about a mistake. You know this is not a malicious act. We classify vulnerabilities and malicious as different, and we may even break those out in future years.

So, when you’re talking about vulnerabilities, which is an oversight or a mistake, the chance that a mistake in the development or the testing dependencies you have would actually translate into an exploitable problem in production is pretty low. So, that’s a second way of kind of categorizing and triaging.

And then the final way—and this is one of WhiteSource’s secret sauces is, we literally have a product called, or a feature called Prioritize. And what that does is, when a vulnerability is there, it analyzes your use of that dependency and essentially parsers, like an AST parser, and it determines whether it is possible to reach that vulnerable code or not. And what we’ve found is that actually reduces down the number of what we call effective vulnerabilities. It reduces that down by 80 percent.

So, you know, even if you didn’t have those first two steps of differentiating between public versus private and Dev versus non-Dev, you can, on average, reduce by about 80 percent, you know, what otherwise, could otherwise overwhelm developers. You know, it’s a challenge. Even 20 percent can be a lot, but the worst thing you can do is get your developers kind of desensitized to that. You wanna make sure that when you ring the fire alarm, that it’s for a good reason.

Shimel: Absolutely, absolutely. Rhys, as I’d mentioned to you before we got started, we start talking about this stuff, and boom, we’re out of time that quickly and we are, [Laughter] we’re way over time, actually.

Arkins: No problem.

Shimel: But it’s all good. I thought we had a great conversation. Sounds like a great report. Please folks, if you’re interested and this is something of interest to you, go over to the WhiteSource site and download the report for yourself. It’s a pretty, even though it’s 11 or 12 pages, it’s not heavy in text, lots of graphics and pie charts.

Arkins: Yeah, it’s very readable. We put a lot of work into it to try to distill it down to the important points.

Shimel: Absolutely—highly, highly recommend downloading this and adding it to your library. Hey, Rhys, thanks for joining us. Keep up the great work over there at WhiteSource.

Arkins: Thanks, Alan. Thanks for having me today.

Shimel: Alrighty. Rhys Arkins, Director of Product at WhiteSource. This is Alan Shimel for DevOps.com and Security Boulevard. You’ve just listened to another DevOps Chats.

— Alan Shimel

Recent Articles By Author

• From CEO Alan Shimel: Futurum Group Acquires Techstrong Group
• KubeCon 2023: Not Your Father’s Tenable
• KubeCon 2023: Bridging the AppSec Tools Gap

More from Alan Shimel