Category Archives: Evaluation Criteria

CISA Gives Us A Starting Point

Posted on by

Everyone struggles with identifying the “right” vulnerabilities to patch. It’s a near universal problem. We are all wondering, “Which vulnerabilities in my long queue are going to get me?”

Most organizations calculate a CVSS (base score or amended) which a solid body of research starting with Alloddi & Massacci, 2014, demonstrates is inadequate to the task.

Exploit Prediction Scoring System (EPSS) is based in the above research, so it could provide a solution. But the web calculator cannot score each open vulnerability in our queue over time: we need to watch the deltas, not a point in time. There’s unfortunately, no working EPSS solution, despite the promise.

CISA have listed 300 “vulnerabilities -currently- being exploited in the wild” in Binding operational directive 22-01.

Finally, CISA have given us a list that isn’t confined to a “Top 10”: Start with these!

Top 10 lists provide some guidance, but what if attackers exploit your #13?

300 is an addressable number in my experience. Besides, you probably don’t have all of them in your apps. > 300. We all can do that much.

The CISA list provides a baseline, a “fix these, now” starting point of actively exploited issues that cuts through the morass of CVSS, EPSS, your security engineer’s discomfort, total vulnerabilities ==> organization “risk”. (If you’re running a programme based upon the total number of open vulnerabilities, you’re being misled.)

https://thehackernews.com/2021/12/why-everyone-needs-to-take-latest-cisa.html

Finally Making JGERR Available

Posted on by

Originally conceived when I was at Cisco, Just Good Enough Risk Rating (JGERR) is a lightweight risk rating approach that attempts to solve some of the problems articulated by Jack Jones’ Factor Analysis Of Information Risk (FAIR). FAIR is a “real” methodology; JGERR might be said to be FAIR’s “poor cousin”.

FAIR, while relatively straightforward, requires some study. Vinay Bansal and I needed something that could be taught in a short time and applied to the sorts of risk assessment moments that regularly occur when assessing a system to uncover the risk posture and to produced a threat model.

Our security architects at Cisco were (probably still are?) very busy people who have to make a series of fast risk ratings during each assessment. A busy architect might have to assess more than one system in a day. That meant that whatever approach we developed had to be fast and easily understandable.

Vinay and I were building on Catherine Nelson and Rakesh Bharania’s Rapid Risk spreadsheet. But it had arithmetic problems as well as did not have a clear separation of risk impact from those terms that will substitute for probability in a risk rating calculation. We had collected hundreds of Rapid Risk scores and we were dissatisfied with the collected results.

Vinay and I developed a new spreadsheet and a new scoring method which actively followed FAIR’s example by separating out terms that need to be thought of (and calculated) separately. Just Good Enough Risk Rating (JGERR) was born. This was about 2008, if I recall correctly?

In 2010, when I was on the steering committee for the SANS What Works in Security Architecture Summits (they are no longer offering these), one of Alan Paller’s ideas was to write a series of short works explaining successful security treatments for common problems. The concept was to model these on the short diagnostic and treatment booklets used by medical doctors to inform each other of standard approaches and techniques.

Michele Guel, Vinay, and myself wrote a few of these as the first offerings. The works were to be peer-reviewed by qualified security architects, which all of our early attempts were. The first “Smart Guide” was published to coincide with a Summit held in September of 2011. However, SANS Institute has apparently cancelled not only the Summit series, but also the Smart Guide idea. None of the guides seem to have been posted to the SANS online library.

Over the years, I’ve presented JGERR at various conferences and it is the basis for Chapter 4 of Securing Systems. Cisco has by now, collected hundreds of JGERR scores. I spoke to a Director who oversaw that programme a year or so ago, and she said that JGERR is still in use. I know that several companies have considered  and/or adapted JGERR for their use.

Still, the JGERR Smart Guide was never published. I’ve been asked for a copy many times over the years. So, I’m making JGERR available from here at brookschoenfield.com should anyone continue to have interest.

cheers,

/brook schoenfield

Seriously? Product Security?

Posted on by

Seriously? You responded to my security due diligence question with that?

Hopefully, there’s a lesson in this tale of woe about what not to do when asked about product security?

This incident has been sticking in my craw for about a year. I think it’s time to get it off my chest. If for no other reason, I want to stop thinking about this terrible customer experience. And yes, for once, I’m going to name the guilty company. I wasn’t under NDA in this situation, as far as I know?

There I was, Enterprise Security Architect for a mid-size company (who shall not be named. No gossip, ever, from this blog). Part of my job was to ensure that vendors’ product security was strong enough to protect my company’s security posture. There’s a due diligence responsibility assigned to most infosec people. In order to fulfill this responsibility, it has become a typical practice to research software vendors’ product security practices.Based upon the results, either mitigate uncovered risks to policy and industry standards or raise the risk to organizational decision makers (and there are always risks, right?).

Every software vendor goes through these due diligence investigations on a regular basis. And I do mean “every”.

I’ve lived on both sides of this fence, conducting the investigations and having my company’s software go through many investigations. This process is now a part of the fabric of doing secure business. There should be nothing surprising about the questions. In past positions, we had a vendor questionnaire, a risk scale based upon the expected responses, and standards against which to measure the vendor. These tools help to build a repeatable process. One of these processes is documented in a SANS Institute Smart Guide released in 2011 and was published by Cisco, as well.

Now, I’m going to name names. Sorry, Google, I’m going to detail just what your Docs sales team said to me. Shame on you!

When I asked about Google Docs product security here is the answer, almost verbatim, that I received from the sales team:

“We’re Google. We can hire Vint Cerf if we want. That is enough.”

Need I point out to my brilliant readers that Dr. Vint Cerf, as far as I know, has never claimed to be an information security expert? I’m sure he knows far more about the design of TCP/IP than I? (but I remind readers that I used to write TCP/IP stacks, so I’m not entirely clueless, either). And, Dr. Cerf probably knows a thing or two about Internet Security, since he runs ICANN?

Still, I can tell you authoritatively that TCP/IP security and Domain Name Registry security are only two (fairly small) areas of an information security due diligence process that is now standard for software vendors to pass.

Besides, the sales team didn’t answer my questions. This was a classic “Appeal to Authority“. And, unfortunately, they didn’t even bother to appeal to a security authority. Sorry Vint; they took your name in vain. I suppose this sort of thing happens to someone of your fame all the time?

Behind the scenes, my counter-part application architect and I immediately killed any possible engagement with Google Docs. Google Sales Team, you lost the sale through that single response. The discussion was over; the possibility of a sale was out, door firmly closed.

One of the interesting results from the wide adoption of The Web has been the need for open and transparent engagement. Organizations that engage honestly gain trust through their integrity, even in the face of organizational mis-steps and faux pas. Organizations who attempt the old fashion paradigm, “control all communications”, lose trust, and lose it rapidly and profoundly. Commercial companies, are you paying attention? This is what democracy looks like (at least in part. But that’s a different post, I think?).

Product security is difficult and demanding. There are many facets that must compliment each other to deliver acceptable risk. And, even with the best intentions and execution, one will still have unexpected vulnerabilities crop up from time to time. We only have to look at development of Microsoft’s product security programme to understand how one of the best in the industry (in my humble opinion) will not catch everything. Do Microsoft bugs surface? Yes. Is the vulnerability level today anywhere near what it was 10 years ago? Not even close. Kudos, Microsoft.

It’s long past the time that any company can coast on reputation. I believe that Google do some very interesting things towards the security of their customers. But their  sales team need to learn a few lessons in humility and transparency. Brand offers very little demonstrable protection. Google, you have to answer those due diligence questionnaires honestly and transparently. Otherwise the Infosec person on the other side has nothing against which to base her/his risk rating. And, in the face of no information, the safest bet is to rate “high risk”. Default deny rule.

It’s a big world out there and if your undiscovered vulnerabilities don’t get’cha now, they will eventually. Beware; be patient; be humble; remain inquisitive; work slowly and carefully. You can quote me on that.

cheers,

/brook

Alan Paller’s Security Architecture Call To Action

Posted on by

I was attending the 2nd SANS What Works in Security Architecture Summit, listening to Alan Paller, Director of Research at the SANS Institute, and Michele Guel, National Cybersecurity Award winner. Alan has given us all a call to action, namely, to build a collection of architectural solutions for security architecture practitioners. The body of work would be akin to Up To Date for medical clinicians.

I believe that creating these guides is one of the next critical steps for security architects as a profession. Why?

It’s my contention that most experienced practioners are carrying a bundle of solution patterns in their heads. Like a doctor, we can eyeball a system, view the right type of system diagram, get a few basic pieces of information, and then begin very quickly to assess which of those patterns may be applicable.

In assessing a system for security, of course, it is often like peeling an onion. More and more issues get uncovered as the details are uncovered. Still, I’ve watched experienced architects very quickly zero in to the most relevant questions. They’ve seen so many systems that they intuitively understand where to dig deeper in order to assess attack vectors and what is not important.

One of the reasons this can happen so quickly is almost entirely local to each practitioner’s environment. Architects must know their environments intimately. That local knowledge will eliminate irrelevant threats and attack patterns. Plus, the architect also knows  which  security controls are pre-existing in supporting systems. These controls have already been vetted.

Along with the local knowledge, the experienced practioner will have a sense of the organization’s risk posture and security goals. These will highlight not only threats and classes of vulnerabilities, but also the value of assets under consideration.

Less obvious, a good security architecture practitioner brings a set of practical solutions that are tested, tried, and true that can be applied to the  analysis.

As a discipline, I don’t think we’ve been particularly good at building a collective knowledge base. There is no extant body of solutions to which a practioner can turn. It’s strictly the “school of hard knocks”; one learns by doing, by making mistakes, by missing important pieces, by internalizing the local solution set, and hopefully, through peer review, if available.

But what happens when a lone practioner is confronted with a new situation or technology area with which he or she is unfamiliar? Without strong peer review and mentorship, to where does the architect turn?

Today, there is nothing at the right level.

One can research protocols, vendors, technical details. Once I’ve done my research, I usually have to infer the architectural patterns from these sources.

And, have you tried getting a vendor to convey a product’s architecture? Reviewing a vendor’s architecture is often frought with difficulties: salespeople usually don’t know, and sales engineers just want to make the product work, typically as a cookie cutter recipe. How many times have I heard, “but no other customer requires this!”

There just aren’t many documented architectural patterns available. And, of the little that is available, most has not been vetted for quality and proof.

Alan Paller is calling on us to set down our solutions. SANS’ new Smart Guide series was created for this purpose. These are intended to be short, concise security architecture solutions that can be understood quickly. These won’t be white papers nor the results of research. The solutions must demonstrate a proven track record.

There are two tasks that must be accomplished for the Smart Guide series to be successful.

  • Authors need to write down the architectures of their successes
  • The series will need reviewers to vet guides so that we can rely on the solutions

I think that the Smart Guides are one of the key steps that will help us all mature security architecture practice. From the current state of Art (often based upon personality) we will force ourselves to:

  • Be succinct
  • Be understandable to a broad range of readers and experience
  • Be methodical
  • Set down all the steps required to implement – especially considering local assumptions that may not be true for others
  • Eliminate excess

And, by vetting these guides for worthwhile content, we should begin to deliver more consistency in the practice. I also think that we may discover that there’s a good deal of consensus in what we do every day**. Smart Guides have to juried by a body of peers in order for us to trust the content.

It’s a Call to Action. Let’s build a respository of best practice. Are you willing to answer the call?

cheers

/brook

** Simply getting together with a fair sampling of people who do what I do makes me believe that there is a fair amount of consensus. But that’s entirely my anecdotal experience in the absence of any better data