“People and collaboration over processes, methodologies, and tools.”

–The Threat Modeling Manifesto, Value #2

During our panel, my dear friend and longtime collaborator, Izar Tarandach, mentioned something that occurs so often when threat modelling as to be usual and customary. This happens so often, it is to be expected rather than being merely typical. We were speaking on Threat modelling for Agile software development practices at OWASP BeNeLux Days 2020. Izar was addressing the need for broad participation in a threat model analysis. As an example, he described the situation when an architect is drawing and explaining a system’s structure (architecture) at a whiteboard, and another participant at the analysis, perhaps even a relatively junior member of the team says, “but that’s not how we had to implement that part. It doesn’t work that way”.

I cannot count the number of times I’ve observed a disparity between what designers believe about system behaviour and how it actually runs. This happens nearly every time! I long since worked the necessity for uncovering discrepancies into my threat modelling methods. That this problem persists might come as a surprise, since Agile methods are meant to address this sort of disconnect directly? Nevertheless, these disconnect continue to bedevil us.

To be honest, I haven’t kept count of the number of projects that I’ve reviewed where an implementer as had to correct a designer. I have no scientific survey to offer. However, when I mention this in my talks and classes, when I speak with security architects (which I’m privileged to do regularly), to a person, we all have experienced this repeatedly, we expect it to occur. 

In my understanding of Agile practice, and particularly SCRUM, with which I’m most familiar and practice, design is a team effort. Everyone involved should understand what is to be implemented, even if only a sub-team, or a single person actually codes and tests. What’s going on here?

I have not studied the cause and effect, so I’m merely guessing. I encourage others to comment and critique, please.  Still, as projects get bigger with greater complexity, there are two behaviours that emerge from the increased complexity that might be responsible for disconnection between designers and implementers:

1. In larger projects, there will be a need to offer structure to and coordinate between multiple teams. I believe that this is what Agile Epics address. Some amount of “design”, if you will (for want of a better word) takes place outside and usually before SCRUM teams start their cycles of sprints. This structuring is intended to foster pieces of the system to be delivered by individual, empowered Agile teams while still coming together properly once each team delivers. My friend Noopur Davis calls this a “skeleton architecture”. The idea is that there will be sufficient structure to have these bits work together, but not so much as to get in the way of a creative and innovative Agile process. That may be a rather tricky razor’s edge to achieve, which is something to consider as we build big, complex systems involving multiple parts. Nonetheless, creating a “skeleton architecture” likely causes some separation between design and implementation.
2. As this pre-structuring becomes a distinct set of tasks (what we usually call architecture), it fosters, by its very nature, structuring specialists, i.e., “architects”. Now we have separation of roles, one of the very things that SCRUM was meant to bridge, and collapsing of which lies at the heart of DevOps philosophy. I’ve written quite a lot about what skills make an architect. Please take a look at Securing Systems, and Secrets Of A Cyber Security Architect for my views, my experiences, and my attempts to describe security architecture as a discipline. A natural and organic separation occurs between structurers and doers, between architects and implementers as each role coalesces. 

Once structuring and implementing split into discrete roles, and I would add, disciplines, then the very nature of Agile empowerment to learn from implementing practically guarantees that implementations diverge from any plans. In the ideal world, these changes would be communicated as a feedback loop from implementation to design. But SCRUM does not specifically account for design divergences in its rituals. Designers need to be present during daily standup meetings, post-sprint retrospectives, and SCRUM planning rituals. Undoubtedly, this critical divergence feedback does occur in some organizations. But, as we see during threat modelling sessions, for whatever reasons, the communications fail, or all too often fail to lead to an appropriately updated design.

Interestingly, the threat model, if treated as a process and not a one-time product can provide precisely the right vehicle for coordination between doing and structure, between coding and architecture. 

“A journey of understanding over a security or privacy snapshot.”

–The Threat Modeling Manifesto, Value #3

Dialog is key to establishing the common understandings that lead to value, while documents record those understandings, and enable measurement.

–The Threat Modeling Manifesto, Principle #4

When implementors must change structures (or protocols, or flows, anything that affects the system’s architecture), this must be reflected in the threat model, which must mirror everyone’s current understanding of the system’s state. That’s because any change to structure has the potential for shifting which threats apply, and the negative impacts from successful exploitations. Threat modelling, by its very nature, must always be holistic. For further detail on why this is so, I suggest any of the threat modelling books, all of which explain this. My own Secrets Of A Cyber Security Architect devoted several sections specifically to the requirement for holism. 

Thus, a threat model that is continually updated can become the vehicle and repository for a design-implementation feedback loop.

Value #5:

“Continuous refinement over a single delivery.”

Threat model must, by its very nature deal in both structure (architecture) and details of implementation: engineering. Threat modelling today remains, as I’ve stated many times, both art and science. One imagines threats as they might proceed through a system – often times, a system that is yet to be built, or is currently in the process of building. So, we may have to imagine the system, as well. 

The threats are science: they operate in particular ways against particular technologies and implementations. That’s hard engineering. Most defenses also are science: they do what they do. No defense is security “magic”. They are all limited.

Threat modelling is an analysis that applies the science of threats and defenses to abstracted imaginings of system through mental activity, i.e., “analysis”. 

Even though a particular weakness may not be present today, or we simply don’t know yet whether it’s present, we think through how attackers would exploit it if it did exist. On the weakness side, we consider every potential weakness that has some likelihood of appearing in the system under analysis, based upon weaknesses of that sort that have appeared at some observable frequency in similar technologies and architectures. This is a conjunction of the art of imagining coupled to the science of past exploitation data, such as we can gather. Art and science.

The Threat Modeling Manifesto’s definition attempts to express this:

“Threat modeling is analyzing representations of a system to highlight concerns about security and privacy characteristics.”

Threat modelling then requires “representations”, that is, imaginings of aspects of the target of the analysis, typically, drawings and other artifacts. We “analyze”, which is to say, the art and science that I briefly described above for “characteristics”, another somewhat imprecise concept.

In order to be successful, we need experts in the structure of the system, its implementation, the system’s verification, experts in the relevant threats and defenses, and probably representatives who can speak to system stakeholder’s needs and expectations. It often helps to also include those people who are charged with driving and coordinating delivery efforts. In other words, one has to have all the art and all the science present in order to do a reasonably decent job. To misquote terribly, “It takes a village to threat model”^[1].

Participation and collaboration are encapsulated in the Threat Modeling Manifesto.

Consider the suggested Threat Modeling Manifesto Pattern, “Varied Viewpoints”:

“Assemble a diverse team with appropriate subject matter experts and cross-functional collaboration.”

And, our Anti-pattern, “Hero Threat Modeler” speaks to a need to avoid over-dependence on a single individual:

“Threat modeling does not depend on one’s innate ability or unique mindset; everyone can and should do it.”

Another frequent occurrence during threat modelling sessions will be when a member of the team who would not have normally been included during leader-only threat modelling names a weakness of which the leaders were unaware, or which other active participants simply missed in their analysis. Diversity in threat modelling applies more system knowledge and understanding, which leads to more complete models. 

Since attackers are adaptive and creative and there are a multiplicity of objectives for attack, diversity while modelling counters attacker variety. I will note that through the many threat models I’ve built (probably 1000’s), I’ve been forced to recognize my limitations, that I am perfectly capable of missing something obvious. It’s my co-modelers who keep me honest and who check my work.

Failure to uncover the differences between representation and implementation risk failure of one’s threat model. As Izar explained, even a lead designer may very likely not understand how the system actually works. To counter this problem, both Izar and I, and probably all the co-authors of the Threat Modeling Manifesto account for these misunderstandings through our threat modelling processes. One way is to ensure that a diverse and knowledge collection of people participate. Another is to begin a session by correcting the current representations of the system through the collected knowledge of participants.

I often say that one of the outcomes of broad threat model participation has nothing to do with the threat model. Attendees come away with a fuller understanding of how the system actually works. The threat model becomes a valued communication mechanism with effects beyond its security and privacy purposes. As both art and science, we find it critical to be inclusive rather than exclusive when threat modelling. Your analyses will improve dramatically.

Cheers,

/brook

---

^[1] Please don’t make assumptions about my personal politics just because I mangled the title of one of Hilary Clinton’s books. Thanks

Bad Design Decisions…

Sometime in 2018, I noticed a pitch for a new “sport” watch that generated its energy from your body. That sounded cool, so I sent the pitch to my spouse. Then I forgot all about it.

Imagine my surprise when the brand new, version 1 watch appeared out from under our Christmas tree.

Immediately, things started to go South. The watch was supposed to turn on; it remained blank, silent, obviously inoperable no matter how I followed the single page of textual directions. Try as I might, it was just a hunk of metal, plastics, and circuits (presumably).

Naturally, I called the Matrix PowerWatch Support who patiently detailed exactly the instructions I’d already determined were not working. I always find that a bit patronizing, since I did take the time describe everything that I’d tried. They sent me the accessory USB charger, gratis, but still no joy with that particular watch. Another call to support.

I should have taken the initial experience as a sign. There would be many more exchanges over the following year+, until I had realized that Matrix PowerWatch had apparently intentionally burned all of its version one buyers.

After six weeks or so of wrangling, Matrix finally sent me a working watch, which, per instructions, started, paired, and appeared to function.

The PowerWatch keeps time, as expected. It also measures steps, though unlike my couple of early Fitbits, the PowerWatch has no ability to calibrate one’s stride. My longer than typical legs usually take in somewhat more distance than others more typically fitted.

Since the PowerWatch is constantly measuring my body temperature (and its disparity with ambient), the watch will “calculate” calories expended, as well. I found this last to be an interesting data point. I didn’t take the calory count as gospel. But the watch could definitely tell me when I’d had a good work out versus days spent at a computer.

The watch also has a stopwatch, timer, and lap counter. I found these almost unworkable given the way the two physical buttons on the watch work, or don’t work much of the time. So I never used these functions. My mobile phone is a lot easier.

The next sign of trouble was syncing the watch to my phone. The instructions said to go to the watch’s settings, scroll down to “Sync”, then press both buttons. This operation only sometimes worked. Pressing both buttons at the same time requires serious hand contortions that I found painful at times, certainly uncomfortable. Little did I know as I started having troubles that sync failures were a symptom of poor design choices for which there would be no remedy.

More calls and emails to Matrix Support, which often went unanswered for days. I suspect that there was actually only a single support person, maybe a couple? Anyway, as you may imagine, the process and the watch were becoming quite a time sink.

Finally, the support engineer told me that I could simply pull the app display down and the watch would sync. Eureka! Why wasn’t that in any instructions or described somewhere on the PowerWatch web site?

Finally, was I past issues and on to happy use? Um, no.

For, in their wisdom, the PowerWatch software engineers had chosen to either write their own Bluetooth stack, or muck with one in some way as to make the stack predictably unstable.

I have had and continue to use without trouble, dozens of Bluetooth devices, the vast majority of which pair and sync (if needed) with phones, tablets, computers nearly seamlessly. Some early devices had weak antennas, so syncing had to be performed in close proximity. But once paired, syncing just happens. Not so, the Matrix PowerWatch.

The watch would sync for a few days. Then the pairing between watch and phone would break in some way. The sync operation would timeout. If I restarted the app, syncing might occur. Once in a while, a reboot of the phone might get it going.

Meanwhile, all the other Bluetooth devices paired with that phone continued to work fine. Which has led me to believe that it’s the watch software at fault.

But you see, the PowerWatch designers, in their infinite lack of foresight and wisdom, provided no way to simply reboot the watch! Better would have been a way to restart the Bluetooth stack. Observing instability, one prepares for it. Actually, one prepares for instability, period, especially in version 1!

I’m pretty sure that a watch system reboot would have solved this problem. But no, the only recourse is to unpair the watch and reset it to start up mode: factory reset. That’s the only option.

Unpairing the watch removes all ones accumulated fitness data (and any personalization) from the watch and application. One is forced to start over completely. There is no other course.

Worse, if you don’t sync the watch for a few days (who wants to go through this horror even once/day?), it both loses fitness data AND the Bluetooth connection blows up. Everything is lost. This happens regularly, predictably.

One of the first design rules that I learned, years ago, was, “never, ever lose your customer’s data”. Failure one.

Rule 2: for difficult or tricky protocols, try to use a vetted implementation. Roll your own at you and your customers’ risk. Perhaps, failure two?

Rule 3: Plan for errors and crashes, especially from conditions you can’t anticipate during development. Failure three, for sure.

If I’d had a way to restart without losing data, say, turn Bluetooth off, then on again, or even a non-destructive reboot, I wouldn’t be writing this today. All software contains bugs. We have learned how to work around that. Except, not the Matrix folks.

As I worked through my debugging process with PowerWatch Customer Support, they promised me that in March, 2020, there would be a new version that “fixed this problem”.

I hope that I’ve learned how to be patient, especially with software fixes. After a lifetime designing, writing, debugging, securing software, I understand the difficulties in finding and fixing bugs, then getting the fixes into the field. I get it.

Waiting, I lived with resetting my watch every 2-4 weeks, losing whatever accumulated data I had, sure. Amongst the many things competing for my time, a sport watch isn’t really at the top of my list. I’d already spent far too much time on PowerWatch.

March finally arrived. I dutifully attempted the upgrade (that was a three-day battle in and of itself). Guess what? The promised upgrade was only for version 2 watches! My watch’s system was precisely the same after the upgrade as before. All problems still active, nothing changed. I’d been misled, purposely?

As you may imagine, I was upset. I still am. Matrix apparently decided that version 1 customers weren’t worth keeping. Maybe there’s some hardware or operating system limitation that prevents the upgrade? I really don’t care. That shouldn’t be my worry.

Was there an upgrade offer from Matrix? That’s often used as a path forward. Say, an offer of 50% off the retail for version 2? Nope. Crickets.

What I didn’t realize is that there was another lurking reason for burning version 1 customers: the battery died a month later. Now, no watch. Just a pile of metal, plastics, and circuits. The free charger still works. But I’ve no watch to charge on it.

I suspect that Matrix realized that not only is the watch inherently unstable due to its Bluetooth issues and poor design. But they also must have understood that these version 1 devices were going to stop functioning soon, anyway. Matrix threw away its first customers. They’ve turned their back on us. It’s “goodbye and good luck”.

My only recourse is to publish my sad story so that others will be wary of a company that clearly doesn’t much care about at least some of its customers. And, engineers who don’t program defensively, who fail to give users ways to deal with their mistakes.

Don’t buy PowerWatch, unless you understand with whom you’re going to be dealing.

Cheers,

/brook