Developing Secure Software

No software is perfect. This is one of the hard truths of software development. But it does not mean that we cannot try to prevent mistakes, bugs, and flaws, mitigate their impact, and improve how we respond to such problems.

In this article, we take a look at the situation we are facing. It is impossible to say how many undiscovered flaws and vulnerabilities there may be, but we have estimates that offer us a good starting point. In the second half of this article, we look at some of the measures we are taking to prevent these problems and at our response for managing vulnerabilities.

The Situation

In order to understand how we can avoid and prevent bugs and flaws and how we should ideally respond to the remaining problems, we need to know which types of problems can occur, how often, and where.

For a general estimate about how many bugs and flaws there will be in thousand lines of code, people often go to Steve McConnell’s book “Code Complete”. Ignoring some exceptional cases, the statistics say that we should expect between 10 and 50 flaws in every thousand lines of code. Statistically speaking, there is an error in every twentieth or one-hundredth line of code that the compiler misses. This is surprisingly frequent. In truth, it should surprise us that our IT systems are working as well as they are.

But it is not a question of numbers. The type of flaw can be the critical factor. As we are concerned with the development of secure software, we need to consider two types: bugs and flaws. Bugs are run-of-the-mill programming errors. Flaws, on the other hand, are issues within the design itself. To find bugs, we can rely on a large toolkit that helps us scour the code for many types of possible errors, as we will explain below under “Prevention”. Flaws, on the other hand, are harder to spot, since they often come from certain assumptions in the software’s design or development that might be wrong or invalid and that can create vulnerabilities for hackers to exploit.

That is the key for prevention: We need to make sure that flaws will not become exploitable vulnerabilities. Not every error in the code can be exploited. If we understand the structures we can use to prevent attackers from exploiting bugs and flaws, we can make sure that potential mistakes in development do not turn into real vulnerabilities and we get a better sense for whether a bug or flaw we noticed can indeed be exploited. This means that we need a sense for the possible effects and repercussions of potential flaws when analyzing our work, and particularly when planning a new product.

Prevention

Software Architecture

The base of every software is a comprehensive requirement analysis. The results of our requirements analysis are used to design the software architecture in functional and technical terms. That design is the collaborative work of experienced software developers and architects certified by the International Software Architecture Qualification Board (iSAQB®), with input from a product-independent team of security experts that put the concept through its paces from its architecture to its use of the cryptographic basics.

Coding & Testing

When we produce the actual software to fit our designs, we use a range of technologies to catch and remedy any errors in the code at the earliest possible point. A proper versioning system should be standard for any software development (Wibu-Systems uses Git) that enforces a peer review principle. No single developer should be able to weave their code through to the final release without any checks and balances.

SonarLint is used for a static code analysis in the development environment; for the continual static analysis of the source code and for testing technical software quality, we employ the commercial versions of SonarQube on our build server. SonarQube includes more than 5000 static analysis processes (MISRA-C, CERT C/C++) and helps us produce genuinely clean code. We get compiler warnings and the results of the static analysis after every build.

Dynamic code analysis is used to check the application during runtime. Fuzzing can simulate a range of inputs and usually reveals many vulnerabilities. For this purpose, we use Radamsa, LibFuzzer, AFL, WinAFL sowie GCC, ASAN, UBSAN, and Valgrind.

For new implementations, we put unit tests in place to keep on top of regression. These unit tests and our automated integration tests are regularly tweaked in response to flaws. Configuration management and orchestration is entrusted to Ansible and Terraform. All in all, we have a large infrastructure in place for development and testing, which we treat like our code.

Our build and test processes are automated with Jenkins and GitLab Runner, because no developer could produce nightly builds for tests on all the operating systems we support and be ready for working with the results the next morning. Should the automated tests reveal any problems, they are prioritized and entered into the backlog, the causes explored, and the problems, ideally, remedied. Critical problems are flagged when they are discovered, irrespective of how they were noticed or who noticed them. This makes sure that tickets for such critical problems are always given our full attention.

Secure products can only be made by developers that know how to write secure code. Regular training on security aspects like the OWASP Top 10, best practices for specific languages, operating systems, or containers, fuzzing and much more are offered to teach developers how to avoid typical flaws in their code. With our mandatory training during onboarding of new staff, we make sure that IT security – and the right awareness of the issue – will never be forgotten. We use internal and external options for regular training on different security aspects, which should give our people a better sense for the various threats out there and the ways to mitigate the risks. In the long run, this will create an ingrained sense for security in all of our people.

Third Party Software

Any software developer knows that not all features are developed from scratch but brought in by libraries made by other developers. A poor choice of libraries can be a serious issue for software security, one that attackers can exploit to gain access to businesses or even their suppliers. Attacks like this, called supply chain attacks, are on the radar of Wibu-Systems, which has taken measures to reduce its dependencies and introduced a careful selection and close monitoring of the components it does bring in from elsewhere. Wibu-Systems considers build processes particularly risky if they load the current code version on the fly from the supplier’s repository. Although this should mean that all recent secure patches are in place, it also entails a risk of as yet unidentified harmful code slipping through the net. For this reason, our build systems are designed so that they do not require, and consequently do not have, online access.

Operation

Wibu-Systems offers not just software products, but also services. Several of our products are hosted on behalf of our clients, e.g. our CmCloud solution. The services hosted by us are permanently watched, monitored, and serviced, as are our operating systems. We believe that products should be continuously updated and our clients always provided with new versions bringing new features and improvements.

For older versions of our products that might be running on operating systems we no longer fully support, we can offer extended support. CodeMeter Software Version 7.21 includes all security updates for our most recent version and even runs on Windows 7.

The Response

With all of these precautions in place, it should be possible to claim security. But we know and admit that this ideal state has not yet been reached, and the authors of these pages would never suggest that all flaws can be eradicated. Remember that a simple flaw can become an unexpected vulnerability. We all remember Log4Shell, and there is no need to bring that litany of mistakes up again.

If we cannot prevent all problems, we need to consider how to cope with flaws and problems. The first and most important aspect is accepting the existence of flaws, both in the software we create and in the software made by third-party developers. The issue is not whether we made or found an error, but how we deal with it. If we go with the assumption that the vulnerabilities were not created intentionally or by gross negligence, there is no reason to get exasperated.

If the problem originated with another company’s software, we usually want three things: First, that we get a new version without the vulnerability as quickly as possible to be able to protect ourselves and our customers from attacks. Second, that we are given as much information as possible about potential consequences, so that we can decide for ourselves how we should respond. And third, that we are always kept up to date.

But what does the other side look like? To get the best possible support for our clients, we need some structures in place at the company whose software was found to have a vulnerability. This starts with a proper point to report such vulnerabilities and goes all the way to a team to coordinate the response to major vulnerabilities (CERT) and defined lines of communication with representatives for every product and with the client out in the field.

Every single aspect deserves full attention (entire books could be written about the details of every single aspect). Let us consider the coordination systems in place and the necessary structures. If a single vulnerability affecting a single product is already a headache, a vulnerability that affects multiple products can become a serious issue for any business. The limited central resources now have to take care of issues on multiple fronts, and the information about the issue has to be communicated, ideally in a coordinated and standardized fashion. A company of Wibu-Systems’ size could not keep an emergency team on hold to take care of all relevant aspects. This makes it even more important to train the necessary procedures and to learn from every incident in the form of improvements to those procedures. One of the improvements that Wibu-Systems introduced is the Product Security Board. It brings together a member of every product team and the CERT, supported if need be by staff from other departments. This allows us to collect and share our know-how quickly. Every Product Security Board Member speaks for the entire developer team, which keeps the lines of communication short and our response fast. CERT supports the analysis of the vulnerabilities by adding expert cryptography know-how and consolidating the resulting information for communication to the client. We included a more detailed report about our responses in KEYnote 43.

 

KEYnote 46 – Edition Fall/Winter 2023