Japanese researchers have developed a method that detects errors in software build dependencies much faster. To do this, they combine eBPF, a Linux technology that can track events in the kernel, with incremental analysis. In tests, this reduces required analysis time to a fraction of existing methods.
An error in a software project’s dependencies can cause a build to produce an incorrect or outdated result. Verifying these relationships is particularly costly for large-scale projects. Researchers at Waseda University believe they can largely overcome this problem with mkcheck2, according to The Register.
During the build process, the tool examines which files and other resources are actually being used and compares that information with the dependencies recognized by the build system. This allows it to detect missing or incorrectly documented relationships.
Verification via the Linux kernel
The key difference from existing solutions lies in how mkcheck2 monitors the build process. Many existing verification methods rely on ptrace, which requires the operating system to repeatedly interrupt processes in order to inspect system calls. This results in additional context switches and can significantly slow down builds.
Mkcheck2 uses eBPF for this purpose. This allows code within the Linux kernel to track events without halting a process at every system call. The researchers combine this tracing with incremental analysis: after a change, the entire project does not need to be re-examined every time.
The difference is significant. In tests involving 300 open-source projects that use Make, the average analysis time per commit dropped from 1,267.49 to 23.56 seconds, a nearly 54-fold improvement. Furthermore, in some tests, error-detection overhead was 99.7 percent lower than with ptrace-based solutions, according to The Register.
Interesting for CI/CD
For development teams, the biggest benefit is that they can now incorporate checks more frequently into the normal development cycle. An analysis that takes tens of minutes is difficult to run with every commit. When the same check can be performed in just a few tens of seconds, integration into CI/CD pipelines becomes much more realistic.
The researchers state that errors in dependency descriptions are a major cause of failed or incorrect builds. Tools like Make and CMake use these relationships to determine which components need to be rebuilt. A missing dependency can cause modified source code to be excluded, while an unnecessary dependency causes unnecessary work.
Linux only for now
Mkcheck2 is not a universal solution. The eBPF technique used ties the current implementation to Linux. Additionally, several more complex scenarios are not yet fully covered. These include dynamically loaded libraries, network dependencies, memory-mapped files, and distributed builds.
Nevertheless, the results show that checking build dependencies does not necessarily entail a significant performance penalty. This may become relevant now that software is being updated more frequently and rapidly, leading to increasingly intensive use of automated build and test processes.