-
SoK: Prudent Evaluation Practices for Fuzzing
Authors:
Moritz Schloegel,
Nils Bars,
Nico Schiller,
Lukas Bernhard,
Tobias Scharnowski,
Addison Crump,
Arash Ale Ebrahim,
Nicolai Bissantz,
Marius Muench,
Thorsten Holz
Abstract:
Fuzzing has proven to be a highly effective approach to uncover software bugs over the past decade. After AFL popularized the groundbreaking concept of lightweight coverage feedback, the field of fuzzing has seen a vast amount of scientific work proposing new techniques, improving methodological aspects of existing strategies, or porting existing methods to new domains. All such work must demonstr…
▽ More
Fuzzing has proven to be a highly effective approach to uncover software bugs over the past decade. After AFL popularized the groundbreaking concept of lightweight coverage feedback, the field of fuzzing has seen a vast amount of scientific work proposing new techniques, improving methodological aspects of existing strategies, or porting existing methods to new domains. All such work must demonstrate its merit by showing its applicability to a problem, measuring its performance, and often showing its superiority over existing works in a thorough, empirical evaluation. Yet, fuzzing is highly sensitive to its target, environment, and circumstances, e.g., randomness in the testing process. After all, relying on randomness is one of the core principles of fuzzing, governing many aspects of a fuzzer's behavior. Combined with the often highly difficult to control environment, the reproducibility of experiments is a crucial concern and requires a prudent evaluation setup. To address these threats to validity, several works, most notably Evaluating Fuzz Testing by Klees et al., have outlined how a carefully designed evaluation setup should be implemented, but it remains unknown to what extent their recommendations have been adopted in practice. In this work, we systematically analyze the evaluation of 150 fuzzing papers published at the top venues between 2018 and 2023. We study how existing guidelines are implemented and observe potential shortcomings and pitfalls. We find a surprising disregard of the existing guidelines regarding statistical tests and systematic errors in fuzzing evaluations. For example, when investigating reported bugs, ...
△ Less
Submitted 16 May, 2024;
originally announced May 2024.
-
Technical Report: Hardening Code Obfuscation Against Automated Attacks
Authors:
Moritz Schloegel,
Tim Blazytko,
Moritz Contag,
Cornelius Aschermann,
Julius Basler,
Thorsten Holz,
Ali Abbasi
Abstract:
Software obfuscation is a crucial technology to protect intellectual property and manage digital rights within our society. Despite its huge practical importance, both commercial and academic state-of-the-art obfuscation methods are vulnerable to a plethora of automated deobfuscation attacks, such as symbolic execution, taint analysis, or program synthesis. While several enhanced obfuscation techn…
▽ More
Software obfuscation is a crucial technology to protect intellectual property and manage digital rights within our society. Despite its huge practical importance, both commercial and academic state-of-the-art obfuscation methods are vulnerable to a plethora of automated deobfuscation attacks, such as symbolic execution, taint analysis, or program synthesis. While several enhanced obfuscation techniques were recently proposed to thwart taint analysis or symbolic execution, they either impose a prohibitive runtime overhead or can be removed in an automated way (e.g., via compiler optimizations). In general, these techniques suffer from focusing on a single attack vector, allowing an attacker to switch to other, more effective techniques, such as program synthesis. In this work, we present Loki, an approach for software obfuscation that is resilient against all known automated deobfuscation attacks. To this end, we use and efficiently combine multiple techniques, including a generic approach to synthesize formally verified expressions of arbitrary complexity. Contrary to state-of-the-art approaches that rely on a few hardcoded generation rules, our expressions are more diverse and harder to pattern match against. Even the most recent state-of-the-art research on Mixed-Boolean Arithmetic (MBA) deobfuscation fails to simplify them. Moreover, Loki protects against previously unaccounted attack vectors such as program synthesis, for which it reduces the success rate to merely 19%. In a comprehensive evaluation, we show that our design incurs significantly less overhead while providing a much stronger protection level compared to existing works.
△ Less
Submitted 17 June, 2022; v1 submitted 16 June, 2021;
originally announced June 2021.
-
A Look at the Dark Side of Hardware Reverse Engineering -- A Case Study
Authors:
Sebastian Wallat,
Marc Fyrbiak,
Moritz Schlögel,
Christof Paar
Abstract:
A massive threat to the modern and complex IC production chain is the use of untrusted off-shore foundries which are able to infringe valuable hardware design IP or to inject hardware Trojans causing severe loss of safety and security. Similarly, market dominating SRAM-based FPGAs are vulnerable to both attacks since the crucial gate-level netlist can be retrieved even in field for the majority of…
▽ More
A massive threat to the modern and complex IC production chain is the use of untrusted off-shore foundries which are able to infringe valuable hardware design IP or to inject hardware Trojans causing severe loss of safety and security. Similarly, market dominating SRAM-based FPGAs are vulnerable to both attacks since the crucial gate-level netlist can be retrieved even in field for the majority of deployed device series. In order to perform IP infringement or Trojan injection, reverse engineering (parts of) the hardware design is necessary to understand its internal workings. Even though IP protection and obfuscation techniques exist to hinder both attacks, the security of most techniques is doubtful since realistic capabilities of reverse engineering are often neglected. The contribution of our work is twofold: first, we carefully review an IP watermarking scheme tailored to FPGAs and improve its security by using opaque predicates. In addition, we show novel reverse engineering strategies on proposed opaque predicate implementations that again enables to automatically detect and alter watermarks. Second, we demonstrate automatic injection of hardware Trojans specifically tailored for third-party cryptographic IP gate-level netlists. More precisely, we extend our understanding of adversary's capabilities by presenting how block and stream cipher implementations can be surreptitiously weakened.
△ Less
Submitted 1 October, 2019;
originally announced October 2019.
