Linux kernel fuzzing and root cause analysis

Skill level: Intermediate to Advanced

Duration: 4 days

Instructor: Vitaly Nikolenko


Course description:

Unlike the Linux kernel exploitation training, this course focuses on vulnerability discovery and root cause analysis rather than developing proof of concept code for specific kernel versions and bypassing various kernel exploitation mitigations. We will discuss a few publicly available kernel fuzzing frameworks and focus on practical approaches for kernel fuzzing.

The first part of this training focuses on common classes of kernel vulnerabilities and ways to identify these vulnerabilities through automated testing and/or static code analysis techniques. Though several fuzzing options / frameworks are discussed, the main focus is on practical fuzzing approaches with code coverage. The training starts off with an introduction to code coverage and implementation details for coverage collection in the kernel. We will discuss various input mutation techniques and ways to improve code coverage (e.g., collecting "live" samples and input deduplication). Some static code analysis techniques for identifying common vulnerabilities will be presented as well, e.g., writing your own LLVM plugins and performing taint analysis to trace user inputs. Several common types of kernel vulnerabilities will be discussed in detail and then generalised as "patterns" that are useful for static analysis.

The second part is dedicated to root cause analysis, i.e., analysing the vulnerability once a stable reproducer is obtained or developing the reproducer if the crash is not stable. Several useful kernel debugging techniques that can aid in root cause analysis will be discussed.

We strongly advise taking the kernel exploitation training first unless you are already familiar with common kernel vulnerability classes / exploitation techniques.

Prerequisites:

  • Python / C / C++ knowledge
  • Basic understanding of the LLVM infrastructure
  • Experience in Linux kernel exploitation / knowledge of common Linux kernel vulnerability classes (consider taking Linux kernel exploitation techniques (x86_64) first)
  • Familiarity with GDB

Who should attend:

  • Reverse engineers, bug hunters and exploit developers
  • Information security professionals experienced in user-land exploitation

Hardware and software:

  • Base OS - Windows, macOS, Linux
  • Virtualisation software that allows you to import VMs in standard OVA/OVF format
  • At least 80GB of free disk space
  • At least 16 GB of RAM

Key learning objectives:

  • Practical approaches for Linux kernel fuzzing
  • Patterns for common kernel vulnerability classes
  • Static code analysis techniques
  • Root cause analysis and debugging techniques

Course agenda:

  • Introduction to kernel fuzzing (random, template-based, guided by code coverage, symbolic/concolic execution engines)
  • Fuzzing guided by code coverage - implementation details
  • Input mutation techniques
  • Practical kernel fuzzing frameworks (syzkaller / kernel AFL, kAFL)
  • KCOV - kernel code coverage implementation
  • In-depth introduction to syzkaller
  • Kernel sanitizers: KASAN, KMSAN, KUBSAN
  • Improving code coverage through input deduplication
  • Identifying common classes of kernel vulnerabilities
    • Use-after-free
    • Heap overflows
    • Out-of-bounds accesses
    • Double fetches
    • Race conditions
  • Hardware options and optimisations
  • Static analysis techniques (Coccinelle, Sparse)
  • Taint analysis with LLVM
  • Manual code analysis techniques
  • Introduction to root-cause analysis
  • Kernel debugging techniques
  • Timeless debugging