Tag Archive | Suricata Developer training

Rust and Suricata

In the newly released Suricata 4.0, one of the major new features is integration of Rust. In the words of the Rust Language project, “Rust is a systems programming language that runs blazingly fast, prevents segfaults, and guarantees thread safety.”

For those of you who were at SuriCon DC, the topic of Rust isn’t new. Pierre Chiffliers’  talk ‘Securing Security Tools’ described why switching existing C/C++ tools to partly use Rust language parsers makes sense. See his slides here: pdf. After his talk, we have started looking into the proof of concept proposed by Pierre.

Rust was new to us so the process involved learning the language.

Major Components

  1. rust language: a safe systems programming language that compiles to native code and can be linked into a C program
  2. nom: a rust macro parser generator
  3. suricata – rust layer: connection between the rust code and the Suricata API’s

Rust Usage in Suricata

Rust parsers might be used for many things:

  • simple header parsers where we’d use a rust/nom parser for a header like DNS
  • specific data types, I’m thinking mostly about ASN.1 and mime here as those are notorious for vulnerabilities
  • full application layer parsing and logging in Suricata

Experiment

After the initial testing and playing by Jason Ish and Victor Julien, we have decided to move forward with Rust. However, as the language is young and our understanding of it is even younger, we will consider it experimental at first.

The experimental phase will have to teach us a couple of things:

  1. how does it work wrt stability and performance
  2. what does it take to support Suricata with Rust extensions
  3. how mature is the Rust ecosystem

Depending on our experiences we’re planning to take between 6 and 12 months for this experimental phase. As we’re considering this experimental we’re going to be quite liberal with regards to updates to the Rust code inside our stable branch.

The initial 4.0 code contains the following parts:

  • Re-implementation of the DNS parser, which is mostly stateless.
  • A NFS parser with logging, file extraction and basic detection integration.
  • A NTP parser done by Pierre that for a significant part is an external crate

Some initial thoughts

As predicted by many, the learning curve for Rust is indeed quite steep. It’s not just the Rust language itself, which has some novel but tricky concepts around lifetimes. It’s also the ‘nom’ framework which although generally easy to work with, can emit really cryptic compiler error messages if small mistakes (like a missing comma) are made.

Rust is really nice though. In our testing it performs well, and its really nice to have code that doesn’t segfault. In Rust, if the compiler accepts the code, it usually works. Also, conditions that may lead to undefined state in C are detected by Rust which will cause the program to to abort, instead of continuing, possibly in a condition where exploitation is possible.

Pierre’s ‘Rusticata’ work

In our initial integration we’re not using much of Pierre’s work yet. The main reasons for this are:

  • Pierre is implementing parsers as external modules (Crates in Rust speak) and we want to limit the number of external dependencies we introduce
  • learning the Rust language and ecosystem all but required that we did everything ourselves, even if it meant redoing what Pierre already did (even in less efficient ways sometimes)

However we have added another layer of ‘experimental’ for including Pierre’s work step by step.

By using the –enable-rust-experimental configure flag the parser Pierre has written are enabled.

Timeline

  • Summer 2017: Suricata 4.0 was released with experimental Rust support, implementing DNS, NFS and NTP
  • Late Fall 2017: Suricata 4.1 with still experimental Rust support, likely adding a few protocols
  • Spring 2018: Suricata 5.0 with default non-experimental (and probably mandatory) Rust support

Trying our Rust support

If you want to play with the current support, here is a page with installation instructions.

On a modern distro, it’s really as simple as installing rustc and cargo, followed by passing –enable-rust to configure. Tested on Ubuntu, Fedora and FreeBSD.

Training

In September we’re offering the third edition of our annual Suricata developer training. In this 5-day event we’re teaching how to extend Suricata. We’re planning to spend about one day on covering the Rust integration.

See: https://suricata_events.eventbrite.com/

Common questions

Q: Are you moving to Rust completely?

A: No, not anytime soon. Time will tell how this experiment evolves.

Q: Does it make Suricata harder to use?

A: No, to end-users the Rust support is transparent other than installing the build dependencies rustc and cargo.

Q: Does it make Suricata harder to compile?

A: No, not on modern distributions. All you need is the rust compiler and cargo.

Q: Where can I learn more?

A: We’re offering a developer training at https://suricata_events.eventbrite.com/

5-Day Suricata Developer Training

A short time ago we announced the first edition of the Suricata Developer training. I thought it be a good time to explain what we have in mind.

First, we’re planning to make this an annual event. I’m very excited about this. It should be great fun to have a week of development related discussions. I’m sure we can all learn a great deal, and share lots of ideas. Of course the social part is a nice addition.

Also, this is a great “stick behind the door” (as we’d say in Dutch) to finally get some much needed dev docs done, including architecture overview diagrams, etc.
What will we be teaching:

  1. General Suricata development basics: everything from git, how to QA, unittests, debugging, etc.
  2. Architecture overview
    1. API’s
    2. threading
    3. packets, flows, detection and output
  3. Extending Suricata — the beef of the training:
    1. packet decoders and detection plugins
    2. app layer protocol detection, parsing, state keeping
    3. app layer detection engine integration
    4. adding logging modules

Structure

Each day will start with lectures on each of the topics. You will get an overview of the API, learn about performance aspects, how threading comes into play, etc.

After this there will be assignments/challenges to apply the newly learned skills. This should be very interactive with lots of room for questions and discussions. We’ll be providing various assignments for multiple skill levels.

Teachers

From OISF we will have lead developer Victor Julien, core developers Eric Leblond and Jason Ish in the room to give the lectures and help answer questions. Also present to assist in general Suricata related questions: Matt Jonkman and Peter Manev.

Participants

So who is this for? We target people who what to learn how to extend Suricata. If you want to add protocols, detection options or maybe new output methods, then this is for you. Or maybe you want to be a ‘core’ developer on Suricata. Then this would be an excellent start of getting into it.

Skill-set:
– advanced C experience
– Linux Experience
– Network / Security Experience
– Basic Suricata End User Experience
=> for testing your code

If you’re not yet experienced with running Suricata, we would like to suggest attending a user training first. If you book your dev seat first hit us up for discount code on one of the training sessions. We’re planning one in Europe before summer, exact date and location is to be announced.

Money

We’ll be charging for the trainings. The revenue is used to cover the cost of the event itself (travel, hotels, etc). Whats left goes into the foundations general development budget. So by attending the training you will support Suricata’s development.

Location

This first edition is generously hosted by Napatech in Copenhagen, Denmark.
If you want to book, please do so through this Eventbrite link:
https://www.eventbrite.com/e/5-day-suricata-developer-training-in-copenhagen-denmark-open-to-the-public-tickets-15667305332
Questions and general feedback and thoughts are welcome!