Archive | news RSS for this section

Suricata 4.1 beta 1 ready for testing

We are proud to announce that the first beta release for the upcoming Suricata 4.1 is ready for testing. This release is brought to you by the OISF development team with the help 25 community contributors.

Download:
https://www.openinfosecfoundation.org/download/suricata-4.1.0-beta1.tar.gz

We invite everyone to test this release and report your experiences to us.

Main features additions

  • SMBv1/2/3 parsing, logging, file extraction
  • AF_PACKET XDP and eBPF support for high speed packet capture
  • JA3 TLS client fingerprinting
  • HTTP: handle sessions that only have a response, or start with a response
  • Windows: MinGW is now supported
  • File extraction v2: deduplication; hash-based naming; json metadata and cleanup tooling
  • Eve metadata: from rules (metadata keyword) and traffic (flowbits etc)
  • Pcap directory mode: process all pcaps in a directory
  • Detect: transformation support
  • Eve: new more compact DNS record format
  • TFTP: basic logging
  • HTTP Flash file decompression support
  • All tickets: https://redmine.openinfosecfoundation.org/versions/105

Special thanks

Giuseppe Longo, Mats Klepsland, Pierre Chifflier, Ralph Broenink, Wolfgang Hotwagner, Danny Browning, Pascal Delalande, Jesper Dangaard Brouer, Maurizio Abba, Alexander Gozman, Antoine LUONG, David DIALLO, Martin Natano, Ruslan Usmanov, Alfredo Cardigliano, Antti Tönkyrä, Brandon Sterne, Clément Galland, Dana Helwig, Daniel Humphries, Gaurav Singh, Nick Price, Philippe Antoine, Thomas Andrejak, Jason Taylor

SuriCon 2018

Come meet the Suricata community and development team to discuss all things Suricata at the fourth edition of the annual Suricata Conference. SuriCon 2018 will be held in November in Vancouver, Canada: https://suricon.net

Our call for presentations is still open, so please submit your ideas!

Also, we’re still looking for sponsors for the event.

About Suricata

Suricata is a high performance Network Threat Detection, IDS, IPS and Network Security Monitoring engine. Open Source and owned by a community run non-profit foundation, the Open Information Security Foundation (OISF). Suricata is developed by the OISF, its supporting vendors and the community.

Suricata 4.0.4 available!

suri-400x400

We are pleased to announce Suricata 4.0.4.  This is a security update fixing a number of security issues, as well as a fair number of regular issues.

Security

CVE-2018-6794 was requested for issue #2440

Changes

  • Bug #2306: suricata 4 deadlocks during failed output log reopening
  • Bug #2361: rule reload hangup
  • Bug #2389: BUG_ON asserts in AppLayerIncFlowCounter (4.0.x)
  • Bug #2392: libhtp 0.5.26 (4.0.x)
  • Bug #2422: [4.0.3] af_packet: a leak that (possibly) breaks an inline channel
  • Bug #2438: various config parsing issues
  • Bug #2439: Fix timestamp offline when pcap timestamp is zero (4.0.x)
  • Bug #2440: stream engine bypass issue (4.0.x)
  • Bug #2441: der parser: bad input consumes cpu and memory (4.0.x)
  • Bug #2443: DNP3 memcpy buffer overflow (4.0.x)
  • Bug #2444: rust/dns: Core Dump with malformed traffic (4.0.x)
  • Bug #2445: http bodies / file_data: thread space creation writing out of bounds

Download

https://www.openinfosecfoundation.org/download/suricata-4.0.4.tar.gz

Special thanks

Wolfgang Hotwagner, Kirill Shipulin, Pierre Chifflier, Alexander Gozman, Martin Natano, Maurizio Abba, Nick Price, Philippe Antoine, AFL

SuriCon 2018

Call for presentations is open and tickets for SuriCon 2018 are available: https://suricon.net/

About Suricata

Suricata is a high performance Network Threat Detection, IDS, IPS and Network Security Monitoring engine. Open Source and owned by a community run non-profit foundation, the Open Information Security Foundation (OISF). Suricata is developed by the OISF, its supporting vendors and the community.

Suricata 4.0.3 available!

suri-400x400

We are pleased to announce Suricata 4.0.3.  This is regular bug fix release fixing various issues.

Note: this release was first released as 4.0.2, but due to a packaging mistake it contained the wrong branch.

Changes

  • Feature #2245: decoder for ieee802.1AH traffic
  • Bug #798: stats.log in yaml config – append option – missing
  • Bug #891: detect-engine.profile does not err out in incorrect values – suricata.yaml
  • Bug #961: max pending packets variable parsing
  • Bug #1185: napatech: cppcheck warning
  • Bug #2215: Lost events writing to unix socket
  • Bug #2230: valgrind memcheck – 4.0.0-dev (rev 1180687)
  • Bug #2250: detect: mixing byte_extract and isdataat leads to FP & FN
  • Bug #2263: content matches disregarded when using dns_query on udp traffic
  • Bug #2274: ParseSizeString in util-misc.c: Null-pointer dereference
  • Bug #2275: ConfGetInt in conf.c: NULL-pointer dereference
  • Bug #2276: conf: NULL-pointer dereference in CoredumpLoadConfig
  • Bug #2293: rules: depth < content rules not rejected
  • Bug #2324: segfault in http_start (4.0.x)
  • Bug #2325: Suricata segfaults on ICMP and flowint check (4.0.x)

Download

https://www.openinfosecfoundation.org/download/suricata-4.0.3.tar.gz

Special thanks

Danny Browning, Harley H, Travis Green, Wolfgang Hotwagner, Edward Fjellskål

About Suricata

Suricata is a high performance Network Threat Detection, IDS, IPS and Network Security Monitoring engine. Open Source and owned by a community run non-profit foundation, the Open Information Security Foundation (OISF). Suricata is developed by the OISF, its supporting vendors and the community.

Suricata 3.2.5 available!

suri-400x400

We are pleased to announce Suricata 3.2.5. This release fixes a number of issues.

This will be the last 3.2 release, as 3.2 will go ‘end of life’ later this month.

Changes

  • Bug #2328: detect: mixing byte_extract and isdataat leads to FP & FN (3.2.x)
  • Bug #2329: various config parsing issues
  • Bug #2330: rules: depth < content rules not rejected (3.2.x)
  • Bug #2331: Suricata segfaults on ICMP and flowint check (3.2.x)

Download

https://www.openinfosecfoundation.org/download/suricata-3.2.5.tar.gz

End of life announcement

The 3.2 branch will be end-of-life in 2 months, so on December 18. After this it will receive no more updates of any kind, so please plan for your upgrade to Suricata 4.0+ before that date.

https://suricata-ids.org/about/eol-policy/

Special thanks

Wolfgang Hotwagner, Harley H, Edward Fjellskål

About Suricata

Suricata is a high performance Network Threat Detection, IDS, IPS and Network Security Monitoring engine. Open Source and owned by a community run non-profit foundation, the Open Information Security Foundation (OISF). Suricata is developed by the OISF, its supporting vendors and the community.

Announcing Suricata-Update

We are excited to announce the first alpha release of our new tool for updating Suricata rules. This is a new rule update tool specifically built for Suricata with a goal of being useful out of the box, even with no configuration.

This release also introduces the Suricata Intel Index, which is currently a list of available rule sources which Suricata-Update is aware of. The idea here is to make it easier for users to find available rule sets, as well as allowing rule writers to make their rules more discoverable.

Features include:

  • Default to Emerging Threats Open ruleset if no configuration provided.
  • Automatic discovery of Suricata version for use in ruleset URLs.
  • Flowbit resolution
  • Enable, disable, drop and modify filters that should be familiar to users of Pulled Pork and Oinkmaster.
  • Easy enabling of additional rule sets from the index.

We invite all interested users to checkout the Quick Start documentation, and leave us feedback on the Suricata-Update issue tracker.

If you are a rule writer and would like to get listed in the index, please  leave a ticket in the issue tracker.

Github Project Page

https://github.com/OISF/suricata-update

Issue Tracker

https://redmine.openinfosecfoundation.org/projects/suricata-update

 

SuriCon 2017 brainstorm summary

At SuriCon in Prague, we spent an afternoon discussing the roadmap for Suricata for the next year. It was a fun an interactive session with lots of discussions and suggestions.

During the session, Matt Jonkman maintained Google spreadsheet, and this post summarizes that. Only issues with ‘high’ priority are mentioned here, as this is already more than we can get done.

We’ve created a high-level ticket that is referenced by all tickets discussed at SuriCon, so this includes the medium and low priority ones: #2309.

Failing better

The idea here is that we should make sure we get more value in ‘failure’ conditions: for example packet loss, or incomplete traffic (due to routing, etc).

A high-level ticket is #2278

Specifically, DNS was brought up: #2272. Also related is the ability to modify memcaps on the fly so that tuning doesn’t always require a full restart of Suricata: #2285.

Suricata sets internal events when protocol anomalies are encountered. These are exposed to the rule language and also used as ‘stats counters’ in the stats.log. A feature request here is to mimic Bro’s ‘weird log’ as well, so create a log output for all these events #2282.

Rule language

Unification and clean up of the ‘buffer’ selection (e.g. ‘content:”abc”; http_uri;’ vs ‘file_data; content:”abc”;’). First step is to agree on a naming scheme and a list of names for all existing buffers: #2285.

Rule writers also asked for simpler ways to express ‘ends with’ and ‘starts with’ (#741, #742) and buffer length (#735).

Being able to write rules that match on both request and response (e.g. HTTP uri and response status) #2280.

Victor is working on a rule ‘transformation API’, allowing buffer transformations (e.g. strip_whitespace). It became clear that the transforms need to support arguments (#1006) and that Lua should be supported (#2290).

File Extraction

Using the SHA256 hash of a file at it’s filename. First store as a temp file, then rename when it’s done. Also, a way to deduplicate storing files #1948

Document best practices for dealing with file extraction #2286.

There is also interest in being able to detect partial file transfers, like when a browser prefetches part of a file #2284.

Eric’s FTP file exaction work is almost complete: #550.

TLS

Multiple people expressed interest in JA3 SSL fingerprinting: #2192. Mats Klepsland is working on that.

While not a finalized standard, TLS 1.3 support (#2279) is important as well.

QA

The need for easy test case / pcap sharing was expressed. E.g. Michal mentioned that the Bro project has pcaps with test cases. Probably at first a wiki page listing sources of test cases. Ticket #2322.

Misc

HTTP byte-range support #1576.

TCP (and defrag) overlap handling simplification: #2281.

Recording pcaps only for alerting streams: #120, #385, #2219.

Traffic ID ruleset: #2291. A ruleset to classify common high bandwidth traffic, such as video streaming services. In part to assist in flow bypass for performance.

Call for help

The tasks above are together a lot of work, and it’s unlikely that we’ll be able to complete all of there. So if you or your organization would like to help, please let us know! All forms of help are welcome: code, funding, test cases, documentation, testing, designs, etc.  We are also growing our team, but can only do this with financial support from this community – if you are interested in donating to help us grow our dev team, please contact us at info@oisf.net.

Suricata 4.0.1 available!

suri-400x400

We are pleased to announce Suricata 4.0.1.  This is regular bug fix release fixing various issues. Also added is much improved Napatech support.

Changes

  • Feature #2114: Redis output: add RPUSH support
  • Feature #2152: Packet and Drop Counters for Napatech
  • Bug #2050: TLS rule mixes up server and client certificates
  • Bug #2064: Rules with dual classtype do not error
  • Bug #2074: detect msg: memory leak
  • Bug #2102: Rules with dual sid do not error
  • Bug #2103: Rules with dual rev do not error
  • Bug #2151: The documentation does not reflect current suricata.yaml regarding cpu-affinity
  • Bug #2194: rust/nfs: sigabrt/rust panic – 4.0.0-dev (rev fc22943)
  • Bug #2197: rust build with lua enabled fails on x86
  • Bug #2201: af_packet: suricata leaks memory with use-mmap enabled and incorrect BPF filter
  • Bug #2207: DNS UDP “Response” parsing recording an incorrect value
  • Bug #2208: mis-structured JSON stats output if interface name is shortened
  • Bug #2226: improve error message if stream memcaps too low
  • Bug #2228: enforcing specific number of threads with autofp does not seem to work
  • Bug #2244: detect state uses broken offset logic (4.0.x)

Download

https://www.openinfosecfoundation.org/download/suricata-4.0.1.tar.gz

Special thanks

Qidu Sy, Phil Young – Napatech, Mats Klepsland, Sascha Steinbiss, Alexander Gozman, Derek Kingsbury, Julian Wecke, Pierre Chifflier, Jason Taylor

Trainings

Conference attendees get a 20% discount!

SuriCon 2017

Less than one month to SuriCon 2017! Come meet the Suricata community and development team to discuss all things Suricata at the third edition of the annual Suricata Conference. SuriCon 2017 will be next month in Prague: https://suricon.net

About Suricata

Suricata is a high performance Network Threat Detection, IDS, IPS and Network Security Monitoring engine. Open Source and owned by a community run non-profit foundation, the Open Information Security Foundation (OISF). Suricata is developed by the OISF, its supporting vendors and the community.

Suricata 3.2.4 available!

suri-400x400

We are pleased to announce Suricata 3.2.4. This a security update fixing important issues. Additionally, it fixes various minor issues.

Changes

  • Bug #2241: smb dcerpc segfaults in StubDataParser (3.2.x)
  • Bug #2231: Redundant content checks may cause Suricata DoS condition on a insignificant traffic rate
  • Bug #2214: detect state uses broken offset logic
  • Bug #2234: TLS rule mixes up server and client certificates (3.2.x)
  • Bug #2235: DNS UDP “Response” parsing recording an incorrect timestamp (3.2.x)
  • Bug #2236: af_packet: suricata leaks memory with use-mmap enabled and incorrect BPF filter (3.2.x)
  • Bug #2237: Redis output: add RPUSH support (3.2.x)
  • Bug #2238: detect duplicate ‘meta’ keywords (3.2.x)
  • Bug #2239: documentation does not reflect current suricata.yaml regarding cpu-affinity (3.2.x)
  • Bug #2242: improve error message if stream memcap too low (3.2.x)
  • Bug #2243: enforcing specific number of threads with autofp does not seem to work (3.2.x)

Download

https://www.openinfosecfoundation.org/download/suricata-3.2.4.tar.gz

End of life announcement

The 3.2 branch will be end-of-life in 2 months, so on December 18. After this it will receive no more updates of any kind, so please plan for your upgrade to Suricata 4.0+ before that date.

https://suricata-ids.org/about/eol-policy/

Special thanks

Jack Covington, Kirill Shipulin – Positive Technologies, Qidu Sy, Mats Klepsland, Derek Kingsbury, Julian Wecke, Alexander Gozman, AFL project, Coverity Scan

Trainings

Conference attendees get a 20% discount!

SuriCon 2017

Less than one month to SuriCon 2017! Come meet the Suricata community and development team to discuss all things Suricata at the third edition of the annual Suricata Conference. SuriCon 2017 will be next month in Prague: https://suricon.net

About Suricata

Suricata is a high performance Network Threat Detection, IDS, IPS and Network Security Monitoring engine. Open Source and owned by a community run non-profit foundation, the Open Information Security Foundation (OISF). Suricata is developed by the OISF, its supporting vendors and the community.

Github repository move

Ever since we’ve used github for Suricata development we’ve used Victors personal repository at inliniac/suricata as the main repository. As of today this has been changed.

The new repository location is OISF/suricata, which lives in the OISF organization at github.

It is a move of Victors original repo, so the existing open pull requests moved to OISF/suricata. Existing URLs should still work, as they are automatically redirected to the OISF/suricata repository.

Other than it looking better and more professional, this move as the advantage of the use of github teams we define in the OISF organization. This means it’s going to be easier to assign reviews to team members and outside contributors. We’ll add regular contributors as ‘outside collaborators’ so PRs can be assigned to them for review.

Like with the personal repository previously at inliniac/suricata, Victor remains the only one with rights to pushing to the official branches (master and master-z.y.x). However in his absence the rest of the team can create emergency branches and releases.

In the coming weeks we’ll try to update all existing documentation references to inliniac/suricata.

Please let us know if there are any unexpected issues from this move.

Rust and Suricata

In the newly released Suricata 4.0, one of the major new features is integration of Rust. In the words of the Rust Language project, “Rust is a systems programming language that runs blazingly fast, prevents segfaults, and guarantees thread safety.”

For those of you who were at SuriCon DC, the topic of Rust isn’t new. Pierre Chiffliers’  talk ‘Securing Security Tools’ described why switching existing C/C++ tools to partly use Rust language parsers makes sense. See his slides here: pdf. After his talk, we have started looking into the proof of concept proposed by Pierre.

Rust was new to us so the process involved learning the language.

Major Components

  1. rust language: a safe systems programming language that compiles to native code and can be linked into a C program
  2. nom: a rust macro parser generator
  3. suricata – rust layer: connection between the rust code and the Suricata API’s

Rust Usage in Suricata

Rust parsers might be used for many things:

  • simple header parsers where we’d use a rust/nom parser for a header like DNS
  • specific data types, I’m thinking mostly about ASN.1 and mime here as those are notorious for vulnerabilities
  • full application layer parsing and logging in Suricata

Experiment

After the initial testing and playing by Jason Ish and Victor Julien, we have decided to move forward with Rust. However, as the language is young and our understanding of it is even younger, we will consider it experimental at first.

The experimental phase will have to teach us a couple of things:

  1. how does it work wrt stability and performance
  2. what does it take to support Suricata with Rust extensions
  3. how mature is the Rust ecosystem

Depending on our experiences we’re planning to take between 6 and 12 months for this experimental phase. As we’re considering this experimental we’re going to be quite liberal with regards to updates to the Rust code inside our stable branch.

The initial 4.0 code contains the following parts:

  • Re-implementation of the DNS parser, which is mostly stateless.
  • A NFS parser with logging, file extraction and basic detection integration.
  • A NTP parser done by Pierre that for a significant part is an external crate

Some initial thoughts

As predicted by many, the learning curve for Rust is indeed quite steep. It’s not just the Rust language itself, which has some novel but tricky concepts around lifetimes. It’s also the ‘nom’ framework which although generally easy to work with, can emit really cryptic compiler error messages if small mistakes (like a missing comma) are made.

Rust is really nice though. In our testing it performs well, and its really nice to have code that doesn’t segfault. In Rust, if the compiler accepts the code, it usually works. Also, conditions that may lead to undefined state in C are detected by Rust which will cause the program to to abort, instead of continuing, possibly in a condition where exploitation is possible.

Pierre’s ‘Rusticata’ work

In our initial integration we’re not using much of Pierre’s work yet. The main reasons for this are:

  • Pierre is implementing parsers as external modules (Crates in Rust speak) and we want to limit the number of external dependencies we introduce
  • learning the Rust language and ecosystem all but required that we did everything ourselves, even if it meant redoing what Pierre already did (even in less efficient ways sometimes)

However we have added another layer of ‘experimental’ for including Pierre’s work step by step.

By using the –enable-rust-experimental configure flag the parser Pierre has written are enabled.

Timeline

  • Summer 2017: Suricata 4.0 was released with experimental Rust support, implementing DNS, NFS and NTP
  • Late Fall 2017: Suricata 4.1 with still experimental Rust support, likely adding a few protocols
  • Spring 2018: Suricata 5.0 with default non-experimental (and probably mandatory) Rust support

Trying our Rust support

If you want to play with the current support, here is a page with installation instructions.

On a modern distro, it’s really as simple as installing rustc and cargo, followed by passing –enable-rust to configure. Tested on Ubuntu, Fedora and FreeBSD.

Training

In September we’re offering the third edition of our annual Suricata developer training. In this 5-day event we’re teaching how to extend Suricata. We’re planning to spend about one day on covering the Rust integration.

See: https://suricata_events.eventbrite.com/

Common questions

Q: Are you moving to Rust completely?

A: No, not anytime soon. Time will tell how this experiment evolves.

Q: Does it make Suricata harder to use?

A: No, to end-users the Rust support is transparent other than installing the build dependencies rustc and cargo.

Q: Does it make Suricata harder to compile?

A: No, not on modern distributions. All you need is the rust compiler and cargo.

Q: Where can I learn more?

A: We’re offering a developer training at https://suricata_events.eventbrite.com/