Tag Archive | suricata

Suricata 4.0.3 available!

suri-400x400

We are pleased to announce Suricata 4.0.3.  This is regular bug fix release fixing various issues.

Note: this release was first released as 4.0.2, but due to a packaging mistake it contained the wrong branch.

Changes

  • Feature #2245: decoder for ieee802.1AH traffic
  • Bug #798: stats.log in yaml config – append option – missing
  • Bug #891: detect-engine.profile does not err out in incorrect values – suricata.yaml
  • Bug #961: max pending packets variable parsing
  • Bug #1185: napatech: cppcheck warning
  • Bug #2215: Lost events writing to unix socket
  • Bug #2230: valgrind memcheck – 4.0.0-dev (rev 1180687)
  • Bug #2250: detect: mixing byte_extract and isdataat leads to FP & FN
  • Bug #2263: content matches disregarded when using dns_query on udp traffic
  • Bug #2274: ParseSizeString in util-misc.c: Null-pointer dereference
  • Bug #2275: ConfGetInt in conf.c: NULL-pointer dereference
  • Bug #2276: conf: NULL-pointer dereference in CoredumpLoadConfig
  • Bug #2293: rules: depth < content rules not rejected
  • Bug #2324: segfault in http_start (4.0.x)
  • Bug #2325: Suricata segfaults on ICMP and flowint check (4.0.x)

Download

https://www.openinfosecfoundation.org/download/suricata-4.0.3.tar.gz

Special thanks

Danny Browning, Harley H, Travis Green, Wolfgang Hotwagner, Edward Fjellskål

About Suricata

Suricata is a high performance Network Threat Detection, IDS, IPS and Network Security Monitoring engine. Open Source and owned by a community run non-profit foundation, the Open Information Security Foundation (OISF). Suricata is developed by the OISF, its supporting vendors and the community.

Suricata 3.2.5 available!

suri-400x400

We are pleased to announce Suricata 3.2.5. This release fixes a number of issues.

This will be the last 3.2 release, as 3.2 will go ‘end of life’ later this month.

Changes

  • Bug #2328: detect: mixing byte_extract and isdataat leads to FP & FN (3.2.x)
  • Bug #2329: various config parsing issues
  • Bug #2330: rules: depth < content rules not rejected (3.2.x)
  • Bug #2331: Suricata segfaults on ICMP and flowint check (3.2.x)

Download

https://www.openinfosecfoundation.org/download/suricata-3.2.5.tar.gz

End of life announcement

The 3.2 branch will be end-of-life in 2 months, so on December 18. After this it will receive no more updates of any kind, so please plan for your upgrade to Suricata 4.0+ before that date.

https://suricata-ids.org/about/eol-policy/

Special thanks

Wolfgang Hotwagner, Harley H, Edward Fjellskål

About Suricata

Suricata is a high performance Network Threat Detection, IDS, IPS and Network Security Monitoring engine. Open Source and owned by a community run non-profit foundation, the Open Information Security Foundation (OISF). Suricata is developed by the OISF, its supporting vendors and the community.

Announcing Suricata-Update

We are excited to announce the first alpha release of our new tool for updating Suricata rules. This is a new rule update tool specifically built for Suricata with a goal of being useful out of the box, even with no configuration.

This release also introduces the Suricata Intel Index, which is currently a list of available rule sources which Suricata-Update is aware of. The idea here is to make it easier for users to find available rule sets, as well as allowing rule writers to make their rules more discoverable.

Features include:

  • Default to Emerging Threats Open ruleset if no configuration provided.
  • Automatic discovery of Suricata version for use in ruleset URLs.
  • Flowbit resolution
  • Enable, disable, drop and modify filters that should be familiar to users of Pulled Pork and Oinkmaster.
  • Easy enabling of additional rule sets from the index.

We invite all interested users to checkout the Quick Start documentation, and leave us feedback on the Suricata-Update issue tracker.

If you are a rule writer and would like to get listed in the index, please  leave a ticket in the issue tracker.

Github Project Page

https://github.com/OISF/suricata-update

Issue Tracker

https://redmine.openinfosecfoundation.org/projects/suricata-update

 

SuriCon 2017 brainstorm summary

At SuriCon in Prague, we spent an afternoon discussing the roadmap for Suricata for the next year. It was a fun an interactive session with lots of discussions and suggestions.

During the session, Matt Jonkman maintained Google spreadsheet, and this post summarizes that. Only issues with ‘high’ priority are mentioned here, as this is already more than we can get done.

We’ve created a high-level ticket that is referenced by all tickets discussed at SuriCon, so this includes the medium and low priority ones: #2309.

Failing better

The idea here is that we should make sure we get more value in ‘failure’ conditions: for example packet loss, or incomplete traffic (due to routing, etc).

A high-level ticket is #2278

Specifically, DNS was brought up: #2272. Also related is the ability to modify memcaps on the fly so that tuning doesn’t always require a full restart of Suricata: #2285.

Suricata sets internal events when protocol anomalies are encountered. These are exposed to the rule language and also used as ‘stats counters’ in the stats.log. A feature request here is to mimic Bro’s ‘weird log’ as well, so create a log output for all these events #2282.

Rule language

Unification and clean up of the ‘buffer’ selection (e.g. ‘content:”abc”; http_uri;’ vs ‘file_data; content:”abc”;’). First step is to agree on a naming scheme and a list of names for all existing buffers: #2285.

Rule writers also asked for simpler ways to express ‘ends with’ and ‘starts with’ (#741, #742) and buffer length (#735).

Being able to write rules that match on both request and response (e.g. HTTP uri and response status) #2280.

Victor is working on a rule ‘transformation API’, allowing buffer transformations (e.g. strip_whitespace). It became clear that the transforms need to support arguments (#1006) and that Lua should be supported (#2290).

File Extraction

Using the SHA256 hash of a file at it’s filename. First store as a temp file, then rename when it’s done. Also, a way to deduplicate storing files #1948

Document best practices for dealing with file extraction #2286.

There is also interest in being able to detect partial file transfers, like when a browser prefetches part of a file #2284.

Eric’s FTP file exaction work is almost complete: #550.

TLS

Multiple people expressed interest in JA3 SSL fingerprinting: #2192. Mats Klepsland is working on that.

While not a finalized standard, TLS 1.3 support (#2279) is important as well.

QA

The need for easy test case / pcap sharing was expressed. E.g. Michal mentioned that the Bro project has pcaps with test cases. Probably at first a wiki page listing sources of test cases. Ticket #2322.

Misc

HTTP byte-range support #1576.

TCP (and defrag) overlap handling simplification: #2281.

Recording pcaps only for alerting streams: #120, #385, #2219.

Traffic ID ruleset: #2291. A ruleset to classify common high bandwidth traffic, such as video streaming services. In part to assist in flow bypass for performance.

Call for help

The tasks above are together a lot of work, and it’s unlikely that we’ll be able to complete all of there. So if you or your organization would like to help, please let us know! All forms of help are welcome: code, funding, test cases, documentation, testing, designs, etc.  We are also growing our team, but can only do this with financial support from this community – if you are interested in donating to help us grow our dev team, please contact us at info@oisf.net.

Suricata 4.0.1 available!

suri-400x400

We are pleased to announce Suricata 4.0.1.  This is regular bug fix release fixing various issues. Also added is much improved Napatech support.

Changes

  • Feature #2114: Redis output: add RPUSH support
  • Feature #2152: Packet and Drop Counters for Napatech
  • Bug #2050: TLS rule mixes up server and client certificates
  • Bug #2064: Rules with dual classtype do not error
  • Bug #2074: detect msg: memory leak
  • Bug #2102: Rules with dual sid do not error
  • Bug #2103: Rules with dual rev do not error
  • Bug #2151: The documentation does not reflect current suricata.yaml regarding cpu-affinity
  • Bug #2194: rust/nfs: sigabrt/rust panic – 4.0.0-dev (rev fc22943)
  • Bug #2197: rust build with lua enabled fails on x86
  • Bug #2201: af_packet: suricata leaks memory with use-mmap enabled and incorrect BPF filter
  • Bug #2207: DNS UDP “Response” parsing recording an incorrect value
  • Bug #2208: mis-structured JSON stats output if interface name is shortened
  • Bug #2226: improve error message if stream memcaps too low
  • Bug #2228: enforcing specific number of threads with autofp does not seem to work
  • Bug #2244: detect state uses broken offset logic (4.0.x)

Download

https://www.openinfosecfoundation.org/download/suricata-4.0.1.tar.gz

Special thanks

Qidu Sy, Phil Young – Napatech, Mats Klepsland, Sascha Steinbiss, Alexander Gozman, Derek Kingsbury, Julian Wecke, Pierre Chifflier, Jason Taylor

Trainings

Conference attendees get a 20% discount!

SuriCon 2017

Less than one month to SuriCon 2017! Come meet the Suricata community and development team to discuss all things Suricata at the third edition of the annual Suricata Conference. SuriCon 2017 will be next month in Prague: https://suricon.net

About Suricata

Suricata is a high performance Network Threat Detection, IDS, IPS and Network Security Monitoring engine. Open Source and owned by a community run non-profit foundation, the Open Information Security Foundation (OISF). Suricata is developed by the OISF, its supporting vendors and the community.

Suricata 3.2.4 available!

suri-400x400

We are pleased to announce Suricata 3.2.4. This a security update fixing important issues. Additionally, it fixes various minor issues.

Changes

  • Bug #2241: smb dcerpc segfaults in StubDataParser (3.2.x)
  • Bug #2231: Redundant content checks may cause Suricata DoS condition on a insignificant traffic rate
  • Bug #2214: detect state uses broken offset logic
  • Bug #2234: TLS rule mixes up server and client certificates (3.2.x)
  • Bug #2235: DNS UDP “Response” parsing recording an incorrect timestamp (3.2.x)
  • Bug #2236: af_packet: suricata leaks memory with use-mmap enabled and incorrect BPF filter (3.2.x)
  • Bug #2237: Redis output: add RPUSH support (3.2.x)
  • Bug #2238: detect duplicate ‘meta’ keywords (3.2.x)
  • Bug #2239: documentation does not reflect current suricata.yaml regarding cpu-affinity (3.2.x)
  • Bug #2242: improve error message if stream memcap too low (3.2.x)
  • Bug #2243: enforcing specific number of threads with autofp does not seem to work (3.2.x)

Download

https://www.openinfosecfoundation.org/download/suricata-3.2.4.tar.gz

End of life announcement

The 3.2 branch will be end-of-life in 2 months, so on December 18. After this it will receive no more updates of any kind, so please plan for your upgrade to Suricata 4.0+ before that date.

https://suricata-ids.org/about/eol-policy/

Special thanks

Jack Covington, Kirill Shipulin – Positive Technologies, Qidu Sy, Mats Klepsland, Derek Kingsbury, Julian Wecke, Alexander Gozman, AFL project, Coverity Scan

Trainings

Conference attendees get a 20% discount!

SuriCon 2017

Less than one month to SuriCon 2017! Come meet the Suricata community and development team to discuss all things Suricata at the third edition of the annual Suricata Conference. SuriCon 2017 will be next month in Prague: https://suricon.net

About Suricata

Suricata is a high performance Network Threat Detection, IDS, IPS and Network Security Monitoring engine. Open Source and owned by a community run non-profit foundation, the Open Information Security Foundation (OISF). Suricata is developed by the OISF, its supporting vendors and the community.

Rust and Suricata

In the newly released Suricata 4.0, one of the major new features is integration of Rust. In the words of the Rust Language project, “Rust is a systems programming language that runs blazingly fast, prevents segfaults, and guarantees thread safety.”

For those of you who were at SuriCon DC, the topic of Rust isn’t new. Pierre Chiffliers’  talk ‘Securing Security Tools’ described why switching existing C/C++ tools to partly use Rust language parsers makes sense. See his slides here: pdf. After his talk, we have started looking into the proof of concept proposed by Pierre.

Rust was new to us so the process involved learning the language.

Major Components

  1. rust language: a safe systems programming language that compiles to native code and can be linked into a C program
  2. nom: a rust macro parser generator
  3. suricata – rust layer: connection between the rust code and the Suricata API’s

Rust Usage in Suricata

Rust parsers might be used for many things:

  • simple header parsers where we’d use a rust/nom parser for a header like DNS
  • specific data types, I’m thinking mostly about ASN.1 and mime here as those are notorious for vulnerabilities
  • full application layer parsing and logging in Suricata

Experiment

After the initial testing and playing by Jason Ish and Victor Julien, we have decided to move forward with Rust. However, as the language is young and our understanding of it is even younger, we will consider it experimental at first.

The experimental phase will have to teach us a couple of things:

  1. how does it work wrt stability and performance
  2. what does it take to support Suricata with Rust extensions
  3. how mature is the Rust ecosystem

Depending on our experiences we’re planning to take between 6 and 12 months for this experimental phase. As we’re considering this experimental we’re going to be quite liberal with regards to updates to the Rust code inside our stable branch.

The initial 4.0 code contains the following parts:

  • Re-implementation of the DNS parser, which is mostly stateless.
  • A NFS parser with logging, file extraction and basic detection integration.
  • A NTP parser done by Pierre that for a significant part is an external crate

Some initial thoughts

As predicted by many, the learning curve for Rust is indeed quite steep. It’s not just the Rust language itself, which has some novel but tricky concepts around lifetimes. It’s also the ‘nom’ framework which although generally easy to work with, can emit really cryptic compiler error messages if small mistakes (like a missing comma) are made.

Rust is really nice though. In our testing it performs well, and its really nice to have code that doesn’t segfault. In Rust, if the compiler accepts the code, it usually works. Also, conditions that may lead to undefined state in C are detected by Rust which will cause the program to to abort, instead of continuing, possibly in a condition where exploitation is possible.

Pierre’s ‘Rusticata’ work

In our initial integration we’re not using much of Pierre’s work yet. The main reasons for this are:

  • Pierre is implementing parsers as external modules (Crates in Rust speak) and we want to limit the number of external dependencies we introduce
  • learning the Rust language and ecosystem all but required that we did everything ourselves, even if it meant redoing what Pierre already did (even in less efficient ways sometimes)

However we have added another layer of ‘experimental’ for including Pierre’s work step by step.

By using the –enable-rust-experimental configure flag the parser Pierre has written are enabled.

Timeline

  • Summer 2017: Suricata 4.0 was released with experimental Rust support, implementing DNS, NFS and NTP
  • Late Fall 2017: Suricata 4.1 with still experimental Rust support, likely adding a few protocols
  • Spring 2018: Suricata 5.0 with default non-experimental (and probably mandatory) Rust support

Trying our Rust support

If you want to play with the current support, here is a page with installation instructions.

On a modern distro, it’s really as simple as installing rustc and cargo, followed by passing –enable-rust to configure. Tested on Ubuntu, Fedora and FreeBSD.

Training

In September we’re offering the third edition of our annual Suricata developer training. In this 5-day event we’re teaching how to extend Suricata. We’re planning to spend about one day on covering the Rust integration.

See: https://suricata_events.eventbrite.com/

Common questions

Q: Are you moving to Rust completely?

A: No, not anytime soon. Time will tell how this experiment evolves.

Q: Does it make Suricata harder to use?

A: No, to end-users the Rust support is transparent other than installing the build dependencies rustc and cargo.

Q: Does it make Suricata harder to compile?

A: No, not on modern distributions. All you need is the rust compiler and cargo.

Q: Where can I learn more?

A: We’re offering a developer training at https://suricata_events.eventbrite.com/

Suricata 4.0 released!

We are thrilled to announce Suricata 4.0. This is a major new release, improving detection capabilities, adding new output options and more protocols.suri-400x400

Improved Detection

Based on valuable feedback from the rule writing teams at Emerging Threats and Positive Technologies we’ve added and improved many rule keywords for inspecting HTTP, SSH and other protocols. TLS additions were contributed by Mats Klepsland at NorCERT, including decoding, logging and matching on TLS serial numbers. Additionally, Suricata now allows rule writers to specify who’s the target in a signature. This information is used in EVE JSON logging to give more context with alerts.

TLS improved, NFS added

More on the TLS side: A major new feature is support for STARTTLS in SMTP and FTP. TLS sessions will now be logged in these cases. More goodness from Mats Klepsland. Also, TLS session resumption logging is now supported thanks to the work of Ray Ruvinskiy. Additional TLS logging improvements were done by Paulo Pacheco.

NFS decoding, logging and file extraction was added as part of the experimental Rust support. Read on for more information about Rust.

More EVE JSON

EVE is extended in several ways:

  • in the case of encapsulated traffic both the inner and outer ip addresses and ports are logged
  • the ‘vars’ facility logs flowbits and other vars. This can also be used to log data extracted from traffic using a PCRE statement in rules
  • EVE can now be rotated based on time
  • EVE was extended to optionally log the HTTP request and/or response bodies
  • the (partial) flow record is added to alert records.

The ‘vars’ facility is one of the main improvements here, as it is now possible for a signature to accurately extract information for logging. For instance, a signature can extract an advertised software version or other information such as the recipient of an email. [https://blog.inliniac.net/2016/12/20/suricata-bits-ints-and-vars/]

First Step into a Safer Future

This is the first release in which we’ve implemented parts in the Rust language using the Nom parser framework. This work is inspired by Pierre Chiffliers’ (ANSSI), talk at SuriCon 2016 (pdf). By compiling with –enable-rust you’ll get a basic NFS parser and a re-implementation of the DNS parser. Feedback on this is highly appreciated.

The Rust support is still experimental, as we are continuing to explore how it functions, performs and what it will take to support it in the community. Additionally we included Pierre Chiffliers Rust parsers work. This uses external Rust parser ‘crates’ and is enabled by using –enable-rust-experimental. Initially this adds a NTP parser.

Under the Hood

A major TCP stream engine update is included. This should lead to better performance and less configuration, especially in IPS mode. First steps in TCP GAP recovery were taken, with implementations for DNS and NFS.

For developers, this release makes extending the detection engine with high performance keywords a lot easier. Adding a new high performance keyword using multi pattern matching does now requires only a few lines of code.

Documentation

David Wharton at SecureWorks has created a section in the documentation for rule writers who have a background in Snort. It documents changes that are relevant for writing rules.

Next steps

Based on the feedback we’ll get we’re expecting to do a 4.0.1 release in a month or so. Then we’ll start work on the next major release, which is 4.1. This is planned for late fall, ETA before SuriCon in Prague.

Feature tickets

  • Feature #806: Implement STARTTLS support
  • Feature #2006: tls: decode certificate serial number
  • Feature #1969: TLS transactions with session resumption are not logged
  • Feature #2129: nfs: parser, logger and detection
  • Feature #2130: dns: rust parser with stateless behaviour
  • Feature #2131: nfs: implement GAP support
  • Feature #2163: ntp parser
  • Feature #2164: rust: external parser crate support
  • Feature #2077: Additional HTTP Header Contents and Negation
  • Feature #2011: eve.alert: print outside IP addresses on alerts on traffic inside tunnels
  • Feature #2095: eve: http body in alert event
  • Feature #1978: Using date in logs name
  • Feature #1998: eve.tls: custom TLS logging
  • Feature #2046: Support custom file permissions per logger
  • Feature #2123: unix-socket: additional runmodes
  • Feature #2132: eve: flowbit and other vars logging
  • Feature #2156: Add app_proto or partial flow entry to alerts
  • Feature #744: Teredo configuration
  • Feature #2061: lua: get timestamps from flow
  • Feature #1953: lua: expose flow_id
  • Feature #1748: lua: expose tx in alert lua scripts
  • Feature #1636: Signal rotation of unified2 log file without restart
  • Feature #2133: unix socket: add/remove hostbits
  • Feature #805: Add support for applayer change

For all other closed tickets please see the full changelog of 4.0.

Download

https://www.openinfosecfoundation.org/download/suricata-4.0.0.tar.gz

Special thanks

Mats Klepsland – for his major contributions: many EVE and TLS features

Pierre Chifflier – for paving the way for the Rust experiment and being very helpful while learning Rust and Nom.

Additionally: Abdullah Ada, Jérémy Beaume, Sebastian Garcia, Alexander Gozman, Giuseppe Longo, Paulo Pacheco, Selivanov Pavel, Ray Ruvinskiy, Peter Sanders, David Wharton, Jon Zeolla, the AFL project and Coverity Scan.

Suricata Trainings and Events

We have several community events and trainings on the calendar and in the works for 2017… here are some of the highlights:

  • 5-Day Developer Deep Dive Training – Sept 11 – 15, 2017, Cork, Ireland – led by Victor Julien, Eric Leblond, and Jason Ish
  • Rule Writing Training @ DerbyCon – Sept 20 – 24, 2017 – SOLD OUT!
  • Rule Writing Training @ SuriCon – Nov 13 – 14, 2017
  • 2-Day Suricata Training @ SuriCon – Nov 13 – 14, 2017
  • SuriCon 2017 – Nov 15 – 17, 2017, Prague

Details and registration for all our events can be found at https://suricata_events.eventbrite.com. Don’t delay as space is limited.

We also offer custom training events for your team – contact us at info@oisf.net for details.

About Suricata

Suricata is a high performance Network Threat Detection, IDS, IPS and Network Security Monitoring engine. Open Source and owned by a community run non-profit foundation, the Open Information Security Foundation (OISF). Suricata is developed by the OISF, its supporting vendors and the community.

Suricata 4.0.0-rc2 ready for testing!

suri-400x400

We are proud to announce that the second release candidate for the upcoming Suricata 4.0.0 is ready for your testing.

We’re aiming for a final 4.0.0 release about 2 weeks from now. Please help us test!

Changes

  • Feature #744: Teredo configuration
  • Feature #1748: lua: expose tx in alert lua scripts
  • Bug #1855: alert number output
  • Bug #1888: noalert in a pass rule disables the rule
  • Bug #1957: PCRE lowercase enforcement in http_host buffer does not allow for upper case in hex-encoding
  • Bug #1958: Possible confusion or bypass within the stream engine with retransmits.
  • Bug #2110: isdataat: keyword memleak
  • Bug #2162: rust/nfs: reachable asserting rust panic
  • Bug #2175: rust/nfs: panic – 4.0.0-dev (rev 7c25a2d)
  • Bug #2176: gcc 7.1.1 ‘format truncation’ compiler warnings
  • Bug #2177: asn1/der: stack overflow

Download

https://www.openinfosecfoundation.org/download/suricata-4.0.0-rc2.tar.gz

Special thanks

AFL project, Abdullah Ada

Trainings

SuriCon 2017

Come meet the Suricata community and development team to discuss all things Suricata at the third edition of the annual Suricata Conference. SuriCon 2017 will be in November in Prague: https://suricon.net

About Suricata

Suricata is a high performance Network Threat Detection, IDS, IPS and Network Security Monitoring engine. Open Source and owned by a community run non-profit foundation, the Open Information Security Foundation (OISF). Suricata is developed by the OISF, its supporting vendors and the community.

Suricata 3.2.3 available!

suri-400x400

We are pleased to announce Suricata 3.2.3. This release fixes a fairly small number of issues. The most important one is an issue we found using AFL in the DER/ASN1 parser. This has the potential to crash your Suricata instance.

Changes

  • Bug #2089: engine file logging race condition (3.2.x)
  • Bug #2173: openbsd: pcap with raw datalink not supported (3.2.x)
  • Bug #2178: asn1/der: stack overflow (3.2.x)
  • Bug #2179: Possible confusion or bypass within the stream engine with retransmits. (3.2.x)
  • Bug #2183: gcc 7.1.1 ‘format truncation’ compiler warnings (3.2.x)

Download

https://www.openinfosecfoundation.org/download/suricata-3.2.3.tar.gz

Special thanks

AFL project

Trainings

SuriCon 2017

Come meet the Suricata community and development team to discuss all things Suricata at the third edition of the annual Suricata Conference. SuriCon 2017 will be in November in Prague: https://suricon.net

About Suricata

Suricata is a high performance Network Threat Detection, IDS, IPS and Network Security Monitoring engine. Open Source and owned by a community run non-profit foundation, the Open Information Security Foundation (OISF). Suricata is developed by the OISF, its supporting vendors and the community.