Tag Archive | starttls

Suricata 4.0 released!

We are thrilled to announce Suricata 4.0. This is a major new release, improving detection capabilities, adding new output options and more protocols.suri-400x400

Improved Detection

Based on valuable feedback from the rule writing teams at Emerging Threats and Positive Technologies we’ve added and improved many rule keywords for inspecting HTTP, SSH and other protocols. TLS additions were contributed by Mats Klepsland at NorCERT, including decoding, logging and matching on TLS serial numbers. Additionally, Suricata now allows rule writers to specify who’s the target in a signature. This information is used in EVE JSON logging to give more context with alerts.

TLS improved, NFS added

More on the TLS side: A major new feature is support for STARTTLS in SMTP and FTP. TLS sessions will now be logged in these cases. More goodness from Mats Klepsland. Also, TLS session resumption logging is now supported thanks to the work of Ray Ruvinskiy. Additional TLS logging improvements were done by Paulo Pacheco.

NFS decoding, logging and file extraction was added as part of the experimental Rust support. Read on for more information about Rust.

More EVE JSON

EVE is extended in several ways:

  • in the case of encapsulated traffic both the inner and outer ip addresses and ports are logged
  • the ‘vars’ facility logs flowbits and other vars. This can also be used to log data extracted from traffic using a PCRE statement in rules
  • EVE can now be rotated based on time
  • EVE was extended to optionally log the HTTP request and/or response bodies
  • the (partial) flow record is added to alert records.

The ‘vars’ facility is one of the main improvements here, as it is now possible for a signature to accurately extract information for logging. For instance, a signature can extract an advertised software version or other information such as the recipient of an email. [https://blog.inliniac.net/2016/12/20/suricata-bits-ints-and-vars/]

First Step into a Safer Future

This is the first release in which we’ve implemented parts in the Rust language using the Nom parser framework. This work is inspired by Pierre Chiffliers’ (ANSSI), talk at SuriCon 2016 (pdf). By compiling with –enable-rust you’ll get a basic NFS parser and a re-implementation of the DNS parser. Feedback on this is highly appreciated.

The Rust support is still experimental, as we are continuing to explore how it functions, performs and what it will take to support it in the community. Additionally we included Pierre Chiffliers Rust parsers work. This uses external Rust parser ‘crates’ and is enabled by using –enable-rust-experimental. Initially this adds a NTP parser.

Under the Hood

A major TCP stream engine update is included. This should lead to better performance and less configuration, especially in IPS mode. First steps in TCP GAP recovery were taken, with implementations for DNS and NFS.

For developers, this release makes extending the detection engine with high performance keywords a lot easier. Adding a new high performance keyword using multi pattern matching does now requires only a few lines of code.

Documentation

David Wharton at SecureWorks has created a section in the documentation for rule writers who have a background in Snort. It documents changes that are relevant for writing rules.

Next steps

Based on the feedback we’ll get we’re expecting to do a 4.0.1 release in a month or so. Then we’ll start work on the next major release, which is 4.1. This is planned for late fall, ETA before SuriCon in Prague.

Feature tickets

  • Feature #806: Implement STARTTLS support
  • Feature #2006: tls: decode certificate serial number
  • Feature #1969: TLS transactions with session resumption are not logged
  • Feature #2129: nfs: parser, logger and detection
  • Feature #2130: dns: rust parser with stateless behaviour
  • Feature #2131: nfs: implement GAP support
  • Feature #2163: ntp parser
  • Feature #2164: rust: external parser crate support
  • Feature #2077: Additional HTTP Header Contents and Negation
  • Feature #2011: eve.alert: print outside IP addresses on alerts on traffic inside tunnels
  • Feature #2095: eve: http body in alert event
  • Feature #1978: Using date in logs name
  • Feature #1998: eve.tls: custom TLS logging
  • Feature #2046: Support custom file permissions per logger
  • Feature #2123: unix-socket: additional runmodes
  • Feature #2132: eve: flowbit and other vars logging
  • Feature #2156: Add app_proto or partial flow entry to alerts
  • Feature #744: Teredo configuration
  • Feature #2061: lua: get timestamps from flow
  • Feature #1953: lua: expose flow_id
  • Feature #1748: lua: expose tx in alert lua scripts
  • Feature #1636: Signal rotation of unified2 log file without restart
  • Feature #2133: unix socket: add/remove hostbits
  • Feature #805: Add support for applayer change

For all other closed tickets please see the full changelog of 4.0.

Download

https://www.openinfosecfoundation.org/download/suricata-4.0.0.tar.gz

Special thanks

Mats Klepsland – for his major contributions: many EVE and TLS features

Pierre Chifflier – for paving the way for the Rust experiment and being very helpful while learning Rust and Nom.

Additionally: Abdullah Ada, Jérémy Beaume, Sebastian Garcia, Alexander Gozman, Giuseppe Longo, Paulo Pacheco, Selivanov Pavel, Ray Ruvinskiy, Peter Sanders, David Wharton, Jon Zeolla, the AFL project and Coverity Scan.

Suricata Trainings and Events

We have several community events and trainings on the calendar and in the works for 2017… here are some of the highlights:

  • 5-Day Developer Deep Dive Training – Sept 11 – 15, 2017, Cork, Ireland – led by Victor Julien, Eric Leblond, and Jason Ish
  • Rule Writing Training @ DerbyCon – Sept 20 – 24, 2017 – SOLD OUT!
  • Rule Writing Training @ SuriCon – Nov 13 – 14, 2017
  • 2-Day Suricata Training @ SuriCon – Nov 13 – 14, 2017
  • SuriCon 2017 – Nov 15 – 17, 2017, Prague

Details and registration for all our events can be found at https://suricata_events.eventbrite.com. Don’t delay as space is limited.

We also offer custom training events for your team – contact us at info@oisf.net for details.

About Suricata

Suricata is a high performance Network Threat Detection, IDS, IPS and Network Security Monitoring engine. Open Source and owned by a community run non-profit foundation, the Open Information Security Foundation (OISF). Suricata is developed by the OISF, its supporting vendors and the community.

Suricata 4.0.0-beta1 ready for testing!

suri-400x400

We are proud to announce that the first release for the upcoming Suricata 4.0.0-beta1 is ready for testing.

This release features our first experimental steps into using the Rust language for creating safer and easier to develop parsers. Inspired by Pierre Chiffliers talk at SuriCon 2016 (pdf). This initial integration does not yet include Pierre’s work, but this will likely change in the near future.
By compiling with –enable-rust you’ll get a basic NFSv3 parser and reimplementation of the DNS parser. Feedback on this is highly appreciated.

A major new feature is support for STARTTLS in SMTP and FTP. TLS sessions will now be logged in these cases. Decoding, logging and matching on TLS sertial numbers was also added. Great work by Mats Klepsland. Also for TLS, session resumption logging is now supported thanks to the work of Ray Ruvinskiy. TLS logging was improved by Paulo Pacheco.

Lots of new HTTP detection options were added to make matching on specific header fields easier and more efficient. New SSH keywords that are fast_pattern capable have also been added. For developers, this release makes extending the detection engine a lot easier.

A major TCP stream engine update is included. This should lead to better performance and less configuration, especially in IPS mode.

EVE is extended in several ways: in the case of encapsulated traffic both the inner and outer ip addresses and ports are logged. The ‘vars’ facility logs flowbits and other vars. This can also be used to extract data from the traffic using PCRE, and then log it. EVE can also be rotated based on time.

David Wharton has created a section in the documentation for rule writers who have a background in Snort. It documents changes that are relevant for writing rules.

Paulo Pacheco has been improving the Redis output performance.

Note that this release finally drops support for CentOS 5, and for libpcap 0.x with it.

Changes

  • Feature #805: Add support for applayer change
  • Feature #806: Implement STARTTLS support
  • Feature #1636: Signal rotation of unified2 log file without restart
  • Feature #1953: lua: expose flow_id
  • Feature #1969: TLS transactions with session resumption are not logged
  • Feature #1978: Using date in logs name
  • Feature #1998: eve.tls: custom TLS logging
  • Feature #2006: tls: decode certificate serial number
  • Feature #2011: eve.alert: print outside IP addresses on alerts on traffic inside tunnels
  • Feature #2046: Support custom file permissions per logger
  • Feature #2061: lua: get timestamps from flow
  • Feature #2077: Additional HTTP Header Contents and Negation
  • Feature #2129: nfs: parser, logger and detection
  • Feature #2130: dns: rust parser with stateless behaviour
  • Feature #2132: eve: flowbit and other vars logging
  • Feature #2133: unix socket: add/remove hostbits
  • Bug #1335: suricata option –pidfile overwrites any file
  • Bug #1470: make install-full can have race conditions on OSX.
  • Bug #1759: CentOS5 EOL tasks
  • Bug #2037: travis: move off legacy support
  • Bug #2039: suricata stops processing when http-log output via unix_stream backs up
  • Bug #2041: bad checksum 0xffff
  • Bug #2044: af-packet: faulty VLAN handling in tpacket-v3 mode
  • Bug #2045: geoip: compile warning on CentOS 7
  • Bug #2049: Empty rule files cause failure exit code without corresponding message
  • Bug #2051: ippair: xbit unset memory leak
  • Bug #2053: ippair: pair is direction sensitive
  • Bug #2070: file store: file log / file store mismatch with multiple files
  • Bug #2072: app-layer: fix memleak on bad traffic
  • Bug #2078: http body handling: failed assertion
  • Bug #2088: modbus: clang-4.0 compiler warnings
  • Bug #2093: Handle TCP stream gaps.
  • Bug #2097: “Name of device should not be null” appears in suricata.log when using pfring with configuration from suricata.yaml
  • Bug #2098: isdataat: fix parsing issue with leading spaces
  • Bug #2108: pfring: errors when compiled with asan/debug
  • Bug #2111: doc: links towards http_header_names
  • Bug #2112: doc: links towards certain http_ keywords not working
  • Bug #2113: Race condition starting Unix Server
  • Bug #2118: defrag – overlap issue in linux policy
  • Bug #2125: ASAN SEGV – Suricata version 4.0dev (rev 922a27e)
  • Optimization #521: Introduce per stream thread segment pool
  • Optimization #1873: Classtypes missing on decoder-events,files, and stream-events

Download

https://www.openinfosecfoundation.org/download/suricata-4.0.0-beta1.tar.gz

Special thanks

Mats Klepsland – for his major contributions: many EVE and TLS features

Pierre Chifflier – for paving the way for the Rust experiment and being very helpful while learning Rust and Nom.

Additionally: Jérémy Beaume, Alexander Gozman, Paulo Pacheco, Ray Ruvinskiy, Peter Sanders, David Wharton, Jon Zeolla

Trainings

SuriCon 2017

Come meet the Suricata community and development team to discuss all things Suricata at the third edition of the annual Suricata Conference. SuriCon 2017 will be in November in Prague: https://suricon.net

About Suricata

Suricata is a high performance Network Threat Detection, IDS, IPS and Network Security Monitoring engine. Open Source and owned by a community run non-profit foundation, the Open Information Security Foundation (OISF). Suricata is developed by the OISF, its supporting vendors and the community.