Tag Archive | unix socket

Suricata 4.0.0-beta1 ready for testing!

suri-400x400

We are proud to announce that the first release for the upcoming Suricata 4.0.0-beta1 is ready for testing.

This release features our first experimental steps into using the Rust language for creating safer and easier to develop parsers. Inspired by Pierre Chiffliers talk at SuriCon 2016 (pdf). This initial integration does not yet include Pierre’s work, but this will likely change in the near future.
By compiling with –enable-rust you’ll get a basic NFSv3 parser and reimplementation of the DNS parser. Feedback on this is highly appreciated.

A major new feature is support for STARTTLS in SMTP and FTP. TLS sessions will now be logged in these cases. Decoding, logging and matching on TLS sertial numbers was also added. Great work by Mats Klepsland. Also for TLS, session resumption logging is now supported thanks to the work of Ray Ruvinskiy. TLS logging was improved by Paulo Pacheco.

Lots of new HTTP detection options were added to make matching on specific header fields easier and more efficient. New SSH keywords that are fast_pattern capable have also been added. For developers, this release makes extending the detection engine a lot easier.

A major TCP stream engine update is included. This should lead to better performance and less configuration, especially in IPS mode.

EVE is extended in several ways: in the case of encapsulated traffic both the inner and outer ip addresses and ports are logged. The ‘vars’ facility logs flowbits and other vars. This can also be used to extract data from the traffic using PCRE, and then log it. EVE can also be rotated based on time.

David Wharton has created a section in the documentation for rule writers who have a background in Snort. It documents changes that are relevant for writing rules.

Paulo Pacheco has been improving the Redis output performance.

Note that this release finally drops support for CentOS 5, and for libpcap 0.x with it.

Changes

  • Feature #805: Add support for applayer change
  • Feature #806: Implement STARTTLS support
  • Feature #1636: Signal rotation of unified2 log file without restart
  • Feature #1953: lua: expose flow_id
  • Feature #1969: TLS transactions with session resumption are not logged
  • Feature #1978: Using date in logs name
  • Feature #1998: eve.tls: custom TLS logging
  • Feature #2006: tls: decode certificate serial number
  • Feature #2011: eve.alert: print outside IP addresses on alerts on traffic inside tunnels
  • Feature #2046: Support custom file permissions per logger
  • Feature #2061: lua: get timestamps from flow
  • Feature #2077: Additional HTTP Header Contents and Negation
  • Feature #2129: nfs: parser, logger and detection
  • Feature #2130: dns: rust parser with stateless behaviour
  • Feature #2132: eve: flowbit and other vars logging
  • Feature #2133: unix socket: add/remove hostbits
  • Bug #1335: suricata option –pidfile overwrites any file
  • Bug #1470: make install-full can have race conditions on OSX.
  • Bug #1759: CentOS5 EOL tasks
  • Bug #2037: travis: move off legacy support
  • Bug #2039: suricata stops processing when http-log output via unix_stream backs up
  • Bug #2041: bad checksum 0xffff
  • Bug #2044: af-packet: faulty VLAN handling in tpacket-v3 mode
  • Bug #2045: geoip: compile warning on CentOS 7
  • Bug #2049: Empty rule files cause failure exit code without corresponding message
  • Bug #2051: ippair: xbit unset memory leak
  • Bug #2053: ippair: pair is direction sensitive
  • Bug #2070: file store: file log / file store mismatch with multiple files
  • Bug #2072: app-layer: fix memleak on bad traffic
  • Bug #2078: http body handling: failed assertion
  • Bug #2088: modbus: clang-4.0 compiler warnings
  • Bug #2093: Handle TCP stream gaps.
  • Bug #2097: “Name of device should not be null” appears in suricata.log when using pfring with configuration from suricata.yaml
  • Bug #2098: isdataat: fix parsing issue with leading spaces
  • Bug #2108: pfring: errors when compiled with asan/debug
  • Bug #2111: doc: links towards http_header_names
  • Bug #2112: doc: links towards certain http_ keywords not working
  • Bug #2113: Race condition starting Unix Server
  • Bug #2118: defrag – overlap issue in linux policy
  • Bug #2125: ASAN SEGV – Suricata version 4.0dev (rev 922a27e)
  • Optimization #521: Introduce per stream thread segment pool
  • Optimization #1873: Classtypes missing on decoder-events,files, and stream-events

Download

https://www.openinfosecfoundation.org/download/suricata-4.0.0-beta1.tar.gz

Special thanks

Mats Klepsland – for his major contributions: many EVE and TLS features

Pierre Chifflier – for paving the way for the Rust experiment and being very helpful while learning Rust and Nom.

Additionally: Jérémy Beaume, Alexander Gozman, Paulo Pacheco, Ray Ruvinskiy, Peter Sanders, David Wharton, Jon Zeolla

Trainings

SuriCon 2017

Come meet the Suricata community and development team to discuss all things Suricata at the third edition of the annual Suricata Conference. SuriCon 2017 will be in November in Prague: https://suricon.net

About Suricata

Suricata is a high performance Network Threat Detection, IDS, IPS and Network Security Monitoring engine. Open Source and owned by a community run non-profit foundation, the Open Information Security Foundation (OISF). Suricata is developed by the OISF, its supporting vendors and the community.

Suricata 3.2.2 available!

suri-400x400

We are pleased to announce Suricata 3.2.2. This release fixes a fairly small number of issues.

It also improves the unix-socket runmode by allowing both ‘single’ and ‘autofp’ runmodes to be specified.

Changes

  • Feature #1675: Support additional runmodes for unix-socket
  • Bug #2043: 3.2.x backport: make install-full can have race conditions on OSX.
  • Bug #2047: af-packet: faulty VLAN handling in tpacket-v3 mode (3.2.x)
  • Bug #2048: bad checksum 0xffff (3.2.x)
  • Bug #2052: ippair: xbit unset memory leak (3.2.x)
  • Bug #2071: file store: file log / file store mismatch with multiple files (3.2.x)
  • Bug #2073: app-layer: fix memleak on bad traffic (3.2.x)
  • Bug #2079: http body handling: failed assertion (3.2.x)
  • Bug #2085: ippair: pair is direction sensitive (3.2.x)
  • Bug #2119: 3.2.x – defrag – overlap issue in linux policy
  • Bug #2122: unix socket: race condition on start up (3.2.x)

Download

https://www.openinfosecfoundation.org/download/suricata-3.2.2.tar.gz

Special thanks

Jérémy Beaume, Alexander Gozman, Zoltan Herczeg and Jon Zeolla

Trainings

SuriCon 2017

Come meet the Suricata community and development team to discuss all things Suricata at the third edition of the annual Suricata Conference. SuriCon 2017 will be in November in Prague: https://suricon.net

About Suricata

Suricata is a high performance Network Threat Detection, IDS, IPS and Network Security Monitoring engine. Open Source and owned by a community run non-profit foundation, the Open Information Security Foundation (OISF). Suricata is developed by the OISF, its supporting vendors and the community.

Suricata 3.0RC3 Available!

Photo by Eric Leblond

We’re happy to announce Suricata 3.0RC3. RC3 fixes a few issues in RC2 that require some more testing. The plan is to release the stable quickly after the holidays, so please help us test this release!

Fixes:

  • Bug #1632: Fail to download large file with browser
  • Bug #1634: Fix non thread safeness of Prelude analyzer
  • Bug #1640: drop log crashes
  • Bug #1645: Race condition in unix manager
  • Bug #1647: FlowGetKey flow-hash.c:240 segmentation fault (master)
  • Bug #1650: DER parsing issue (master)

Get the release here:

http://www.openinfosecfoundation.org/download/suricata-3.0RC3.tar.gz

Known issues & missing features

In a development release like this things may not be as polished yet. So please handle with care. That said, if you encounter issues, please let us know! As always, we are doing our best to make you aware of continuing development and items within the engine that are not yet complete or optimal. With this in mind, please notice the list we have included of known items we are working on.  See issues for an up to date list and to report new issues. See Known_issues for a discussion and time line for the major issues.

About Suricata

Suricata is a high performance Network IDS, IPS and Network Security Monitoring engine. Open Source and owned by a community run non-profit foundation, the Open Information Security Foundation (OISF). Suricata is developed by the OISF, its supporting vendors and the community.

Suricata 2.0beta1 Available!

Photo by Eric Leblond

The OISF development team is proud to announce Suricata 2.0beta1. This is the first beta release for the upcoming 2.0 version.

This release greatly improved our HTTP handling by upgrading libhtp support to 0.5.5 and by redesigning transaction handling, which increases HTTP performance as well[1]. On the performance side, a large CUDA overhaul greatly improves our GPU performance[2]. Also new in this release is a DNS parser, logger and detection support.

Get the new release here: http://www.openinfosecfoundation.org/download/suricata-2.0beta1.tar.gz

[1] http://www.poona.me/2013/05/suricata-transaction-engine-re-designed.html#performance
[2] http://www.poona.me/2013/06/suricata-cuda-engine-re-designed.html#performance

New features

  • Luajit flow vars and flow ints support (#593)
  • DNS parser, logger and keyword support (#792), funded by Emerging Threats
  • deflate support for HTTP response bodies (#470, #775)

Improvements

  • update to libhtp 0.5 (#775)
  • improved gzip support for HTTP response bodies (#470, #775)
  • redesigned transaction handling, improving both accuracy and performance (#753)
  • redesigned CUDA support (#729)
  • Be sure to always apply verdict to NFQ packet (#769)
  • stream engine: SACK allocs should adhere to memcap (#794)
  • stream: deal with multiple different SYN/ACK’s better (#796)
  • stream: Randomize stream chunk size for raw stream inspection (#804)
  • Introduce per stream thread ssn pool (#519)
  • “pass” IP-only rules should bypass detection engine after matching (#718)
  • Generate error if bpf is used in IPS mode (#777)
  • Add support for batch verdicts in NFQ, thanks to Florian Westphal
  • Update Doxygen config, thanks to Phil Schroeder
  • Improve libnss detection, thanks to Christian Kreibich

Fixes

  • Fix a FP on rules looking for port 0 and fragments (#847), thanks to Rmkml
  • OS X unix socket build fixed (#830)
  • bytetest, bytejump and byteextract negative offset failure (#827)
  • Fix fast.log formatting issues (#771), thanks to Rmkml
  • Invalidate negative depth (#774), thanks to Rmkml
  • Fixed accuracy issues with relative pcre matching (#791)
  • Fix deadlock in flowvar capture code (#802)
  • Improved accuracy of file_data keyword (#817)
  • Fix af-packet ips mode rule processing bug (#819), thanks to Laszlo Madarassy
  • stream: fix injecting pseudo packet too soon leading to FP (#883), thanks to Francis Trudeau

Special thanks

We’d like to thank the following people and corporations for their contributions and feedback:

  • Rmkml
  • Laszlo Madarassy
  • Ken Steele, Tilera
  • Florian Westphal
  • Christian Kreibich
  • Francis Trudeau
  • Phil Schroeder
  • Ivan Ristic
  • Emerging Threats
  • Coverity

Known issues & missing features

In a beta release like this things may not be as polished yet. So please handle with care. That said, if you encounter issues, please let us know!

As always, we are doing our best to make you aware of continuing development and items within the engine that are not yet complete or optimal. With this in mind, please notice the list we have included of known items we are working on.

See issues for an up to date list and to report new issues. See Known_issues for a discussion and time line for the major issues.

About Suricata

Suricata is a high performance Network IDS, IPS and Network Security Monitoring engine. Open Source and owned by a community run non-profit foundation, the Open Information Security Foundation (OISF). Suricata is developed by the OISF, its supporting vendors and the community.

Suricata 1.4.4 released!

Photo by Eric LeblondThe OISF development team is pleased to announce Suricata 1.4.4. This is a small but important update over the 1.4.3 release, fixing some important bugs.

Get the new release here: suricata-1.4.4.tar.gz

Fixes

  • Bug #834: Unix socket – showing as compiled when it is not desired to do so
  • Bug #841: configure –enable-unix-socket does not err out if libs/pkgs are not present
  • Bug #846: FP on IP frag and sig using udp port 0, thanks to Rmkml
  • Bug #864: fix pass action not working correctly in all cases, thanks Kevin Branch
  • Bug #876: http connect tunnel crash fixed
  • Bug #877: Flowbit check with content doesn’t match consistently, thanks to Francis Trudeau

Special thanks

  • Rmkml
  • Francis Trudeau
  • Kevin Branch

Known issues & missing features

As always, we are doing our best to make you aware of continuing development and items within the engine that are not yet complete or optimal. With this in mind, please notice the list we have included of known items we are working on.

See issues for an up to date list and to report new issues. See Known_issues for a discussion and time line for the major issues.

About Suricata

Suricata is a high performance Network IDS, IPS and Network Security Monitoring engine. Open Source and owned by a community run non-profit foundation, the Open Information Security Foundation (OISF). Suricata is developed by the OISF, its supporting vendors and the community.

Suricata 1.4.3 released!

Photo by Eric LeblondThe OISF development team is pleased to announce Suricata 1.4.3. This is a small but important update over the 1.4.2 release, fixing some important bugs.

Get the new release here: suricata-1.4.3.tar.gz

Fixes

  • Fix missed detection in bytetest, bytejump and byteextract for negative offset (#828)
  • Fix IPS mode being unable to drop tunneled packets (#826)
  • Fix OS X Unix Socket build (#829)

Special thanks

  • Laszlo Madarassy
  • Will Metcalf

Known issues & missing features

As always, we are doing our best to make you aware of continuing development and items within the engine that are not yet complete or optimal. With this in mind, please notice the list we have included of known items we are working on.

See issues for an up to date list and to report new issues. See Known_issues for a discussion and time line for the major issues.

About Suricata

Suricata is a high performance Network IDS, IPS and Network Security Monitoring engine. Open Source and owned by a community run non-profit foundation, the Open Information Security Foundation (OISF). Suricata is developed by the OISF, its supporting vendors and the community.

Suricata 1.4.1 released!

Photo by Eric LeblondThe OISF development team is proud to announce Suricata 1.4.1. This is a major update over the 1.4 release, adding some exiting features, many improvements and fixing some important bugs.

Get the new release here: suricata-1.4.1.tar.gz

The most interesting new feature is the GeoIP support. Great contribution by Ignacio Sanchez. It adds “geoip” rule keyword that allows you to match on source of destination of a packet per country.

New features

  • GeoIP keyword, allowing matching on Maxmind’s database, contributed by Ignacio Sanchez (#559)
  • Introduce http_host and http_raw_host keywords (#733, #743)
  • Add python module for interacting with unix socket (#767)
  • Add new unix socket commands: fetching config, counters, basic runtime info (#764, #765)

Improvements

  • Big Napatech support update by Matt Keeler
  • Configurable sensor id in unified2 output, contributed by Jake Gionet (#667)
  • FreeBSD IPFW fixes by Nikolay Denev
  • Add “default” interface setting to capture configuration in yaml (#679)
  • Make sure “snaplen” can be set by the user (#680)
  • Improve HTTP URI query string normalization (#739)
  • Improved error reporting in MD5 loading (#693)
  • Improve reference.config parser error reporting (#737)
  • Improve build info output to include all configure options (#738)

Fixes

  • Segfault in TLS parsing reported by Charles Smutz (#725)
  • Fix crash in teredo decoding, reported by Rmkml (#736)
  • fixed UDPv4 packets without checksum being detected as invalid (#760)
  • fixed DCE/SMB parsers getting confused in some fragmented cases (#764)
  • parsing ipv6 address/subnet parsing in thresholding was fixed by Jamie Strandboge (#697)
  • FN: IP-only rule ip_proto not matching for some protocols (#689)
  • Fix build failure with other libhtp installs (#688)
  • Fix malformed yaml loading leading to a crash (#694)
  • Various Mac OS X fixes (#700, #701, #703)
  • Fix for autotools on Mac OS X by Jason Ish (#704)
  • Fix AF_PACKET under high load not updating stats (#706)

Special thanks

  • Ignacio Sanchez
  • Matt Keeler — nPulse
  • Jake Gionet
  • Nikolay Denev
  • Jason Ish — Endace
  • Jamie Strandboge
  • Charles Smutz
  • Rmkml

Known issues & missing features

As always, we are doing our best to make you aware of continuing development and items within the engine that are not yet complete or optimal. With this in mind, please notice the list we have included of known items we are working on.

See issues for an up to date list and to report new issues. See Known_issues for a discussion and time line for the major issues.

About Suricata

Suricata is a high performance Network IDS, IPS and Network Security Monitoring engine. Open Source and owned by a community run non-profit foundation, the Open Information Security Foundation (OISF). Suricata is developed by the OISF, its supporting vendors and the community.

Suricata 1.4 released!

Photo by Eric LeblondThe OISF development team is proud to announce Suricata 1.4. This release is a major improvement over the previous releases with regard to performance, scalability and accuracy. Also, a number of great features have been added.

Get the new release here: suricata-1.4.tar.gz

The biggest new features of this release are the Unix Socket support, IP Reputation support and the addition of the Luajit keyword. Each of these new features are still in active development, and should be approached with some care.

The 1.4 release improves performance and scalability a lot. The IP Defrag engine was rewritten to scale better, various packet acquisition methods were improved and various parts of the detection engine were optimized further.

The configuration file has evolved but backward compatibility is provided. We thus encourage you to update your suricata configuration file. Upgrade guidance is provided here: Upgrading_Suricata_13_to_Suricata_14

New features

  • Unix socket mode for batched processing of series of pcap (#571, #552) (experimental)
  • Interaction with Suricata via uix socket (#571, #552) (experimental)
  • IP Reputation: loading and matching (#647) (experimental)
  • New keyword: “luajit” to inspect packet, payload and all HTTP buffers with a Lua script (#346) (experimental)
  • Delayed detect initialization. Starts processing packets right away and loads detection engine in the background (#522)
  • Support for pkt_data keyword was added (#423)
  • Improved –list-keywords commandline option gives detailed info for supported keyword, including doc link (#435)
  • User and group to run as can now be set in the config file
  • Add stream event to match on overlaps with different data in stream reassembly (#603)
  • Decoding of IPv4-in-IPv6, IPv6-in-IPv6 and Teredo tunnels (#462, #514, #480)
  • Rules can be set to inspect only IPv4 or IPv6 (#494)
  • Added ability to control per server HTTP parser settings in much more detail (#503)
  • Make HTTP request and response body inspection sizes configurable per HTTP server config (#560)
  • Filesize keyword for matching on sizes of files in HTTP (#489)
  • Custom HTTP logging contributed by Ignacio Sanchez (#530)
  • TLS certificate logging and fingerprint computation and keyword by Jean-Paul Roliers (#443)
  • TLS certificate store to disk feature Jean-Paul Roliers (#444)
  • AF_PACKET IPS support (#516)
  • NFQ fail open support (#507)
  • PCAP/AF_PACKET/PF_RING packet stats are now printed in stats.log (#561, #625)
  • Support for Napatech cards through their 3rd generation driver was added by Matt Keeler from Npulse (#430, #619)
  • Endace support improved
  • New runmode for users of pcap wrappers (Myricom, PF_RING, others)

Improvements

  • Add contrib directory to the dist (#567)
  • Performance improvements to signatures with dsize option
  • Improved rule analyzer: print fast_pattern along with the rule (#558)
  • Fixes to stream engine reducing the number of events generated (#604)
  • Stream.inline option new defaults to “auto”, meaning enabled in IPS mode, disabled in IDS mode (#592)
  • HTTP handling in OOM condition was greatly improved (#557)
  • Filemagic keyword performance was improved (#585)
  • Updated bundled libhtp to 0.2.11
  • Build system improvements and cleanups
  • Live reloads now supports HTTP rule updates better (#522)
  • AF_PACKET performance improvements (#197, #415)
  • Make defrag more configurable (#517, #528)
  • Improve pool performance (#518)
  • Improve file inspection keywords by adding a separate API (#531)
  • Example threshold.config file provided (#302)

Changes since 1.4rc1

  • Decoder event matching fixed (#672)
  • Unified2 would overwrite files if file rotation happened within a second of file creation, leading to loss of events/alerts (#665)
  • Add more events to IPv6 extension header anomolies (#678)
  • Fix ICMPv6 payload and checksum calculation (#677, #674)
  • Clean up flow timeout handling (#656)
  • Fix a shutdown bug when using AF_PACKET under high load (#653)
  • Fix TCP sessions being cleaned up to early (#652)

Credits

  • Jason Ish — Endace
  • Ludovico Cavedon — Lastline
  • Last G
  • Matt Keeler — Npulse
  • Chris Wakelin
  • Will Metcalf
  • Ivan Ristic
  • Kyle Creyts
  • Michael Hoffrath
  • Rmkml
  • Jean-Paul Roliers
  • Ignacio Sanchez
  • Michel Saborde
  • Simon Moon
  • Coverity

Known issues & missing features

As always, we are doing our best to make you aware of continuing development and items within the engine that are not yet complete or optimal. With this in mind, please notice the list we have included of known items we are working on.

See issues for an up to date list and to report new issues. See Known_issues for a discussion and time line for the major issues.

About Suricata

Suricata is a high performance Network IDS, IPS and Network Security Monitoring engine. Open Source and owned by a community run non-profit foundation, the Open Information Security Foundation (OISF). Suricata is developed by the OISF, its supporting vendors and the community.

Suricata 1.4rc1 Available!

Photo by Eric LeblondThe OISF development team is proud to announce Suricata 1.4rc1, the first (and hopefully only) release candidate for the upcoming 1.4 version.

This release adds two major new features: a unix socket command mode, allowing for easy processing of large numbers of pcap files, and IP reputation. Both features are considered experimental.

Get the new release here: suricata-1.4rc1.tar.gz

New features

  • Interactive unix socket mode (#571, #552)
  • IP Reputation: loading and matching (#647)
  • Improved –list-keywords commandline option gives detailed info for supported keyword, including doc link (#435)

Improvements

  • Rule analyzer improvement wrt ipv4/ipv6, invalid rules (#494)
  • User-Agent added to file log and filestore meta files (#629)
  • Endace DAG supports live stats and at exit drop stats (#638)
  • Add support for libhtp event “request port doesn’t match tcp port” (#650)

Fixes

  • Rules with negated addresses will not be considered IP-only (#599)
  • Rule reloads complete much faster in low traffic conditions (#526)
  • Suricata -h now displays all available options (#419)
  • Luajit configure time detection was improved (#636)
  • Flow manager mutex used w/o initialization (#628)
  • Cygwin work around for windows shell mangling interface string (#372)
  • Fix a Prelude output crash with alerts generated by rules w/o classtype or msg (#648)
  • CLANG compiler build fixes (#649)
  • Several fixes found by code analyzers

Credits

We’d like to thank the following people and corporations for their contributions and feedback:

  • Jason Ish — Endace
  • Ludovico Cavedon — Lastline
  • Last G

Known issues & missing features

This is a “release candidate”-quality release so the stability should be good although unexpected corner cases might happen. If you encounter one, please let us know!

As always, we are doing our best to make you aware of continuing development and items within the engine that are not yet complete or optimal. With this in mind, please notice the list we have included of known items we are working on.

See issues for an up to date list and to report new issues. See Known_issues for a discussion and time line for the major issues.

About Suricata

Suricata is a high performance Network IDS, IPS and Network Security Monitoring engine. Open Source and owned by a community run non-profit foundation, the Open Information Security Foundation (OISF). Suricata is developed by the OISF, its supporting vendors and the community.