Tag Archive | luajit

Suricata 2.0beta1 Available!

Photo by Eric Leblond

The OISF development team is proud to announce Suricata 2.0beta1. This is the first beta release for the upcoming 2.0 version.

This release greatly improved our HTTP handling by upgrading libhtp support to 0.5.5 and by redesigning transaction handling, which increases HTTP performance as well[1]. On the performance side, a large CUDA overhaul greatly improves our GPU performance[2]. Also new in this release is a DNS parser, logger and detection support.

Get the new release here: http://www.openinfosecfoundation.org/download/suricata-2.0beta1.tar.gz

[1] http://www.poona.me/2013/05/suricata-transaction-engine-re-designed.html#performance
[2] http://www.poona.me/2013/06/suricata-cuda-engine-re-designed.html#performance

New features

  • Luajit flow vars and flow ints support (#593)
  • DNS parser, logger and keyword support (#792), funded by Emerging Threats
  • deflate support for HTTP response bodies (#470, #775)

Improvements

  • update to libhtp 0.5 (#775)
  • improved gzip support for HTTP response bodies (#470, #775)
  • redesigned transaction handling, improving both accuracy and performance (#753)
  • redesigned CUDA support (#729)
  • Be sure to always apply verdict to NFQ packet (#769)
  • stream engine: SACK allocs should adhere to memcap (#794)
  • stream: deal with multiple different SYN/ACK’s better (#796)
  • stream: Randomize stream chunk size for raw stream inspection (#804)
  • Introduce per stream thread ssn pool (#519)
  • “pass” IP-only rules should bypass detection engine after matching (#718)
  • Generate error if bpf is used in IPS mode (#777)
  • Add support for batch verdicts in NFQ, thanks to Florian Westphal
  • Update Doxygen config, thanks to Phil Schroeder
  • Improve libnss detection, thanks to Christian Kreibich

Fixes

  • Fix a FP on rules looking for port 0 and fragments (#847), thanks to Rmkml
  • OS X unix socket build fixed (#830)
  • bytetest, bytejump and byteextract negative offset failure (#827)
  • Fix fast.log formatting issues (#771), thanks to Rmkml
  • Invalidate negative depth (#774), thanks to Rmkml
  • Fixed accuracy issues with relative pcre matching (#791)
  • Fix deadlock in flowvar capture code (#802)
  • Improved accuracy of file_data keyword (#817)
  • Fix af-packet ips mode rule processing bug (#819), thanks to Laszlo Madarassy
  • stream: fix injecting pseudo packet too soon leading to FP (#883), thanks to Francis Trudeau

Special thanks

We’d like to thank the following people and corporations for their contributions and feedback:

  • Rmkml
  • Laszlo Madarassy
  • Ken Steele, Tilera
  • Florian Westphal
  • Christian Kreibich
  • Francis Trudeau
  • Phil Schroeder
  • Ivan Ristic
  • Emerging Threats
  • Coverity

Known issues & missing features

In a beta release like this things may not be as polished yet. So please handle with care. That said, if you encounter issues, please let us know!

As always, we are doing our best to make you aware of continuing development and items within the engine that are not yet complete or optimal. With this in mind, please notice the list we have included of known items we are working on.

See issues for an up to date list and to report new issues. See Known_issues for a discussion and time line for the major issues.

About Suricata

Suricata is a high performance Network IDS, IPS and Network Security Monitoring engine. Open Source and owned by a community run non-profit foundation, the Open Information Security Foundation (OISF). Suricata is developed by the OISF, its supporting vendors and the community.

Suricata 1.4 released!

Photo by Eric LeblondThe OISF development team is proud to announce Suricata 1.4. This release is a major improvement over the previous releases with regard to performance, scalability and accuracy. Also, a number of great features have been added.

Get the new release here: suricata-1.4.tar.gz

The biggest new features of this release are the Unix Socket support, IP Reputation support and the addition of the Luajit keyword. Each of these new features are still in active development, and should be approached with some care.

The 1.4 release improves performance and scalability a lot. The IP Defrag engine was rewritten to scale better, various packet acquisition methods were improved and various parts of the detection engine were optimized further.

The configuration file has evolved but backward compatibility is provided. We thus encourage you to update your suricata configuration file. Upgrade guidance is provided here: Upgrading_Suricata_13_to_Suricata_14

New features

  • Unix socket mode for batched processing of series of pcap (#571, #552) (experimental)
  • Interaction with Suricata via uix socket (#571, #552) (experimental)
  • IP Reputation: loading and matching (#647) (experimental)
  • New keyword: “luajit” to inspect packet, payload and all HTTP buffers with a Lua script (#346) (experimental)
  • Delayed detect initialization. Starts processing packets right away and loads detection engine in the background (#522)
  • Support for pkt_data keyword was added (#423)
  • Improved –list-keywords commandline option gives detailed info for supported keyword, including doc link (#435)
  • User and group to run as can now be set in the config file
  • Add stream event to match on overlaps with different data in stream reassembly (#603)
  • Decoding of IPv4-in-IPv6, IPv6-in-IPv6 and Teredo tunnels (#462, #514, #480)
  • Rules can be set to inspect only IPv4 or IPv6 (#494)
  • Added ability to control per server HTTP parser settings in much more detail (#503)
  • Make HTTP request and response body inspection sizes configurable per HTTP server config (#560)
  • Filesize keyword for matching on sizes of files in HTTP (#489)
  • Custom HTTP logging contributed by Ignacio Sanchez (#530)
  • TLS certificate logging and fingerprint computation and keyword by Jean-Paul Roliers (#443)
  • TLS certificate store to disk feature Jean-Paul Roliers (#444)
  • AF_PACKET IPS support (#516)
  • NFQ fail open support (#507)
  • PCAP/AF_PACKET/PF_RING packet stats are now printed in stats.log (#561, #625)
  • Support for Napatech cards through their 3rd generation driver was added by Matt Keeler from Npulse (#430, #619)
  • Endace support improved
  • New runmode for users of pcap wrappers (Myricom, PF_RING, others)

Improvements

  • Add contrib directory to the dist (#567)
  • Performance improvements to signatures with dsize option
  • Improved rule analyzer: print fast_pattern along with the rule (#558)
  • Fixes to stream engine reducing the number of events generated (#604)
  • Stream.inline option new defaults to “auto”, meaning enabled in IPS mode, disabled in IDS mode (#592)
  • HTTP handling in OOM condition was greatly improved (#557)
  • Filemagic keyword performance was improved (#585)
  • Updated bundled libhtp to 0.2.11
  • Build system improvements and cleanups
  • Live reloads now supports HTTP rule updates better (#522)
  • AF_PACKET performance improvements (#197, #415)
  • Make defrag more configurable (#517, #528)
  • Improve pool performance (#518)
  • Improve file inspection keywords by adding a separate API (#531)
  • Example threshold.config file provided (#302)

Changes since 1.4rc1

  • Decoder event matching fixed (#672)
  • Unified2 would overwrite files if file rotation happened within a second of file creation, leading to loss of events/alerts (#665)
  • Add more events to IPv6 extension header anomolies (#678)
  • Fix ICMPv6 payload and checksum calculation (#677, #674)
  • Clean up flow timeout handling (#656)
  • Fix a shutdown bug when using AF_PACKET under high load (#653)
  • Fix TCP sessions being cleaned up to early (#652)

Credits

  • Jason Ish — Endace
  • Ludovico Cavedon — Lastline
  • Last G
  • Matt Keeler — Npulse
  • Chris Wakelin
  • Will Metcalf
  • Ivan Ristic
  • Kyle Creyts
  • Michael Hoffrath
  • Rmkml
  • Jean-Paul Roliers
  • Ignacio Sanchez
  • Michel Saborde
  • Simon Moon
  • Coverity

Known issues & missing features

As always, we are doing our best to make you aware of continuing development and items within the engine that are not yet complete or optimal. With this in mind, please notice the list we have included of known items we are working on.

See issues for an up to date list and to report new issues. See Known_issues for a discussion and time line for the major issues.

About Suricata

Suricata is a high performance Network IDS, IPS and Network Security Monitoring engine. Open Source and owned by a community run non-profit foundation, the Open Information Security Foundation (OISF). Suricata is developed by the OISF, its supporting vendors and the community.

Suricata 1.4rc1 Available!

Photo by Eric LeblondThe OISF development team is proud to announce Suricata 1.4rc1, the first (and hopefully only) release candidate for the upcoming 1.4 version.

This release adds two major new features: a unix socket command mode, allowing for easy processing of large numbers of pcap files, and IP reputation. Both features are considered experimental.

Get the new release here: suricata-1.4rc1.tar.gz

New features

  • Interactive unix socket mode (#571, #552)
  • IP Reputation: loading and matching (#647)
  • Improved –list-keywords commandline option gives detailed info for supported keyword, including doc link (#435)

Improvements

  • Rule analyzer improvement wrt ipv4/ipv6, invalid rules (#494)
  • User-Agent added to file log and filestore meta files (#629)
  • Endace DAG supports live stats and at exit drop stats (#638)
  • Add support for libhtp event “request port doesn’t match tcp port” (#650)

Fixes

  • Rules with negated addresses will not be considered IP-only (#599)
  • Rule reloads complete much faster in low traffic conditions (#526)
  • Suricata -h now displays all available options (#419)
  • Luajit configure time detection was improved (#636)
  • Flow manager mutex used w/o initialization (#628)
  • Cygwin work around for windows shell mangling interface string (#372)
  • Fix a Prelude output crash with alerts generated by rules w/o classtype or msg (#648)
  • CLANG compiler build fixes (#649)
  • Several fixes found by code analyzers

Credits

We’d like to thank the following people and corporations for their contributions and feedback:

  • Jason Ish — Endace
  • Ludovico Cavedon — Lastline
  • Last G

Known issues & missing features

This is a “release candidate”-quality release so the stability should be good although unexpected corner cases might happen. If you encounter one, please let us know!

As always, we are doing our best to make you aware of continuing development and items within the engine that are not yet complete or optimal. With this in mind, please notice the list we have included of known items we are working on.

See issues for an up to date list and to report new issues. See Known_issues for a discussion and time line for the major issues.

About Suricata

Suricata is a high performance Network IDS, IPS and Network Security Monitoring engine. Open Source and owned by a community run non-profit foundation, the Open Information Security Foundation (OISF). Suricata is developed by the OISF, its supporting vendors and the community.

Suricata 1.4beta2 Available for testing!

Photo by Eric LeblondThe OISF development team is proud to announce Suricata 1.4beta2. This is the second beta release for the upcoming 1.4 version.

The main addition of this release is a usable lua scripting keyword for detection: luajit. This keyword allows you to run Lua scripts as part of the detection engine, allowing inspection beyond what the rule language offers. While not cheap, performance is not bad at all due to use of the luajit engine.

This release also brings major performance enhancements. We’re able to get virtually packet loss free with AF_PACKET on our ISP test box with 6gbps-9gpbs of sustained traffic on commodity hardware with 7k rules.

Get the new release here: suricata-1.4beta2.tar.gz

New features

  • New keyword: “luajit” to inspect packet, payload and all HTTP buffers with a Lua script (#346)
  • Added ability to control per server HTTP parser settings in much more detail (#503)

Improvements

  • Rewrite of IP Defrag engine to improve performance and fix locking logic (#512, #540)
  • Big performance improvement in inspecting decoder, stream and app layer events (#555)
  • Pool performance improvements (#541)
  • Improved performance of signatures with simple pattern setups (#577)
  • Bundled docs are installed upon make install (#527)
  • Support for a number of global vs rule thresholds was added (#425)
  • Improved rule profiling performance
  • If not explicit fast_pattern is set, pick HTTP patterns over stream patterns. HTTP method, stat code and stat msg are excluded.

Fixes

  • Fix compilation on architectures other than x86 and x86_64 (#572)
  • Fix FP with anchored pcre combined with relative matching (#529)
  • Fix engine hanging instead of exitting if the pcap device doesn’t exist (#533)
  • Work around for potential FP, will get properly fixed in next release (#574)
  • Improve ERF handling. Thanks to Jason Ish
  • Always set cluster_id in PF_RING
  • IPFW: fix broken broadcast handling
  • AF_PACKET kernel offset issue, IPS fix and cleanup
  • Fix stream engine sometimes resending the same data to app layer
  • Fix multiple issues in HTTP multipart parsing
  • Fixed a lockup at shutdown with NFQ (#537)

Credits

We’d like to thank the following people and corporations for their contributions and feedback:

  • Jason Ish – Endace
  • Chris Wakelin
  • Rmkml

Known issues & missing features

In a beta release like this things may not be as polished yet. So please handle with care. That said, if you encounter issues, please let us know! As always, we are doing our best to make you aware of continuing development and items within the engine that are not yet complete or optimal.  With this in mind, please notice the list we have included of known items we are working on.

See issues for an up to date list and to report new issues. See Known_issues for a discussion and time line for the major issues.

About Suricata

Suricata is a high performance Network IDS, IPS and Network Security Monitoring engine. Open Source and owned by a community run non-profit foundation, the Open Information Security Foundation (OISF). Suricata is developed by the OISF, its supporting vendors and the community.

Suricata 1.4beta1 released

The OISF development team is proud to announce Suricata 1.4beta1. This is the first beta release for the upcoming 1.4 version. It is the result of major effort by the OISF team with significant help from community contributors Ignacio Sanchez and Simon Moon.

Get the new release here: suricata-1.4beta1.tar.gz

New features

– Custom HTTP logging contributed by Ignacio Sanchez (#530)
– TLS certificate logging and fingerprint computation and keyword by Jean-Paul Roliers (#443)
– TLS certificate store to disk feature Jean-Paul Roliers (#444)
– Decoding of IPv4-in-IPv6, IPv6-in-IPv6 and Teredo tunnels (#462, #514, #480)
– AF_PACKET IPS support (#516)
– Rules can be set to inspect only IPv4 or IPv6 (#494)
– filesize keyword for matching on sizes of files in HTTP (#489)
– Delayed detect initialization. Starts processing packets right away and loads detection engine in the background (#522)
– NFQ fail open support (#507)
– Highly experimental lua scripting support for detection

Improvements

– Live reloads now supports HTTP rule updates better (#522)
– AF_PACKET performance improvements (#197, #415)
– Make defrag more configurable (#517, #528)
– Improve pool performance (#518)
– Improve file inspection keywords by adding a separate API (#531)
– Example threshold.config file provided (#302)

Fixes

– Fix building of perf profiling code on i386 platform. By Simon Moon (#534)
– Various spelling corrections by Simon Moon (#533)

Credits

We’d like to thank the following people and corporations for their contributions and feedback:

Jean-Paul Roliers
Ignacio Sanchez
Michel Saborde
Simon Moon
Coverity

Known issues & missing features

In a beta release like this things may not be as polished yet. So please handle with care. That said, if you encounter issues, please let us know! As always, we are doing our best to make you aware of continuing development and items within the engine that are not yet complete or optimal.  With this in mind, please notice the list we have included of known items we are working on.

See Issues for an up to date list and to report new issues. See Known issues for a discussion and time line for the major issues.