Tag Archive | napatech

Call for testing: announcing Suricata 5.0.0-beta1

We’re happy to present the first beta in the upcoming Suricata 5.0 series. In 5.0 we’re making a couple of large changes.

Rust

The most visible is that our Rust support is no longer optional. We’re convinced that Rust is a perfect match for Suricata, and we plan to increase its footprint in our code base steadily. By making it mandatory we’re able to remove parallel implementations and focus fully on making the Rust code better.

Protocol Detection

The protocol detection engine has been extended to provide better accuracy as well as support for dealing with asynchronous flows. These async flows are sometimes picked up in the wrong direction and the protocol detection engine can now reverse them.

Decoder Anomaly records in EVE

A new log record type has been added: ‘anomaly’. This logs the stream and decoder events that are set by the packet decoders. This is inspired by Zeeks (Bro) ‘weird’ log.

EVE improvements

VLAN and capture interface is now part of many more EVE records, even if they are flow records or records based on flow time out.

An option to log all HTTP headers to the EVE http records has been added.

Packet Capture

Netmap support has been rewritten so the more advanced features of netmap, such as vale switches, can be used now.

Napatech usability has been improved.

Rule language: Sticky Buffers (in progress)

As discussed at the Suricon 2018 brainstorm session, a new rule keyword scheme is being introduced. It takes the existing ‘sticky buffer’ approach with new keyword names to avoid confusion. The new scheme is <proto>.<buffer>, so for example ‘http.uri’ for the URI inspection.

A number of HTTP keywords have been added.

Unified Lua inspection mixed with the sticky buffers has also been implemented.

Python 3

With Python 2’s EOL approaching, we’ve made sure that all Suricata’s python code is Python 3 compliant.

Removals

Following our deprecation policy, we have removed the following parts: the plain text dns.log, the old files-json.log and support for the Tilera architecture.

https://suricata-ids.org/about/deprecation-policy/

Many more things

https://redmine.openinfosecfoundation.org/versions/115

Time line

We’re planning the first release candidate in about a month, with the final about a month later. So early July.

Get involved

If you’re interested in helping out, we’d be happy to accept patches, documentation, test reports and other kind of feedback.

Download from:

https://www.openinfosecfoundation.org/downloads/suricata-5.0.0-beta1.tar.gz

5-Day Suricata Developer Training

A short time ago we announced the first edition of the Suricata Developer training. I thought it be a good time to explain what we have in mind.

First, we’re planning to make this an annual event. I’m very excited about this. It should be great fun to have a week of development related discussions. I’m sure we can all learn a great deal, and share lots of ideas. Of course the social part is a nice addition.

Also, this is a great “stick behind the door” (as we’d say in Dutch) to finally get some much needed dev docs done, including architecture overview diagrams, etc.
What will we be teaching:

  1. General Suricata development basics: everything from git, how to QA, unittests, debugging, etc.
  2. Architecture overview
    1. API’s
    2. threading
    3. packets, flows, detection and output
  3. Extending Suricata — the beef of the training:
    1. packet decoders and detection plugins
    2. app layer protocol detection, parsing, state keeping
    3. app layer detection engine integration
    4. adding logging modules

Structure

Each day will start with lectures on each of the topics. You will get an overview of the API, learn about performance aspects, how threading comes into play, etc.

After this there will be assignments/challenges to apply the newly learned skills. This should be very interactive with lots of room for questions and discussions. We’ll be providing various assignments for multiple skill levels.

Teachers

From OISF we will have lead developer Victor Julien, core developers Eric Leblond and Jason Ish in the room to give the lectures and help answer questions. Also present to assist in general Suricata related questions: Matt Jonkman and Peter Manev.

Participants

So who is this for? We target people who what to learn how to extend Suricata. If you want to add protocols, detection options or maybe new output methods, then this is for you. Or maybe you want to be a ‘core’ developer on Suricata. Then this would be an excellent start of getting into it.

Skill-set:
– advanced C experience
– Linux Experience
– Network / Security Experience
– Basic Suricata End User Experience
=> for testing your code

If you’re not yet experienced with running Suricata, we would like to suggest attending a user training first. If you book your dev seat first hit us up for discount code on one of the training sessions. We’re planning one in Europe before summer, exact date and location is to be announced.

Money

We’ll be charging for the trainings. The revenue is used to cover the cost of the event itself (travel, hotels, etc). Whats left goes into the foundations general development budget. So by attending the training you will support Suricata’s development.

Location

This first edition is generously hosted by Napatech in Copenhagen, Denmark.
If you want to book, please do so through this Eventbrite link:
https://www.eventbrite.com/e/5-day-suricata-developer-training-in-copenhagen-denmark-open-to-the-public-tickets-15667305332
Questions and general feedback and thoughts are welcome!

Suricata 1.4.1 released!

Photo by Eric LeblondThe OISF development team is proud to announce Suricata 1.4.1. This is a major update over the 1.4 release, adding some exiting features, many improvements and fixing some important bugs.

Get the new release here: suricata-1.4.1.tar.gz

The most interesting new feature is the GeoIP support. Great contribution by Ignacio Sanchez. It adds “geoip” rule keyword that allows you to match on source of destination of a packet per country.

New features

  • GeoIP keyword, allowing matching on Maxmind’s database, contributed by Ignacio Sanchez (#559)
  • Introduce http_host and http_raw_host keywords (#733, #743)
  • Add python module for interacting with unix socket (#767)
  • Add new unix socket commands: fetching config, counters, basic runtime info (#764, #765)

Improvements

  • Big Napatech support update by Matt Keeler
  • Configurable sensor id in unified2 output, contributed by Jake Gionet (#667)
  • FreeBSD IPFW fixes by Nikolay Denev
  • Add “default” interface setting to capture configuration in yaml (#679)
  • Make sure “snaplen” can be set by the user (#680)
  • Improve HTTP URI query string normalization (#739)
  • Improved error reporting in MD5 loading (#693)
  • Improve reference.config parser error reporting (#737)
  • Improve build info output to include all configure options (#738)

Fixes

  • Segfault in TLS parsing reported by Charles Smutz (#725)
  • Fix crash in teredo decoding, reported by Rmkml (#736)
  • fixed UDPv4 packets without checksum being detected as invalid (#760)
  • fixed DCE/SMB parsers getting confused in some fragmented cases (#764)
  • parsing ipv6 address/subnet parsing in thresholding was fixed by Jamie Strandboge (#697)
  • FN: IP-only rule ip_proto not matching for some protocols (#689)
  • Fix build failure with other libhtp installs (#688)
  • Fix malformed yaml loading leading to a crash (#694)
  • Various Mac OS X fixes (#700, #701, #703)
  • Fix for autotools on Mac OS X by Jason Ish (#704)
  • Fix AF_PACKET under high load not updating stats (#706)

Special thanks

  • Ignacio Sanchez
  • Matt Keeler — nPulse
  • Jake Gionet
  • Nikolay Denev
  • Jason Ish — Endace
  • Jamie Strandboge
  • Charles Smutz
  • Rmkml

Known issues & missing features

As always, we are doing our best to make you aware of continuing development and items within the engine that are not yet complete or optimal. With this in mind, please notice the list we have included of known items we are working on.

See issues for an up to date list and to report new issues. See Known_issues for a discussion and time line for the major issues.

About Suricata

Suricata is a high performance Network IDS, IPS and Network Security Monitoring engine. Open Source and owned by a community run non-profit foundation, the Open Information Security Foundation (OISF). Suricata is developed by the OISF, its supporting vendors and the community.

Suricata 1.4 released!

Photo by Eric LeblondThe OISF development team is proud to announce Suricata 1.4. This release is a major improvement over the previous releases with regard to performance, scalability and accuracy. Also, a number of great features have been added.

Get the new release here: suricata-1.4.tar.gz

The biggest new features of this release are the Unix Socket support, IP Reputation support and the addition of the Luajit keyword. Each of these new features are still in active development, and should be approached with some care.

The 1.4 release improves performance and scalability a lot. The IP Defrag engine was rewritten to scale better, various packet acquisition methods were improved and various parts of the detection engine were optimized further.

The configuration file has evolved but backward compatibility is provided. We thus encourage you to update your suricata configuration file. Upgrade guidance is provided here: Upgrading_Suricata_13_to_Suricata_14

New features

  • Unix socket mode for batched processing of series of pcap (#571, #552) (experimental)
  • Interaction with Suricata via uix socket (#571, #552) (experimental)
  • IP Reputation: loading and matching (#647) (experimental)
  • New keyword: “luajit” to inspect packet, payload and all HTTP buffers with a Lua script (#346) (experimental)
  • Delayed detect initialization. Starts processing packets right away and loads detection engine in the background (#522)
  • Support for pkt_data keyword was added (#423)
  • Improved –list-keywords commandline option gives detailed info for supported keyword, including doc link (#435)
  • User and group to run as can now be set in the config file
  • Add stream event to match on overlaps with different data in stream reassembly (#603)
  • Decoding of IPv4-in-IPv6, IPv6-in-IPv6 and Teredo tunnels (#462, #514, #480)
  • Rules can be set to inspect only IPv4 or IPv6 (#494)
  • Added ability to control per server HTTP parser settings in much more detail (#503)
  • Make HTTP request and response body inspection sizes configurable per HTTP server config (#560)
  • Filesize keyword for matching on sizes of files in HTTP (#489)
  • Custom HTTP logging contributed by Ignacio Sanchez (#530)
  • TLS certificate logging and fingerprint computation and keyword by Jean-Paul Roliers (#443)
  • TLS certificate store to disk feature Jean-Paul Roliers (#444)
  • AF_PACKET IPS support (#516)
  • NFQ fail open support (#507)
  • PCAP/AF_PACKET/PF_RING packet stats are now printed in stats.log (#561, #625)
  • Support for Napatech cards through their 3rd generation driver was added by Matt Keeler from Npulse (#430, #619)
  • Endace support improved
  • New runmode for users of pcap wrappers (Myricom, PF_RING, others)

Improvements

  • Add contrib directory to the dist (#567)
  • Performance improvements to signatures with dsize option
  • Improved rule analyzer: print fast_pattern along with the rule (#558)
  • Fixes to stream engine reducing the number of events generated (#604)
  • Stream.inline option new defaults to “auto”, meaning enabled in IPS mode, disabled in IDS mode (#592)
  • HTTP handling in OOM condition was greatly improved (#557)
  • Filemagic keyword performance was improved (#585)
  • Updated bundled libhtp to 0.2.11
  • Build system improvements and cleanups
  • Live reloads now supports HTTP rule updates better (#522)
  • AF_PACKET performance improvements (#197, #415)
  • Make defrag more configurable (#517, #528)
  • Improve pool performance (#518)
  • Improve file inspection keywords by adding a separate API (#531)
  • Example threshold.config file provided (#302)

Changes since 1.4rc1

  • Decoder event matching fixed (#672)
  • Unified2 would overwrite files if file rotation happened within a second of file creation, leading to loss of events/alerts (#665)
  • Add more events to IPv6 extension header anomolies (#678)
  • Fix ICMPv6 payload and checksum calculation (#677, #674)
  • Clean up flow timeout handling (#656)
  • Fix a shutdown bug when using AF_PACKET under high load (#653)
  • Fix TCP sessions being cleaned up to early (#652)

Credits

  • Jason Ish — Endace
  • Ludovico Cavedon — Lastline
  • Last G
  • Matt Keeler — Npulse
  • Chris Wakelin
  • Will Metcalf
  • Ivan Ristic
  • Kyle Creyts
  • Michael Hoffrath
  • Rmkml
  • Jean-Paul Roliers
  • Ignacio Sanchez
  • Michel Saborde
  • Simon Moon
  • Coverity

Known issues & missing features

As always, we are doing our best to make you aware of continuing development and items within the engine that are not yet complete or optimal. With this in mind, please notice the list we have included of known items we are working on.

See issues for an up to date list and to report new issues. See Known_issues for a discussion and time line for the major issues.

About Suricata

Suricata is a high performance Network IDS, IPS and Network Security Monitoring engine. Open Source and owned by a community run non-profit foundation, the Open Information Security Foundation (OISF). Suricata is developed by the OISF, its supporting vendors and the community.

Suricata 1.4beta3 Available for testing!

Photo by Eric LeblondThe OISF development team is proud to announce Suricata 1.4beta3. This is the third beta release for the upcoming 1.4 version.

This is release has significant improvements to the packet acquisition. The Napatech capture card support has been updated by our supporter Npulse. The Pcap, PF_RING and AF_PACKET capture methods now feature live drop stats.

Get the new release here: suricata-1.4beta3.tar.gz

New features

  • support for Napatech cards through their 3rd generation driver was added by Matt Keeler from Npulse (#430, #619)
  • support for pkt_data keyword was added
  • user and group to run as can now be set in the config file
  • make HTTP request and response body inspection sizes configurable per HTTP server config (#560)
  • PCAP/AF_PACKET/PF_RING packet stats are now printed in stats.log (#561, #625)
  • add stream event to match on overlaps with different data in stream reassembly (#603)

Improvements

  • add contrib directory to the dist (#567)
  • performance improvements to signatures with dsize option
  • improved rule analyzer: print fast_pattern along with the rule (#558)
  • fixes to stream engine reducing the number of events generated (#604)
  • stream.inline option new defaults to “auto”, meaning enabled in IPS mode, disabled in IDS mode (#592)
  • HTTP handling in OOM condition was greatly improved (#557)
  • filemagic keyword performance was improved (#585)
  • updated bundled libhtp to 0.2.11
  • build system improvements and cleanups

Fixes

  • fixes and improvements to daemon mode (#624)
  • fix drop rules not working correctly when thresholded (#613)
  • fixed a possible FP when a regular and “chopped” fast_pattern were the same (#581)
  • fix a false possitive condition in http_header (#607)
  • fix inaccuracy in byte_jump keyword when using “from_beginning” option (#627)
  • fixes to rule profiling (#576)
  • cleanups and misc fixes (#379, #395)
  • fix to SSL record parsing

Credits

We’d like to thank the following people and corporations for their contributions and feedback:

  • Matt Keeler – Npulse
  • Chris Wakelin
  • Rmkml
  • Will Metcalf
  • Ivan Ristic
  • Kyle Creyts
  • Michael Hoffrath

Known issues & missing features

In a beta release like this things may not be as polished yet. So please handle with care. That said, if you encounter issues, please let us know! As always, we are doing our best to make you aware of continuing development and items within the engine that are not yet complete or optimal. With this in mind, please notice the list we have included of known items we are working on.

See issues for an up to date list and to report new issues. See Known_issues for a discussion and time line for the major issues.

About Suricata

Suricata is a high performance Network IDS, IPS and Network Security Monitoring engine. Open Source and owned by a community run non-profit foundation, the Open Information Security Foundation (OISF). Suricata is developed by the OISF, its supporting vendors and the community.