Tag Archive | ja3

Release Notes for 5.0.0

New installations

In the default configuration Suricata-Update will download the ET Open ruleset. In the Suricata 5.0 optimized version, JA3 rules are added and enabled by default. See below for instructions on how to disable these rules with Suricata-Update.

The new default configuration has a number of extra EVE loggers enabled by default. These are the ‘anomaly’ logger, and loggers for the snmp, ftp protocols. In 4.1 Rust was optional, unlike in 5.0. This means that loggers for smb, nfs, tftp, ikev2, krb5 are now also enabled by default. As a result, logging volume may be higher than expected. Logging for these protocols can be enabled/disabled in the eve-log section in the suricata.yaml. 

Upgrading notes

When using Suricata with ET (Open or Pro) rules managed by Suricata-Update, the ruleset will automatically switch to the 5.0 version of the ruleset. This has a number of consequences.

  1. The ET 5.0 ruleset use a different classification scheme. Suricata 5.0 will issue warnings if rules use an unknown classtype. Update your classification.config from the one Suricata 5.0 ships or the ET ruleset version to suppress these warnings.
  2. If JA3 is enabled in the Suricata configuration (or not specified), the ET5 JA3 rules will be enabled by Suricata-Update. These rules have been quite noisy in the past. If they are alerting too frequently, the rules can be disabled in Suricata-Update.

Read more about upgrading https://suricata.readthedocs.io/en/latest/upgrade.html

Telling Suricata-Update to disable JA3 rules

  1. By filename. If all the JA3 rules are in a specific file like you find in ET Open and ET Pro, you can use Suricata update to disable all files in a rule. In /etc/suricata/disable.conf add the line: filename: rules/emerging-ja3.rules
  2. By regular expression. As all the rules we see in the ET Open and Pro ruleset are using the ja3_hash keyword, we can disable JA3 rules by using a regular expression looking for the ja3_keyword. This has the benefit of matching across all filenames. In your /etc/suricata/disable.conf, add the line: re: ja3_hash;

Read more  at https://suricata-update.readthedocs.io/en/1.1.0/update.html#example-configuration-to-disable-rules-disable-conf

EVE DNS Logging

Suricata 5.0 will default to the version 2 style of DNS logging in EVE if a version is not provided in the configuration. This is something to note if you are upgrading from 4.0, or 4.1 without Rust, as your EVE DNS log format will change. To continue using the version 1 format, you must update your configuration to include “version: 1”. See the documentation at https://suricata.readthedocs.io/en/latest/output/eve/eve-json-output.html#dns-v1-format for more information. However, we recommend moving to the version 2 style output, as it is more compact, and where enhancements to DNS logging will occur.

Reporting issues

To see what issues are already reported, see https://redmine.openinfosecfoundation.org/versions/138. If you run into an issue that isn’t listed, please open a new ticket.

Announcing Suricata 5.0.0

The OISF’s Suricata development team is proud to announce Suricata 5.0.0. This release brings many new features and improvements.

RDP, SNMP, FTP and SIP

Three new protocol parsers and loggers, all community contributions. Zach Kelly created a Rust RDP parser, while Giuseppe Longo created SIP support. Rust master Pierre Chifflier contributed SNMP support. Since RDP and SIP were merged late in our development cycle they are disabled by default in the configuration. For FTP we have added an EVE logging facility.

JA3S

After contributing JA3 support in Suricata 4.1, Mats Klepsland has been working on JA3S support. JA3S is now available to the rule language and in the TLS logging output.

Datasets

Still experimental at this time, the initial work to support datasets is part of this release. It allows matching on large amounts of data. It is controlled from the rule language and will work with any ‘sticky buffer’.

See documentation at https://suricata.readthedocs.io/en/suricata-5.0.0/rules/datasets.html

We’ve already heard of people using this with millions of IOCs.

Documentation

With the help of many community members we’ve been improving the user documentation. Please see: https://suricata.readthedocs.io/en/suricata-5.0.0/

HTTP evader

We’ve been working hard to cover the final set of HTTP evader cases. This work has mostly gone into the bundled libhtp 0.5.31.

Rust

The most visible is that our Rust support is no longer optional. We’re convinced that Rust is a perfect match for Suricata, and we plan to increase its footprint in our code base steadily. By making it mandatory we’re able to remove parallel implementations and focus fully on making the Rust code better.

Protocol Detection

The protocol detection engine has been extended to provide better accuracy as well as support for dealing with asynchronous flows. These async flows are sometimes picked up in the wrong direction and the protocol detection engine can now reverse them.

Decoder Anomaly records in EVE

A new log record type has been added: ‘anomaly’. This logs the stream and decoder events that are set by the packet decoders. This is inspired by Zeeks (Bro) ‘weird’ log.

EVE improvements

VLAN and capture interface is now part of many more EVE records, even if they are flow records or records based on flow time out.

An option to log all HTTP headers to the EVE http records has been added.

Packet Capture

Eric Leblond has been working hard to getting hardware offload support working for eBPF. On Netronome cards the eBPF based flow bypass can now be offloaded to the NIC. As eBPF is becoming a standard in the Linux space, we are hoping to see other hardware offload soon as well.

Netmap support has been rewritten so the more advanced features of netmap, such as vale switches, can be used now.

Napatech usability has been improved.

Rule language: Sticky Buffers

As discussed at the Suricon 2018 brainstorm session, a new rule keyword scheme is being introduced. It takes the existing ‘sticky buffer’ approach with new keyword names to avoid confusion. The new scheme is <proto>.<buffer>, so for example ‘http.uri’ for the URI inspection.

A number of HTTP keywords have been added.

Unified Lua inspection mixed with the sticky buffers has also been implemented.

Python 3

With Python 2’s EOL approaching, we’ve made sure that all Suricata’s python code is Python 3 compliant.

Removals

Following our deprecation policy, we have removed the following parts: the plain text dns.log, the old files-json.log and support for the Tilera architecture.

https://suricata-ids.org/about/deprecation-policy/

All tickets

Beta 1 tickets: https://redmine.openinfosecfoundation.org/versions/115

RC 1 tickets: https://redmine.openinfosecfoundation.org/versions/128

Final tickets: https://redmine.openinfosecfoundation.org/versions/129

Download

https://suricata-ids.org/download/

Suricata 4.1 released!

After a longer than intended release development cycle, the OISF development team is proud to present Suricata 4.1.

Main new features are inclusion of the protocols SMBv1/2/3, NFSv4, Kerberos, FTP, DHCP, IKEv2. All of them have been implemented in Rust to ensure their introduction will not be compromising to the security and the stability of the complete system.

Support for tracking and logging TLS 1.3 has been added, including JA3 support.

On performance side, one of the main improvements is the availability of capture bypass for AF_PACKET implemented on top of the new eXpress Data Path (XDP) capability of Linux kernel. Windows users will benefit from the 4.1 release with a new IPS mode based on WinDivert.

All new protocols require Rust so Suricata 4.1 is not really 4.1 if you don’t have Rust. This is why the build system is now enabling Rust by default if it is available on the build machine.

This is the first release where Suricata-Update 1.0, the new Suricata rule updater, is bundled.

Protocol updates

  • SMBv1/2/3 parsing, logging, file extraction
  • TLS 1.3 parsing and logging (Mats Klepsland)
  • JA3 TLS client fingerprinting (Mats Klepsland)
  • TFTP: basic logging (Pascal Delalande and Clément Galland)
  • FTP: file extraction
  • Kerberos parser and logger (Pierre Chifflier)
  • IKEv2 parser and logger (Pierre Chifflier)
  • DHCP parser and logger
  • Flow tracking for ICMPv4
  • Initial NFS4 support
  • HTTP: handle sessions that only have a response, or start with a response
  • HTTP Flash file decompression support (Giuseppe Longo)

Output and logging

  • File extraction v2: deduplication; hash-based naming; json metadata and cleanup tooling
  • Eve metadata: from rules (metadata keyword) and traffic (flowbits etc)
  • Eve: new more compact DNS record format (Giuseppe Longo)
  • Pcap directory mode: process all pcaps in a directory (Danny Browning)
  • Compressed PCAP logging (Max Fillinger)
  • Expanded XFF support (Maurizio Abba)
  • Community Flow Id support (common ID between Suricata and Bro/Zeek)

Packet Capture

  • AF_PACKET XDP and eBPF support for high speed packet capture
  • Windows IPS: WinDivert support (Jacob Masen-Smith)
  • PF_RING: usability improvements

Misc

  • Windows: MinGW is now supported
  • Detect: transformation keyword support
  • Bundled Suricata-Update
  • Per device multi-tenancy

Minor Changes since 4.1rc2

  • Coverity fixes and annotations
  • Update Suricata-Update to 1.0.0

Security

  • SMTP crash issue was fixed: CVE-2018-18956
  • Robustness of defrag against FragmentSmack was improved
  • Robustness of TCP reassembly against SegmentSmack was improved

Download

https://www.openinfosecfoundation.org/download/suricata-4.1.0.tar.gz

Get paid to work on Suricata!

Enjoying the testing? Or want to help out with other parts of the project?
We are looking for people, so reach out to us if you’re interested.

Special thanks

Mats Klepsland, Pierre Chifflier, Giuseppe Longo, Ralph Broenink, Danny Browning, Maurizio Abba, Pascal Delalande, Wolfgang Hotwagner, Jason Taylor, Jesper Dangaard Brouer, Alexander Gozman, Konstantin Klinger, Max Fillinger, Antoine LUONG, David DIALLO, Jacob Masen-Smith, Martin Natano, Ruslan Usmanov, Alfredo Cardigliano, Antti Tönkyrä, Brandon Sterne, Chris Speidel, Clément Galland, Dana Helwig, Daniel Humphries, Elazar Broad, Gaurav Singh, Hilko Bengen, Nick Price, Philippe Antoine, Renato Botelho, Thomas Andrejak, Paulo Pacheco, Henning Perl, Kirill Shipulin, Christian Kreibich, Tilli Juha-Matti.

Trainings

Check out the latest training offerings at https://suricata-ids.org/training/

The 2019 calendar of trainings will be out soon – check back here or follow us on Twitter (@OISFoundation) for all training announcements.

Suricon 2018

Suricon 2018 Vancouver is next week and it’s still possible to join! https://suricon.net/

About Suricata

Suricata is a high performance Network Threat Detection, IDS, IPS and Network Security Monitoring engine. Open Source and owned by a community run non-profit foundation, the Open Information Security Foundation (OISF). Suricata is developed by OISF, its supporting vendors and the community.

Call for testing: Suricata 4.1rc2 released

Suricata 4.1rc2 is ready for testing. We’re hoping that this will be the final release candidate so that 4.1 can be released just before Suricon next month.

Main new features are inclusion of the protocols SMBv1/2/3, NFSv4, Kerberos,FTP, DHCP, IKEv2, as well as improvements on Linux capture side via AF_PACKET XDP support and on Windows IPS side via WinDivert. The growth of Rust usage inside Suricata continues as most of the new protocols have been implemented in Rust.

Most important change for going from RC1 to RC2 is that we have enabled Rust support by default. If Rust is installed, it will be used.

Protocol updates

  • SMBv1/2/3 parsing, logging, file extraction
  • TLS 1.3 parsing and logging (Mats Klepsland)
  • JA3 TLS client fingerprinting (Mats Klepsland)
  • TFTP: basic logging (Pascal Delalande and Clément Galland)
  • FTP: file extraction
  • Kerberos parser and logger (Pierre Chifflier)
  • IKEv2 parser and logger (Pierre Chifflier)
  • DHCP parser and logger
  • Flow tracking for ICMPv4
  • Initial NFS4 support
  • HTTP: handle sessions that only have a response, or start with a response
  • HTTP Flash file decompression support (Giuseppe Longo)

Output and logging

  • File extraction v2: deduplication; hash-based naming; json metadata and cleanup tooling
  • Eve metadata: from rules (metadata keyword) and traffic (flowbits etc)
  • Eve: new more compact DNS record format (Giuseppe Longo)
  • Pcap directory mode: process all pcaps in a directory (Danny Browning)
  • Compressed PCAP logging (Max Fillinger)
  • Expanded XFF support (Maurizio Abba)
  • Community Flow Id support (common ID between Suricata and Bro/Zeek)

Packet Capture

  • AF_PACKET XDP and eBPF support for high speed packet capture
  • Windows IPS: WinDivert support (Jacob Masen-Smith)

Misc

  • Windows: MinGW is now supported
  • Detect: transformation keyword support
  • Bundled Suricata-Update
  • Per device multi-tenancy

Major changes since 4.1rc1

  • Rust support is enabled by default
  • Community Flow Id support (common ID between Suricata and Bro/Zeek)
  • Updates and fixes for dealing with SegmentSmack/FragmentSmack
  • Update Suricata-Update to 1.0.0rc2

Get paid to work on Suricata!

Enjoying the testing? Or want to help out with other parts of the project?
We are looking for people, so reach out to us if you’re interested.

Special thanks

Mats Klepsland, Jason Taylor, Maurizio Abba, Konstantin Klinger, Giuseppe Longo, Danny Browning, Hilko Bengen, Jacob Masen-Smith, Pascal Delalande, Travis Green, Christian Kreibich

Trainings

Check out the latest training offerings at https://suricata-ids.org/training/

SuriCon

SuriCon 2018 Vancouver next month, you can still join! https://suricon.net/agenda-vancouver/

About Suricata

Suricata is a high performance Network Threat Detection, IDS, IPS and Network Security Monitoring engine. Open Source and owned by a community run non-profit foundation, the Open Information Security Foundation (OISF). Suricata is developed by the OISF, its supporting vendors and the community.

Call for testing: Suricata 4.1rc1 released

It’s summer, so an excellent time for some testing! Suricata 4.1 release candidate 1 is here to be tried out. The release brings a lot of new features.

Get the release here: https://www.openinfosecfoundation.org/download/suricata-4.1.0-rc1.tar.gz

Main new features are inclusion of the protocols SMBv1/2/3, NFSv4, Kerberos, FTP, DHCP, IKEv2, as well as improvements on Linux capture side via AF_PACKET XDP support and on Windows IPS side via WinDivert. The progress in Rust usage inside Suricata continues as most of the new protocols have been implemented in Rust.

We invite everyone to test this release and report your experiences to us.

Protocol updates

  • SMBv1/2/3 parsing, logging, file extraction
  • JA3 TLS client fingerprinting (Mats Klepsland)
  • TFTP: basic logging (Pascal Delalande and Clément Galland)
  • FTP: file extraction
  • Kerberos parser and logger (Pierre Chifflier)
  • IKEv2 parser and logger (Pierre Chifflier)
  • DHCP parser and logger
  • Flow tracking for ICMPv4
  • Initial NFS4 support
  • HTTP: handle sessions that only have a response, or start with a response
  • HTTP Flash file decompression support (Giuseppe Longo)

Output and logging

  • File extraction v2: deduplication; hash-based naming; json metadata and cleanup tooling
  • Eve metadata: from rules (metadata keyword) and traffic (flowbits etc)
  • Eve: new more compact DNS record format (Giuseppe Longo)
  • Pcap directory mode: process all pcaps in a directory (Danny Browning)
  • Compressed PCAP logging (Max Fillinger)
  • Expanded XFF support (Maurizio Abba)

Packet Capture

  • AF_PACKET XDP and eBPF support for high speed packet capture
  • Windows IPS: WinDivert support (Jacob Masen-Smith)

Misc

  • Windows: MinGW is now supported
  • Detect: transformation keyword support
  • Bundled Suricata-Update

Major changes since 4.1beta1

  • WinDivert support
  • Kerberos parser and logger
  • IKEv2 parser and logger
  • DHCP parser and logger
  • Flow tracking for ICMPv4
  • Initial NFS4 support
  • Compressed PCAP logging
  • Expanded XFF support
  • Decode GRE over IP (Paulo Pacheco)
  • Multi-tenancy fixes
  • SMB improvements for midstream pickup
  • Update Suricata-Update to 1.0.0rc1

Security

CVE-2018-10242, CVE-2018-10244 (suricata)
CVE-2018-10243 (libhtp)

Get paid to work on Suricata!

Enjoying the testing? Or want to help out with other parts of the project?
We are looking for people, so reach out to us if you’re interested.

Special thanks

Henning Perl, Kirill Shipulin, Pierre Chifflier, Mats Klepsland, Max Fillinger, Alexander Gozman, Danny Browning, Giuseppe Longo, Maurizio Abba, Pascal Delalande, Chris Speidel, Elazar Broad, Jacob Masen-Smith, Renato Botelho, Paulo Pacheco, Jason Taylor

Trainings

Check out the latest training offerings at https://suricata-ids.org/training/

SuriCon

SuriCon 2018 Vancouver agenda is up! https://suricon.net/agenda-vancouver/

About Suricata

Suricata is a high performance Network Threat Detection, IDS, IPS and Network Security Monitoring engine. Open Source and owned by a community run non-profit foundation, the Open Information Security Foundation (OISF). Suricata is developed by the OISF, its supporting vendors and the community.

Suricata 4.1 beta 1 ready for testing

We are proud to announce that the first beta release for the upcoming Suricata 4.1 is ready for testing. This release is brought to you by the OISF development team with the help 25 community contributors.

Download:
https://www.openinfosecfoundation.org/download/suricata-4.1.0-beta1.tar.gz

We invite everyone to test this release and report your experiences to us.

Main features additions

  • SMBv1/2/3 parsing, logging, file extraction
  • AF_PACKET XDP and eBPF support for high speed packet capture
  • JA3 TLS client fingerprinting
  • HTTP: handle sessions that only have a response, or start with a response
  • Windows: MinGW is now supported
  • File extraction v2: deduplication; hash-based naming; json metadata and cleanup tooling
  • Eve metadata: from rules (metadata keyword) and traffic (flowbits etc)
  • Pcap directory mode: process all pcaps in a directory
  • Detect: transformation support
  • Eve: new more compact DNS record format
  • TFTP: basic logging
  • HTTP Flash file decompression support
  • All tickets: https://redmine.openinfosecfoundation.org/versions/105

Special thanks

Giuseppe Longo, Mats Klepsland, Pierre Chifflier, Ralph Broenink, Wolfgang Hotwagner, Danny Browning, Pascal Delalande, Jesper Dangaard Brouer, Maurizio Abba, Alexander Gozman, Antoine LUONG, David DIALLO, Martin Natano, Ruslan Usmanov, Alfredo Cardigliano, Antti Tönkyrä, Brandon Sterne, Clément Galland, Dana Helwig, Daniel Humphries, Gaurav Singh, Nick Price, Philippe Antoine, Thomas Andrejak, Jason Taylor

SuriCon 2018

Come meet the Suricata community and development team to discuss all things Suricata at the fourth edition of the annual Suricata Conference. SuriCon 2018 will be held in November in Vancouver, Canada: https://suricon.net

Our call for presentations is still open, so please submit your ideas!

Also, we’re still looking for sponsors for the event.

About Suricata

Suricata is a high performance Network Threat Detection, IDS, IPS and Network Security Monitoring engine. Open Source and owned by a community run non-profit foundation, the Open Information Security Foundation (OISF). Suricata is developed by the OISF, its supporting vendors and the community.