Tag Archive | ftp

Announcing Suricata 5.0.0

The OISF’s Suricata development team is proud to announce Suricata 5.0.0. This release brings many new features and improvements.

RDP, SNMP, FTP and SIP

Three new protocol parsers and loggers, all community contributions. Zach Kelly created a Rust RDP parser, while Giuseppe Longo created SIP support. Rust master Pierre Chifflier contributed SNMP support. Since RDP and SIP were merged late in our development cycle they are disabled by default in the configuration. For FTP we have added an EVE logging facility.

JA3S

After contributing JA3 support in Suricata 4.1, Mats Klepsland has been working on JA3S support. JA3S is now available to the rule language and in the TLS logging output.

Datasets

Still experimental at this time, the initial work to support datasets is part of this release. It allows matching on large amounts of data. It is controlled from the rule language and will work with any ‘sticky buffer’.

See documentation at https://suricata.readthedocs.io/en/suricata-5.0.0/rules/datasets.html

We’ve already heard of people using this with millions of IOCs.

Documentation

With the help of many community members we’ve been improving the user documentation. Please see: https://suricata.readthedocs.io/en/suricata-5.0.0/

HTTP evader

We’ve been working hard to cover the final set of HTTP evader cases. This work has mostly gone into the bundled libhtp 0.5.31.

Rust

The most visible is that our Rust support is no longer optional. We’re convinced that Rust is a perfect match for Suricata, and we plan to increase its footprint in our code base steadily. By making it mandatory we’re able to remove parallel implementations and focus fully on making the Rust code better.

Protocol Detection

The protocol detection engine has been extended to provide better accuracy as well as support for dealing with asynchronous flows. These async flows are sometimes picked up in the wrong direction and the protocol detection engine can now reverse them.

Decoder Anomaly records in EVE

A new log record type has been added: ‘anomaly’. This logs the stream and decoder events that are set by the packet decoders. This is inspired by Zeeks (Bro) ‘weird’ log.

EVE improvements

VLAN and capture interface is now part of many more EVE records, even if they are flow records or records based on flow time out.

An option to log all HTTP headers to the EVE http records has been added.

Packet Capture

Eric Leblond has been working hard to getting hardware offload support working for eBPF. On Netronome cards the eBPF based flow bypass can now be offloaded to the NIC. As eBPF is becoming a standard in the Linux space, we are hoping to see other hardware offload soon as well.

Netmap support has been rewritten so the more advanced features of netmap, such as vale switches, can be used now.

Napatech usability has been improved.

Rule language: Sticky Buffers

As discussed at the Suricon 2018 brainstorm session, a new rule keyword scheme is being introduced. It takes the existing ‘sticky buffer’ approach with new keyword names to avoid confusion. The new scheme is <proto>.<buffer>, so for example ‘http.uri’ for the URI inspection.

A number of HTTP keywords have been added.

Unified Lua inspection mixed with the sticky buffers has also been implemented.

Python 3

With Python 2’s EOL approaching, we’ve made sure that all Suricata’s python code is Python 3 compliant.

Removals

Following our deprecation policy, we have removed the following parts: the plain text dns.log, the old files-json.log and support for the Tilera architecture.

https://suricata-ids.org/about/deprecation-policy/

All tickets

Beta 1 tickets: https://redmine.openinfosecfoundation.org/versions/115

RC 1 tickets: https://redmine.openinfosecfoundation.org/versions/128

Final tickets: https://redmine.openinfosecfoundation.org/versions/129

Download

https://suricata-ids.org/download/

Please help us test Suricata 5.0.0-rc1

We are looking for testers for a new development release in the Suricata 5 series: Suricata 5.0.0-rc1. Please help us test so we can release the final on October 15th.

Curious about whats new? Here are the highlights:

RDP, SNMP, FTP and SIP

Three new protocol parsers and loggers, both community contributions. Zach Kelley created a Rust RDP parser, while Giuseppe Longo created SIP support. Rust master Pierre Chifflier contributed SNMP support. Since RDP and SIP were merged late in our development cycle they are disabled by default in the configuration. For FTP we have added a EVE logging facility.

JA3S

After contributing JA3 support in Suricata 4.1, Mats Klepsland has been working on JA3S support. JA3S is now available to the rule language and in the TLS logging output.

eBPF/XDP

Eric Leblond has been working hard to getting hardware offload support working for eBPF. On Netronome cards the eBPF based flow bypass can now be offloaded to the NIC.

Datasets

Still experimental at this time, the initial work to support datasets is part of this release. It allows matching on large amounts of data. It is controlled from the rule language and will work with any ‘sticky buffer’. https://suricata.readthedocs.io/en/suricata-5.0.0-rc1/rules/datasets.html

HTTP evader

We’ve been working hard to cover the final set of HTTP evader cases. This work has mostly gone into the bundled libhtp 0.5.31.

More 5.0 changes

Please see the beta1 announcement for many more changes in the upcoming 5.0 release: https://suricata-ids.org/2019/04/30/call-for-testing-announcing-suricata-5-0-0-beta1/

For a complete list of closed tickets in 5.0.0-rc1, please see https://redmine.openinfosecfoundation.org/versions/128

Release schedule

This release has been delayed quite a bit. We had originally hoped to have it ready for you in July. This means that to get the final out before Suricon next month we have quite an aggressive schedule. We want to release the final no later than October 15th. We can use all the help we can get with testing and polishing to meet that goal. Thanks in advance!

Download from:

https://www.openinfosecfoundation.org/downloads/suricata-5.0.0-rc1.tar.gz

Suricata 4.1 released!

After a longer than intended release development cycle, the OISF development team is proud to present Suricata 4.1.

Main new features are inclusion of the protocols SMBv1/2/3, NFSv4, Kerberos, FTP, DHCP, IKEv2. All of them have been implemented in Rust to ensure their introduction will not be compromising to the security and the stability of the complete system.

Support for tracking and logging TLS 1.3 has been added, including JA3 support.

On performance side, one of the main improvements is the availability of capture bypass for AF_PACKET implemented on top of the new eXpress Data Path (XDP) capability of Linux kernel. Windows users will benefit from the 4.1 release with a new IPS mode based on WinDivert.

All new protocols require Rust so Suricata 4.1 is not really 4.1 if you don’t have Rust. This is why the build system is now enabling Rust by default if it is available on the build machine.

This is the first release where Suricata-Update 1.0, the new Suricata rule updater, is bundled.

Protocol updates

  • SMBv1/2/3 parsing, logging, file extraction
  • TLS 1.3 parsing and logging (Mats Klepsland)
  • JA3 TLS client fingerprinting (Mats Klepsland)
  • TFTP: basic logging (Pascal Delalande and Clément Galland)
  • FTP: file extraction
  • Kerberos parser and logger (Pierre Chifflier)
  • IKEv2 parser and logger (Pierre Chifflier)
  • DHCP parser and logger
  • Flow tracking for ICMPv4
  • Initial NFS4 support
  • HTTP: handle sessions that only have a response, or start with a response
  • HTTP Flash file decompression support (Giuseppe Longo)

Output and logging

  • File extraction v2: deduplication; hash-based naming; json metadata and cleanup tooling
  • Eve metadata: from rules (metadata keyword) and traffic (flowbits etc)
  • Eve: new more compact DNS record format (Giuseppe Longo)
  • Pcap directory mode: process all pcaps in a directory (Danny Browning)
  • Compressed PCAP logging (Max Fillinger)
  • Expanded XFF support (Maurizio Abba)
  • Community Flow Id support (common ID between Suricata and Bro/Zeek)

Packet Capture

  • AF_PACKET XDP and eBPF support for high speed packet capture
  • Windows IPS: WinDivert support (Jacob Masen-Smith)
  • PF_RING: usability improvements

Misc

  • Windows: MinGW is now supported
  • Detect: transformation keyword support
  • Bundled Suricata-Update
  • Per device multi-tenancy

Minor Changes since 4.1rc2

  • Coverity fixes and annotations
  • Update Suricata-Update to 1.0.0

Security

  • SMTP crash issue was fixed: CVE-2018-18956
  • Robustness of defrag against FragmentSmack was improved
  • Robustness of TCP reassembly against SegmentSmack was improved

Download

https://www.openinfosecfoundation.org/download/suricata-4.1.0.tar.gz

Get paid to work on Suricata!

Enjoying the testing? Or want to help out with other parts of the project?
We are looking for people, so reach out to us if you’re interested.

Special thanks

Mats Klepsland, Pierre Chifflier, Giuseppe Longo, Ralph Broenink, Danny Browning, Maurizio Abba, Pascal Delalande, Wolfgang Hotwagner, Jason Taylor, Jesper Dangaard Brouer, Alexander Gozman, Konstantin Klinger, Max Fillinger, Antoine LUONG, David DIALLO, Jacob Masen-Smith, Martin Natano, Ruslan Usmanov, Alfredo Cardigliano, Antti Tönkyrä, Brandon Sterne, Chris Speidel, Clément Galland, Dana Helwig, Daniel Humphries, Elazar Broad, Gaurav Singh, Hilko Bengen, Nick Price, Philippe Antoine, Renato Botelho, Thomas Andrejak, Paulo Pacheco, Henning Perl, Kirill Shipulin, Christian Kreibich, Tilli Juha-Matti.

Trainings

Check out the latest training offerings at https://suricata-ids.org/training/

The 2019 calendar of trainings will be out soon – check back here or follow us on Twitter (@OISFoundation) for all training announcements.

Suricon 2018

Suricon 2018 Vancouver is next week and it’s still possible to join! https://suricon.net/

About Suricata

Suricata is a high performance Network Threat Detection, IDS, IPS and Network Security Monitoring engine. Open Source and owned by a community run non-profit foundation, the Open Information Security Foundation (OISF). Suricata is developed by OISF, its supporting vendors and the community.

Call for testing: Suricata 4.1rc1 released

It’s summer, so an excellent time for some testing! Suricata 4.1 release candidate 1 is here to be tried out. The release brings a lot of new features.

Get the release here: https://www.openinfosecfoundation.org/download/suricata-4.1.0-rc1.tar.gz

Main new features are inclusion of the protocols SMBv1/2/3, NFSv4, Kerberos, FTP, DHCP, IKEv2, as well as improvements on Linux capture side via AF_PACKET XDP support and on Windows IPS side via WinDivert. The progress in Rust usage inside Suricata continues as most of the new protocols have been implemented in Rust.

We invite everyone to test this release and report your experiences to us.

Protocol updates

  • SMBv1/2/3 parsing, logging, file extraction
  • JA3 TLS client fingerprinting (Mats Klepsland)
  • TFTP: basic logging (Pascal Delalande and Clément Galland)
  • FTP: file extraction
  • Kerberos parser and logger (Pierre Chifflier)
  • IKEv2 parser and logger (Pierre Chifflier)
  • DHCP parser and logger
  • Flow tracking for ICMPv4
  • Initial NFS4 support
  • HTTP: handle sessions that only have a response, or start with a response
  • HTTP Flash file decompression support (Giuseppe Longo)

Output and logging

  • File extraction v2: deduplication; hash-based naming; json metadata and cleanup tooling
  • Eve metadata: from rules (metadata keyword) and traffic (flowbits etc)
  • Eve: new more compact DNS record format (Giuseppe Longo)
  • Pcap directory mode: process all pcaps in a directory (Danny Browning)
  • Compressed PCAP logging (Max Fillinger)
  • Expanded XFF support (Maurizio Abba)

Packet Capture

  • AF_PACKET XDP and eBPF support for high speed packet capture
  • Windows IPS: WinDivert support (Jacob Masen-Smith)

Misc

  • Windows: MinGW is now supported
  • Detect: transformation keyword support
  • Bundled Suricata-Update

Major changes since 4.1beta1

  • WinDivert support
  • Kerberos parser and logger
  • IKEv2 parser and logger
  • DHCP parser and logger
  • Flow tracking for ICMPv4
  • Initial NFS4 support
  • Compressed PCAP logging
  • Expanded XFF support
  • Decode GRE over IP (Paulo Pacheco)
  • Multi-tenancy fixes
  • SMB improvements for midstream pickup
  • Update Suricata-Update to 1.0.0rc1

Security

CVE-2018-10242, CVE-2018-10244 (suricata)
CVE-2018-10243 (libhtp)

Get paid to work on Suricata!

Enjoying the testing? Or want to help out with other parts of the project?
We are looking for people, so reach out to us if you’re interested.

Special thanks

Henning Perl, Kirill Shipulin, Pierre Chifflier, Mats Klepsland, Max Fillinger, Alexander Gozman, Danny Browning, Giuseppe Longo, Maurizio Abba, Pascal Delalande, Chris Speidel, Elazar Broad, Jacob Masen-Smith, Renato Botelho, Paulo Pacheco, Jason Taylor

Trainings

Check out the latest training offerings at https://suricata-ids.org/training/

SuriCon

SuriCon 2018 Vancouver agenda is up! https://suricon.net/agenda-vancouver/

About Suricata

Suricata is a high performance Network Threat Detection, IDS, IPS and Network Security Monitoring engine. Open Source and owned by a community run non-profit foundation, the Open Information Security Foundation (OISF). Suricata is developed by the OISF, its supporting vendors and the community.