OMI contributes 6 million lines of DFDL schema code

DFDL, the Data Format Description Language, is a standard for describing binary and text data formats declaratively. Apache Daffodil is a major open-source implementation. DFDL emerged from standards work in the 2000s, became an Open Grid Forum standard, and was later standardized as ISO/IEC 23415.

DFDL is an ambitious format description language that tries to describe preexisting formats and protocols rather than dictating how a protocol should work. This is in contrast to systems such as Protobuf, Avro, Thrift, and SBE, which generally start with a schema and prescribe how that schema is encoded on the wire. That means DFDL can model:

• byte and bit order
• arbitrary bit fields
• alignment and padding
• fixed and variable lengths
• delimiters and escape schemes
• character encodings
• optional and repeating structures
• arrays whose sizes come from other fields
• alternate record layouts
• discriminators and assertions
• expressions that calculate lengths, offsets, or values while parsing
• binary and textual representations inside the same format

DFDL, with its domain-specific XML tags, is the broadest existing attempt at modelling a large range of formats. The public DFDLSchemas organization contains schemas for formats including CSV, ISO 8583 financial messages, NACHA payments, UN/EDIFACT, HL7 healthcare messages, network protocols, PCAPs, images, geospatial formats, and military messaging formats.

Apache Daffodil's own examples demonstrate the same DFDL engine handling formats as different as CSV and packet captures.

IBM supports DFDL in IBM App Connect Enterprise, where it can be used to describe and process arbitrary text and binary message formats.

The European Space Agency developed DFDL4S for space-data processing, using DFDL-based descriptions in systems dealing with satellite data.

DFDL also has significant use in defense, cybersecurity, and cross-domain systems. Apache Daffodil has been associated with work at organizations including DARPA, the Naval Postgraduate School, Raytheon BBN, and GE Research. The DFDL ecosystem includes descriptions of military formats such as MIL-STD-2045, VMF, USMTF, and Link 16 / MIL-STD-6016 / NATO STANAG 5516 (NATO's standard secure battlefield data sharing format).

One indication of the complexity DFDL can represent is Link 16. An Apache Daffodil presentation from 2025 describes a closed-source Link 16 DFDL schema exceeding 800,000 lines.

That context makes OMI's output especially interesting.

6 million lines

The large volume of OMI's generated schema code shows how incredibly varied the trading-protocol landscape has become. For comparison, the same Apache presentation measures the entire Apache Daffodil project, including implementation source, unit tests, and its DFDL test suite, at roughly 397,000 lines.

OMI has now generated more than 6 million lines of highly detailed DFDL describing financial protocols.

A DFDL infoset implementation such as Apache Daffodil can take a schema and use it to create parsers and unparsers:

protocol specification → DFDL → parser / unparser

OMI models thousands of financial protocols. Those models sit above any type of DSL and contain enough information to generate production code, documentation, dissectors, tests, formal-verification artifacts, working exchange simulators, and now millions of lines of DFDL.

So the pipeline becomes:

protocol specification → OMI model → DFDL → parser / unparser

One level up

DFDL itself is intended to reduce the overall code written to create new parsers. OMI moves that abstraction up another level: instead of manually writing and maintaining the DFDL schemas, we programmatically calculate and generate these also. This is how we were able to quickly multiply the existing open-source DFDL schema code base.

OMI's contribution should make it easier for anyone familiar with the DFDL ecosystem to handle financial protocols, and it also stress tests the OMI specification models. DFDL is one of the most expressive standardized declarative systems available for describing the physical representation of data, and we are proud to contribute to the project.