SRX: Sharing the Rules Behind Text Segmentation

Why segmentation affects reuse

A translation workflow often works with pieces of text rather than a whole document at once. If two tools divide the same content differently, previously translated units may be harder to align and reuse.

Consider “Dr. Patel opened the file. It was empty.” A simplistic split after every full stop would create an unwanted boundary after the title. Lists, abbreviations, numbers and formatting can introduce further cases a project needs to handle.

The point is not that every document needs a custom rule. It is that the segmentation assumptions behind a resource should be understood before blaming a mismatch on the translations themselves.

An illustrative compatibility check

Our suggested exercise is to prepare a short corpus containing common abbreviations, numbered instructions, quotations and punctuation from the actual source language. Run it through the outgoing and incoming environments and compare the boundaries.

Where a difference occurs, identify whether it comes from a rule, a document filter or a tool setting. Preserve the corpus as regression material for later migrations.

These suggestions are editorial advice. They do not constitute the specification's formal conformance procedure.

Connection to LISA and later archives

SRX is directly part of the LISA legacy and complements translation-memory exchange. The ETSI work programme records its later SRX work item as stopped. Unicode's ULI archive discusses localization interoperability but explicitly says the subcommittee was disbanded in 2020. Neither archive should be described as an active SRX maintenance body. ETSI work record, Unicode ULI.

AI and segmentation

For an AI-assisted evaluator, our recommendation is to retain stable identifiers and enough surrounding context to interpret a segment. A disagreement may arise from a fragmented input rather than a bad translation. SRX does not prescribe a model's context window or tokenization.