Hyphenation: what you have to do

Hyphenation needs one thing that does not come with the package: the patterns. Everything else is two lines. This page is the whole road, from the file on disk to a justified column that no longer has rivers in it.

Why the patterns are not included. Every published pattern set carries terms of its own — the German ones the LaTeX Project Public License over LGPL, the American English ones a BSD-style notice over the plain TeX table — and tclpdf is MIT. Shipping them would make the smallest of those licences the one every user of the package has to read. So the file stays where it is, with its own licence, and the caller points at it.

1. Get a pattern file

The format is libhyphen's .dic — what LibreOffice, Hunspell and the hyphen library install, hyph_de_DE.dic and its fifty siblings. It names its own encoding on the first line and, usually, its own minima in the lines after it.

Where	Path
Most Linux systems	/usr/share/hyphen/
macOS or Windows with LibreOffice	…/LibreOffice.app/Contents/Resources/extensions/dict-*/
Nothing installed	the LibreOffice dictionary repository, https://github.com/LibreOffice/dictionaries — one file per language, no build step

A TeX .pat.txt file holds the same patterns and can be handed to load once its first line names an encoding — but that is a conversion done once, outside, not a second reader.

2. Load it

The hyphenation module is one nothing loads by itself, so it needs a package require of its own; the patterns have to be there before any block asks for them.

package require tclpdf
package require tclpdf::hyphenate

::tclpdf::hyphenate load de-DE /usr/share/hyphen/hyph_de_DE.dic -left 2 -right 2
::tclpdf::hyphenate load en-US /usr/share/hyphen/hyph_en_US.dic

The tag is RFC 3066, the same spelling the document's language takes. A tag is looked up exactly first and then by its primary subtag, so patterns loaded as de serve a document set in de-AT, and de-DE serves one set in de — the tables differ far less between the regions of a language than between languages.

-left 2 -right 2 for German is not decoration. The minima say how many letters have to stay on either side of a break, and hyph_de_DE.dic states neither of the two that matter — it carries only the compound ones. Without them the plain TeX defaults 2 and 3 apply, and a right minimum of 3 refuses the two-letter endings German breaks off every day. hyph_en_US.dic states its own LEFTHYPHENMIN 2 and RIGHTHYPHENMIN 3, so English needs nothing. -min is the shortest word broken at all, 5 unless said otherwise, and -exceptions {Wachs-tu-be Ur-in-stinkt} takes words spelled with the breaks in them, looked up before the patterns and taken exactly as given.

3. Ask for it on the block

$doc language de-DE
$doc text $paragraph -at {20 40} -width 78 -align justify -hyphenate 1
$doc text $english   -at {20 40} -width 78 -align justify -hyphenate en-US

-hyphenate takes 0, 1 or a language tag. 0 is the default; 1 uses the language the document itself declares with language; anything else is a tag. Only those two literal values are the switch and every other value is a tag — no is a Tcl false and the RFC 3066 tag for Norwegian, and a boolean reading would have turned hyphenation off for the one language whose name says otherwise. textLines and textHeight take the option as well, so a block is measured the way it will be set. A table hyphenates too, but takes it as a style key rather than as an option — {hyphenate de-DE} in -style, in a section style, in a column description or in a cell's own style — because a table can carry a German description column beside an English note, and a language said once for the whole table could not say that. A language nobody loaded refuses the whole table by name, TCLPDF HYPHENATE LANGUAGE, before its first cell is drawn.

4. Check what it does

word gives the pieces a word falls into, and they always concatenate back to what was handed in, so they can be joined with a hyphen or used to cut the original string. languages says what is loaded.

% ::tclpdf::hyphenate word de-DE Silbentrennung
Sil ben tren nung
% ::tclpdf::hyphenate word de-DE Abend
Abend
% ::tclpdf::hyphenate word en-US hyphenation
hy phen ation
% ::tclpdf::hyphenate languages
de-DE en-US
% ::tclpdf::hyphenate languages de-DE
tag de-DE patterns 8718 exceptions 0 left 2 right 2 min 5

Abend comes back whole, and that is the minima at work — the alternative is a line ending in A-. A one-element list is deliberately the same answer as "no break was found": neither is a failure.

What is not broken

A word shorter than -min, a run of fewer letters than the minima allow, a word with a digit anywhere in it, a word written in capitals throughout, and anything shaped like a URL, an address or a file name — anything that still holds a dot, a colon, a slash or an at sign once the punctuation has been taken off its ends. A word that carries a hyphen of its own is broken in its parts but never at the hyphen it already has, or the line would end in two of them.

Soft hyphens win outright. Where the text carries U+00AD marks of its own, those are the offers used and the patterns stay out: it is one source or the other, never both, because whoever wrote the marks knew that word.

What the result looks like

The hyphen set at the break is a real one, U+002D, so the line ends the way a reader expects. In a tagged document it sits in a Span with an empty ActualText, which is how ISO 32000-1 says to tell a reader that the character is not part of the text — extracting such a line gives the word back whole. An untagged document is written exactly as it was before, bracket and all left out.

Decomposed text hyphenates like composed text. A word is canonically composed before the patterns see it and the breaks are mapped back, so heru + U+0308 + ber falls where herüber falls. The minima count letters, and a combining mark is not one, so a break never falls between a letter and its own mark.

When something is missing

A language that is not loaded is refused by name — TCLPDF HYPHENATE LANGUAGE — rather than set unhyphenated in silence: a document that quietly is not what was asked for is worse than one that stops. -hyphenate 1 on a document that declares no language raises TCLPDF HYPHENATE UNSET. Both leave three ways on: load the file, name another language, or drop the option and know that the block is set without it.