tclpdf

tclpdf 1.5 — what it does, and what it deliberately does not
Login

tclpdf 1.5 — what it does, and what it deliberately does not

A pure Tcl package that writes PDF documents, and reads enough of them to take a page over, continue a file, or say what a file contains. No compiler, no binary extension, no Tk. It needs Tcl 8.6.11 or newer and runs under Tcl 9 as well; zlib is a built-in command rather than a package, and tdom is needed for the XMP metadata packet of a PDF/A, PDF/UA or ZUGFeRD document — that is the one place it is required. The SVG module uses it too where it is present, and prefers it: measured, it parses 27 to 48 times faster than the parser this package brings along. Without tdom, SVG still works, and an entity expansion is refused by either parser before anything is drawn.

This note lists the whole of it. What a feature costs, what it was measured against and why a decision fell the way it did is in the manual at the feature itself — here it is one line each, so that the shape of the package is visible in one reading. It replaces the note of the same name for 1.4; what changed between the two is one road — runs inside a paragraph, with tags and Markdown as its notations and lists on top — and the numbers that grew with it.

In numbers

Version 1.5
Modules 108 Tcl files, loaded lazily — package require tclpdf registers all of them and loads one; a document that draws a rectangle loads nine, one with a line of text sixteen, and a script that never touches a picture, an SVG, an invoice, a signature or a foreign file never reads the other eighty-seven
Tests 4616, in 89 test files, green under Tcl 8.6.18 and Tcl 9.0.4
Examples 87 scripts writing 96 documents, every one run, validated and rendered on every acceptance
Documented commands 131 command forms — 120 methods of a document, 11 package commands — every command, option and value with a test of its own
Reviews eight norm-review rounds up to 1.3, the last two with twelve and fourteen reviewers reading the code against the standards it claims and 29 and 74 findings, all built, each with a test that a single reverted line turns red; 1.4 and 1.5 add features to that state rather than another round
Dependencies none for the core; tdom required for XMP and preferred for SVG, tzint if you want barcodes

make check is the full acceptance in one run: the suite under every interpreter present, every example, qpdf --check over every document, veraPDF against the profile each document itself claims, Mustangproject over every hybrid invoice, pdfsig over every signed or stamped file, every interactive form filled and read back, pdfinfo as a plain reader, the reference code of the packaged skill under both interpreters, whether the error code table in the manual is the one the source makes, and whether the manual is newer than its source.

Documents and pages

Pages in mm, pt, cm or in, in the formats a document names or in any size given; y counts from the top, which is the opposite of PDF's own convention and the one thing to know before writing the first line. All five page boxes (media, crop, bleed, trim, art), page rotation, page labels for the reader's page field, and the viewer preferences a file can ask for. Document information, an outline of bookmarks, links to a URL, to a page or to a structure element, and named destinations. Content is written straight out — a document of a thousand pages does not hold a thousand pages in memory.

Output goes to a file or to a channel, and a channel may be a pipe or a socket: a CGI response is written with the headers in front of it. Two writes of the same document are byte-identical, which is what makes a regression measurable at all. Every feature is checked against the PDF version the document is written as — a key that needs 1.5 in a 1.4 file is refused by name, not written and hoped for.

Graphics

Lines, rectangles, circles, ellipses, arcs in three styles, polygons, Bézier curves and arbitrary paths; dashing, line joins and caps, the miter limit, clipping, transformations, the graphics state stack. Fill and stroke in RGB, CMYK, grey, Lab, a spot colour as Separation, an ICC profile as ICCBased, and 148 colour names. Transparency as constant alpha, the sixteen blend modes, all seven shading types — axial, radial, function-based, the free-form and lattice-form Gouraud meshes, Coons and tensor patches — tiling patterns, overprint with its mode, and form XObjects: draw once, place as often as wanted, scaled, rotated about the point named, faded.

Layers (optional content) as a reader shows them: a group is declared, drawn into, switched on or off, and several groups can be tied into a radio set. A PDF/A document may carry them. A Tk canvas is drawn into a page as vectors — an example rather than a feature, because a canvas gives away everything a drawing command needs.

Every number that reaches the file is checked on the value the file will carry, not on the value given: a dash, a shading stop, a pattern step or a box that would round to a forbidden zero is refused before the operator is written, and a placement whose matrix would be singular is refused the same way.

Text and fonts

The fourteen standard faces without embedding anything, and four formats to embed: TrueType, subset to the glyphs actually used, as Type0/CIDFontType2 with Identity-H and a ToUnicode CMap that gives a character its own CID where two share a glyph; OpenType with CFF outlines, a bare CFF table, and Type 1, all whole; a TrueType collection is read by face. A Type 3 font whose glyphs are drawn by the caller — a glyph is a piece of the same drawing the page uses, so it may carry colour, a gradient or a picture. Colour fonts in three forms: the layers of a COLR version 0 face, the paint graph of a COLR version 1 face with its gradients, clips and compositions, and the PNG pictures of an sbix face such as Apple Color Emoji, each drawn into a Type 3 font so that a two-colour logo or an emoji sets like text; what each glyph was built from — bands, clips, a constant opacity or a soft mask — is reported, so a caller who needs a document free of soft masks can count them. Variable fonts are embedded as the instance asked for, with the metrics of that instance from MVAR and the vertical metrics from the variations. A fallback chain names the faces that may set what the family cannot, per character, and the line still comes out of a text extractor whole. Vertical writing with Identity-V.

Runs inside a paragraph, new in 1.5: a bold, italic or bold-italic word, an underlined or struck one, a link, and a heading of three levels in the middle of flowing text, with the line breaker running over all of it and measuring every piece in its own face; a run keeps the paragraph's size and colour, a heading is a paragraph of its own — larger, bold, never the last line of a page — and a list item is one too, indented with a bullet or a counted number in front of its first line. The primitive is a list of {text options} pairs; two notations become that list, tags (<b>, <i>, <u>, <s>, <a href>, <h1> to <h3>, <ul>, <ol>, <li>) and Markdown (**, *, ~~, [text](url), #, -, 1.), both read strictly — a run left open is refused with its position, never guessed at — and both proven to set the same bytes. An embedded family reaches its bold face through font family, and a face the family does not have is refused by name rather than set regular in silence. Underline and strikeout take their place and weight from the face's own tables, the AFM's for the standard fourteen. In a tagged document the headings are H1 to H3, a list is L with LI, Lbl and LBody, checked green by veraPDF for PDF/UA-1 and UA-2.

Single lines and paragraphs with justification, indents, a height limit that hands back what did not fit, a type area the document is given, columns balanced on the last page, text flowing around shapes, leader rows, text along a path, page numbers with a placeholder for the total, and a pagination that is linear in the length of the text. Kerning and the standard ligatures from the face's own tables, both on by default. GPOS in full for the horizontal line: pair and single adjustment, cursive attachment, chained contextual positioning, and marks placed by their anchors on a base, on the component of a ligature they belong to, and on another mark — so decomposed text comes out where composed text comes out, on the baseline and on a path alike. GSUB in full: ligatures, contextual and chained substitution, reverse chaining, with the feature stages HarfBuzz runs and measured against hb-shape word for word. Hyphenation by Liang patterns, per language, off by default, a two-level dictionary read as its two levels; the patterns are not shipped, because every published set carries a licence of its own and this package is MIT. A break hyphen in a tagged document is bracketed so that the word extracts whole.

Right-to-left text: Hebrew by order alone, Arabic with its contextual forms, ligatures and marks from the face's own tables, Nastaliq with its cursive descent, digits and brackets where UAX #9 puts them, marks in the order UTR #53 asks for. What cannot be set correctly is refused by name rather than drawn wrong — a line that mixes both directions in one call, and the scripts that need a shaper this package does not have.

Text is filled, outlined, both, or drawn nowhere at all — the last of those is how a scanned page carries its recognised text under the picture of itself.

Pictures

JPEG and PNG, both passed through unchanged wherever possible: a JPEG goes in as DCTDecode with the colour transform its markers call for, a PNG without alpha as FlateDecode with its own predictor, and a colour-key transparency stays a colour-key mask on the same pass-through. Only an alpha channel is decoded, into a soft mask. TIFF is read too — the format scans arrive in: Deflate, PackBits, CCITT Group 3 and 4 and Technote 2 JPEG are passed through as the PDF filter that undoes them, LZW is unpacked and rewritten as Flate so that PDF/A holds, and a picture whose compression carries state from one row to the next becomes one image per strip, stacked by the placement. Inline images where the caller asks for them.

A one-bit picture can be a stencil mask that lets the fill colour through its bits — one logo embedded once and placed in as many colours as wanted — and any picture can be the soft mask of another. An embedded ICC profile becomes the picture's colour space. Every format is asked what resolution it states about itself — a PNG pHYs chunk, a JFIF density, an Exif tag, a TIFF XResolution — so a 200 dpi scan comes out 219 mm wide rather than the 610 mm that assuming 72 dpi would give it. Two pictures are two pictures even when a hash says otherwise: the bytes decide. Every question that can turn a picture or its mask away is asked before an object is written.

SVG is drawn as vectors, not rasterised: shapes, paths, transforms, groups, reuse, nested viewports, viewBox, gradients with their stops and inheritance, clip paths and masks, group opacity, dash offset and miter limit, units from em to mm, switch, visibility, currentColor, and text set with the package's own font handling. What a drawing loses — a filter, a <style> block, a spreadMethod, an error in path data — is reported by name, never dropped in silence, and a drawing that is refused leaves the page untouched.

Tables

Column widths that follow the content or are given, cell wrapping and hyphenation per column or cell, alignment including on the decimal point, three themes, per-cell styling, row and column spans, page breaks with the header repeated and the footer where it belongs, a table too wide for the page broken across its width and tagged as one table, and a layout that can be asked for before anything is drawn.

Annotations and forms

Sticky notes, stamps, the four ways to mark a passage — highlight, underline, strikeout, squiggly — the five geometry annotations (line, square, circle, polygon, polyline) and a file attachment, each with an appearance the package draws itself, so no reader has to invent one. Interactive form fields of six kinds — text, check box, radio group, push button, list box, combo box — with their appearance streams written here, /NeedAppearances never set, a fillable field's face embedded whole so the recipient can type what the writer never wrote, and every field a structure element under PDF/UA. An annotation belongs to a page: inside a form script it is refused, because its rectangle would be in the wrong space.

Tagged, accessible and archivable

A structure tree with the standard types and the 2.0 additions, nesting checked as it is built and refused where ISO/TS 32005 forbids it, artifacts marked as such, alternative text, expansions, structure destinations, /Ref between an element and the one it points at, and a structure script that rolls back what it opened when it fails before it marked anything. PDF/A parts 2 and 3 in the levels B, U and A, with the output intent, the XMP packet and the colour guard that holds every colour space against the intent. PDF/UA-1 and PDF/UA-2, and the WTPDF profile for well-tagged documents; what UA-2 forbids — a Link element with two targets, a footnote without its reference pair, a page destination, a second caption — is refused rather than written. Every contributor to the XMP packet, PDF/UA, ZUGFeRD and a caller's own extension schema, shares one pdfaExtension:schemas bag, so the combinations validate together.

Electronic invoices

ZUGFeRD / Factur-X and Order-X as PDF/A-3B: the ICC profile, the output intent, the XMP extension schema, the embedded XML with its relationship, its modification date and the associated-files entry, all from one call; an XRECHNUNG profile under the name the standard reserves for it. The visible page and the attached XML are the same document, which is the whole point of the format — and the acceptance run reads the attachment back with Mustangproject to prove it.

Reading an existing PDF

One page of a foreign file is taken over as a form and placed as often as wanted — a letterhead is taken over instead of being rebuilt, its layers reach the document's own, and its marked content is stripped so that the page can be a described figure under PDF/UA. Both cross-reference flavours are read, the classic table and cross-reference streams with object streams, hybrid files included, incremental updates followed, every object identified by number and generation, every filter of the standard decoded and refused where its data are not what the encoding allows. A finished file is continued by an incremental update, so every byte already in it stays where it is. And what a file says about itself is answered as a dictionary: version, cross-reference flavour, revisions, identifier, encryption, pages and their sizes, the information dictionary, the PDF/A and PDF/UA claims, tagging, form type, output intents, attachments, signatures, certification — with fonts, pages, metadata and the fields of a form as separate calls, because they cost more.

Protection

Encryption with the standard security handler in revision 6 — AES-256 for strings, streams and the file as a whole, the metadata left in the clear on request — in pure Tcl, with no cryptographic library to build. Permissions are named rather than numbered. The deprecated revisions and RC4 are deliberately not offered.

Signatures: an approval signature over the whole file, invisible or with a place on a page and a description a screen reader can announce, signed as the document is written; a certification signature with its permission level; a two-stage road for signing that happens elsewhere and later; and a second signature onto an already signed file as an incremental update, so the first signature keeps the bytes it covers. What is signed comes from outside — a card, an HSM, a signing service, openssl — because that is where a key belongs.

Document timestamps, new in 1.4: an RFC 3161 token over every byte of a finished file — signed or not — appended as an incremental update, so a file proves when it existed without anyone having to prove who wrote it. The authority is the caller's choice with a public default: any RFC 3161 service over HTTP or HTTPS, or a command prefix that owns the transport entirely, which is also how the tests run without a network, against a local openssl ts. The reply is verified against the request's own digest and nonce before a byte is written, a stamped PDF/A file stays PDF/A — the acceptance run measures that — and any number of stamps may follow over the years, each covering everything before it: how an archive keeps a signature provable after its certificate expires.

What is deliberately left out

Not everything absent is missing. These are decisions, each with a reason at the feature in the manual:

Left out Why
Object and cross-reference streams for writing they compress the structure, not the content; measured on this package's own documents the gain is a fraction of a percent, and the cost is that the file stops being readable with an editor. Both are read
Linearisation it optimises the first page over a slow network for a reader that fetches byte ranges; it buys nothing for a document that is generated and then sent whole
LZW, CCITT, JBIG2 and JPX as writers every one of them is worse than Flate for what this package produces, and the good ones are patented history. LZW and CCITT are read, which is where they actually occur; JBIG2 and JPX pass through inside an imported page
Progressive JPEG, interlaced PNG PDF has no place for either; they would have to be decoded and re-encoded, which is exactly what the pass-through avoids
TIFF with an alpha channel, tiles, separate planes, BigTIFF a second stage of the TIFF reader that the stock measured did not contain
CalRGB and CalGray Lab covers what a calibrated space is asked for, and an ICC profile covers the rest
CBDT and OpenType SVG colour fonts two more formats for what COLR and sbix already do; the faces that carry them were measured and none of them lacks a COLR or sbix table
Alternate substitution (GSUB type 3) a choice the caller would have to make per glyph, and no face in the stock offers one worth an option
Multimedia, 3D, JavaScript, portfolios not a document, and each of them a security question of its own
Halftones and transfer functions a printer's setting, not a document's
Filling a foreign form reading its fields is there; filling would mean redrawing every appearance with a font the file brings and this package did not embed
Nested lists, and runs that change the size or the colour a list has one level, and a run changes the face and its decorations only: the size belongs to the paragraph, which is what keeps every line one height — a heading changes it as a paragraph of its own
Bidirectional text in one call (UAX #9) separate calls cover the address block, which is where mixed writing actually appears; the full algorithm was measured and priced, and the answer was that the stock does not need it
PDF/X there is no validator for it here, and building a promise that the acceptance run cannot measure is worse than not making it
GIF and BMP neither occurs in the stock this package was measured against; TIFF did, and was built
EPS as artwork it is PostScript, and drawing it means interpreting PostScript
Encryption revisions before 6, and RC4 broken, and offering them would invite their use

Where the limits are stated

Every refusal in this package names itself, the thing it refuses and the way out, and carries an error code that a trap can catch — TCLPDF FONT GLYPH, TCLPDF IMPORT, TCLPDF TIFF, TCLPDF COLORFONT, TCLPDF SVG and the rest are a documented contract, listed in the manual under Error codes in a table the source itself generates, fifty-eight topics long. That is the deliberate opposite of writing a file that looks like what was asked for and is not: a missing glyph, a colour space the output intent does not admit, a foreign file whose cross-reference chain runs in a circle or whose object stream names another object than its table, a picture that would draw nothing, a loop that would never return — each stops the call rather than reaching the reader, and stops it before a byte of the stream, a resource or a structure element is written.

Where something is knowingly left as it is, the manual says so beside the feature: how the two-level hyphenation differs from libhyphen's own program in about one German compound in twenty, that a mask whose data fail only while its streams are built leaves the ICC profile of its carrier as an unreferenced object, that a <style> block in an SVG is reported rather than read, and that a fillable field overrides an explicit -subset 1 on its face. A limit that is named is a limit; one that is not would be a defect.