Why do I need a shaping engine?: HarfBuzz Manual

Why do I need a shaping engine?

Text shaping is an integral part of preparing text for display. Before a Unicode sequence can be rendered, the codepoints in the sequence must be mapped to the corresponding glyphs provided in the font, and those glyphs must be positioned correctly relative to each other. For many of the scripts supported in Unicode, these steps involve script-specific layout rules, including complex joining, reordering, and positioning behavior. Implementing these rules is the job of the shaping engine.

Text shaping is a fairly low-level operation. HarfBuzz is used directly by text-handling libraries like Pango, as well as by the layout engines in Firefox, LibreOffice, and Chromium. Unless you are writing one of these layout engines yourself, you will probably not need to use HarfBuzz: normally, a layout engine, toolkit, or other library will turn text into glyphs for you.

However, if you are writing a layout engine or graphics library yourself, then you will need to perform text shaping, and this is where HarfBuzz can help you.

Here are some specific scenarios where a text-shaping engine like HarfBuzz helps you:

OpenType fonts contain a set of glyphs (that is, shapes to represent the letters, numbers, punctuation marks, and all other symbols), which are indexed by a glyph ID.

A particular glyph ID within the font does not necessarily correlate to a predictable Unicode codepoint. For instance, some fonts have the letter "a" as glyph ID 1, but many others do not. In order to retrieve the right glyph from the font to display "a", you need to consult the table inside the font (the cmap table) that maps Unicode codepoints to glyph IDs. In other words, text shaping turns codepoints into glyph IDs.
Many OpenType fonts contain ligatures: combinations of characters that are rendered as a single unit. For instance, it is common for the "f, i" letter sequence to appear in print as the single ligature glyph "ﬁ".

Whether you should render an "f, i" sequence as fi or as "ﬁ" does not depend on the input text. Instead, it depends on the whether or not the font includes an "ﬁ" glyph and on the level of ligature application you wish to perform. The font and the amount of ligature application used are under your control. In other words, text shaping involves querying the font's ligature tables and determining what substitutions should be made.
While ligatures like "ﬁ" are optional typographic refinements, some languages require certain substitutions to be made in order to display text correctly.

For example, in Tamil, when the letter "TTA" (ட) letter is followed by the vowel sign "U" (ு), the pair must be replaced by the single glyph "டு". The sequence of Unicode characters "ட,ு" needs to be substituted with a single "டு" glyph from the font.

But "டு" does not have a Unicode codepoint. To find this glyph, you need to consult the table inside the font (the GSUB table) that contains substitution information. In other words, text shaping chooses the correct glyph for a sequence of characters provided.
Similarly, each Arabic character has four different variants corresponding to the different positions it might appear in within a sequence. Inside a font, there will be separate glyphs for the initial, medial, final, and isolated forms of each letter, each at a different glyph ID.

Unicode only assigns one codepoint per character, so a Unicode string will not tell you which glyph variant to use for each character. To decide, you need to analyze the whole string and determine the appropriate glyph for each character based on its position. In other words, text shaping chooses the correct form of the letter by its position and returns the correct glyph from the font.
Other languages involve marks and accents that need to be rendered in specific positions relative a base character. For instance, the Moldovan language includes the Cyrillic letter "zhe" (ж) with a breve accent, like so: "ӂ".

Some fonts will provide this character as a single zhe-with-breve glyph, but other fonts will not and, instead, will expect the rendering engine to form the character by superimposing the separate "ж" and "˘" glyphs.

But exactly where you should draw the breve depends on the height and width of the preceding zhe glyph. To find the right position, you need to consult the table inside the font (the GPOS table) that contains positioning information. In other words, text shaping tells you whether you have a precomposed glyph within your font or if you need to compose a glyph yourself out of combining marks—and, if so, where to position those marks.

If tasks like these are something that you need to do, then you need a text shaping engine. You could use Uniscribe if you are writing Windows software; you could use CoreText on macOS; or you could use HarfBuzz.

In the rest of this manual, the text will assume that the reader is that implementor of a text-layout engine.