This is an implementation of the Unicode Standards Annex #14's line breaking algorithm. It provides a fast and convenient way to determine line breaking opportunities in text.
Note that this algorithm does not support break opportunities that require morphological analysis. In order to handle such cases, please consult a system that provides this kind of capability, such as a hyphenation algorithm.
Also note that this system is completely unaware of layouting decisions. Any kind of layouting decisions, such as which breaks to pick, how to space between words, how to handle bidirectionality, and what to do in emergency situations when there are no breaks on an overfull line are left up to the user.
The system passes all tests offered by the Unicode standard.
The system will compile binary database files on first load. Should anything go wrong during this process, a note is produced on load. If you would like to prevent this automated loading, push
*features* before loading. You can then manually load the database files when convenient through
Once loaded, you can produce a list of line breaks for a string with
list-breaks or break a string at every opportunity with
break-string. Typically however you will want to scan for the next break as you move along the string during layouting. To do so, create a breaker with
make-breaker, and call
next-break whenever the next line break opportunity is required.
In pseudo-code, that could look something like this. We assume the local nickname
(loop with breaker = (uax-14:make-breaker string) with start = 0 and last = 0 do (multiple-value-bind (pos mandatory) (uax-14:next-break breaker) (cond (mandatory (insert-break pos) (setf start pos)) ((beyond-extents-p start pos) (if (< last start) ; Force a break if we are overfull. (loop while (beyond-extents-p start pos) do (let ((next (find-last-fitting-cluster start))) (insert-break next) (setf start next)) finally (setf pos start)) (insert-break last)))) (setf last pos)))
The following files are from their corresponding external sources, last accessed on 2019.09.03:
At the time, Unicode 12.1 was considered the latest version.
The code in this project is largely based on the linebreak project by Devon Govett et al.
- No documentation provided.
Compiles the database files from their sources. This will load an optional part of the system and compile the database files to an efficient byte representation. If the compilation is successful, LOAD-DATABASES is called automatically. See *LINE-BREAK-DATABASE-FILE* See *PAIR-TABLE-FILE* See LOAD-DATABASES
Returns a breaker that can find line break opportunities in the given string. If the optional breaker argument is supplied, the supplied breaker is modified and reset to work with the new string instead. This allows you to re-use a breaker. Note that while you may pass a non-simple string, modifying this string without resetting any breaker using it will result in undefined behaviour. See BREAKER
Returns the next line breaking opportunity of the breaker, if any. Returns two values: POSITION --- The character index in the string at which the break is located, or NIL if no further breaks are possible. MANDATORY --- Whether the break must be made at this location. Note that there is always in the very least one break opportunity, namely at the end of the string. However, after consuming this break opportunity, NEXT-BREAK will return NIL. Note that you may have to insert additional line breaks as required by the layout constraints. See BREAKER