Remove Line Breaks

Text copied from a PDF, an email client or a terminal often arrives with a line break every seventy or eighty characters, because the source was wrapped for a fixed width. Pasted into anything that wraps text itself, it becomes a column of short lines. Removing those breaks is the fix, and doing it bluntly destroys something you wanted to keep.

The distinction that matters is between a break inside a paragraph and a break between paragraphs. The first is an artefact of the original layout and should become a space. The second is structure and should survive. A rule that removes every line break turns a document into one continuous block, which is harder to read than what you started with and very tedious to undo by hand.

The usual heuristic is that a single break is a wrap and two consecutive breaks are a paragraph boundary, and it holds for most pasted text. It fails on documents that separate paragraphs with indentation rather than a blank line, and on text where every line is deliberately its own line: poetry, addresses, code, lists, tabular data. For those, joining lines is simply wrong and the heuristic will do it anyway.

Hyphenation is the PDF-specific trap. Text extracted from a justified PDF may contain words split across a line break with a hyphen, and joining naively leaves the hyphen in the middle of a word. Removing every such hyphen is also wrong, because some of them are real hyphens in genuinely hyphenated words that happened to land at a line end. This is the case that needs a human glance rather than a rule.

Windows and Unix line endings differ, and a document assembled from both sources contains both. That is invisible on screen and shows up as stray characters or double spacing after processing, so normalising line endings is worth doing as part of the same pass.

As always the text is processed in the tab and never uploaded, which matters because the most common source of wrapped text is a document someone emailed you.