Use withFile instead of openFile so the handle is closed even if
reading throws. (B.hGetContents closes the handle on successful
end-of-file, but an exception mid-read would previously leak it.)
Co-Authored-By: Claude <noreply@anthropic.com>
toText and toTextLazy unconditionally ran a CR-removing filter over
the input, allocating a full copy of the document even in the common
case where no CRs are present. Check for a CR first (B.elem, a fast
memchr) and reuse the input buffer unchanged if none is found; for
the lazy variant, do this chunk-wise to preserve laziness.
On a 10 MB LF-only input this makes toText over 4x faster; when CRs
are present the extra scan is not measurable.
Co-Authored-By: Claude <noreply@anthropic.com>
With raw_html disabled (the default for the HTML reader), an inline
<style> element is now ignored rather than passed through as a
RawInline; the raw_html-enabled behavior is still tested.
Co-Authored-By: Claude <noreply@anthropic.com>
pCodeBlock put the code element's attributes first when merging,
relying on the old last-duplicate-wins behavior of toStringAttr to
give the pre element's attributes precedence. Now that the first
duplicate wins, put pre's attributes first.
Co-Authored-By: Claude <noreply@anthropic.com>
The inline dispatcher already knows which span-like element it is
looking at, so there is no need for pSpanLike to try a parser for
every element in htmlSpanLikeElements.
Co-Authored-By: Claude <noreply@anthropic.com>
A space was appended to the remaining input to guarantee a
TagPosition token after the parsed tag; since the input is a strict
Text, this copied the entire remaining input every time htmlTag was
called (e.g. for every inline HTML tag in a markdown document),
giving quadratic behavior in tag-dense documents. Instead, handle
the case where the tag is the final token by computing the
end-of-input position directly.
Co-Authored-By: Claude <noreply@anthropic.com>
pIframe fetches the iframe's src and parses it recursively; a
cycle of iframes embedding each other caused infinite recursion.
Track the nesting depth and skip iframes (with a warning) more
than 5 levels deep.
Co-Authored-By: Claude <noreply@anthropic.com>
Previously `<ol class="fancy lower-roman">` got DefaultStyle,
because the whole class attribute was compared against the known
style names. Check each class individually. As a side effect, an
unrecognized class no longer prevents falling back to the style
attribute.
Co-Authored-By: Claude <noreply@anthropic.com>
The ReferenceNotFound warning was logged after the whole document
had been parsed, so it always pointed at the end of the input.
Record the position of the first reference to each note and use it
in the warning.
Co-Authored-By: Claude <noreply@anthropic.com>
Avoids a linear scan per noteref (quadratic overall in the number
of notes). As before, the most recently parsed note with a given
identifier wins.
Co-Authored-By: Claude <noreply@anthropic.com>
Inline <style> elements (invalid but common, see #10643) were
always turned into RawInline, even with the raw_html extension
disabled. Skip them with a warning in that case, as is already
done for block-level style elements.
Co-Authored-By: Claude <noreply@anthropic.com>
toStringAttr deduplicated attributes with a right fold, so the last
occurrence of a duplicated attribute was kept. HTML specifies that
the first occurrence wins (and that lang takes precedence over
xml:lang). Deduplicate left to right with a seen-set instead.
Co-Authored-By: Claude <noreply@anthropic.com>
The column specs are zipped with the computed alignments, so a
<col> list shorter than the actual number of columns truncated the
colspecs, and the table builder then dropped the cells in the extra
columns. Pad the width list with ColWidthDefault instead.
Co-Authored-By: Claude <noreply@anthropic.com>
The </tr> closing tag is optional in HTML. Previously pRow and
pHeaderRow required it, so a table using the omitted form was
silently degraded to a sequence of plain blocks. The closing tag
is now optional; pHeaderRow instead checks that the row isn't
followed by a <td> cell, so body rows are still parsed by pRow.
Co-Authored-By: Claude <noreply@anthropic.com>
`<input>` is a void element, so `<input type="checkbox">` produces
no TagClose token. Previously pCheckbox required one and failed
after consuming the open tag, silently destroying the surrounding
list structure. Use pSelfClosing, as with `<br>`.
Co-Authored-By: Claude <noreply@anthropic.com>
The parser for \iftrue, \iffalse, and \ifmmode stopped at the first
\else or \fi it saw, so a nested conditional inside a skipped branch
(e.g. an unsupported primitive like \ifdim) made the outer
conditional end early and leaked tokens from the skipped branch.
Replace the sub-parse with a pure token scan that tracks nesting
depth, recognizing TeX's primitive conditionals as well as
\newif-defined ones, and (like TeX) does not expand tokens while
skipping a branch.
Co-Authored-By: Claude <noreply@anthropic.com>
The `opt` parsers in LaTeX.hs and Citation.hs parsed the bracketed
tokens in a separate runParserT call, discarding any state changes
(macro definitions, labels, notes) made while parsing them. Since
an optional argument is not a TeX group, such changes should persist.
Use parseFromToks, which runs the sub-parse in the same parser run.
This also removes a per-optional-argument state copy.
Co-Authored-By: Claude <noreply@anthropic.com>
Commit 47610fe3d (#10781) disabled the `--`, `---`, and `"` ligatures
inside `\texttt`, but single `` ` `` and `'` characters were still
converted to curly quotes, so `\texttt{it's}` produced code containing
a Unicode right single quotation mark. Gate those conversions on the
ligatures flag too, so quote characters inside `\texttt` stay as
typed. Update test/command/3958.md accordingly.
Co-Authored-By: Claude <noreply@anthropic.com>
The dispatch maps (inlineCommands, blockCommands, environments) have
types that mention the parser monad, so they are compiled as functions
of the PandocMonad dictionary and were being rebuilt from scratch --
hundreds of Map entries -- every time a command or environment was
dispatched. SPECIALIZE pragmas can't fix this on the CLI path, since
the reader table abstracts over the monad dictionary.
Instead, add a ReaderT layer to the LP monad carrying a LaTeXEnv
record with the three dispatch maps, built once per parse in readLaTeX
(and once per chunk in rawLaTeXParser's callers), and consult it via
askEnv at the dispatch sites. Internal `lift $ runParserT` sub-parses
share the environment automatically.
Benchmark (1.8MB LaTeX file): allocations drop from 17.85GB to 9.97GB
and wall time from ~4.7s to ~3.4s. On a stress test of 100k \emph
commands, allocations drop from 36GB to 10.5GB.
Co-Authored-By: Claude <noreply@anthropic.com>
doMacros runs before every token consumption; when the head of the
stream is not a control sequence no expansion can occur, so avoid
scanning and rewriting the input stream (which allocated a new
TokStream per consumed token) in that case.
Co-Authored-By: Claude <noreply@anthropic.com>
peekTok went through lookAhead (satisfyTok (const True)), paying
for tokenPrim position bookkeeping, the raw-token state update, and
lookAhead state save/restore on every call; peekTok is used by the
main inline and block dispatchers for every element. Just examine
the head of the input stream after expanding macros.
Co-Authored-By: Claude <noreply@anthropic.com>
macroDef is tried for every control sequence in inline and block
context, and previously attempted ~30 controlSeq alternatives in
turn before failing. Peek at the next control sequence and check
membership in a Set of macro-defining command names first, so the
common case fails after a single lookup.
Co-Authored-By: Claude <noreply@anthropic.com>
untokenize used foldr with Text (<>), which repeatedly copies the
accumulated suffix and is quadratic in the total text length. Use
T.concat on a list of chunks instead, preserving the space
insertion after control words (#5836). Simplify untoken to return
the token text directly (equivalent to the old behavior with an
empty accumulator).
Co-Authored-By: Claude <noreply@anthropic.com>
Previously satisfyTok updated an IntMap of reversed token lists,
one entry per active withRaw scope, so every consumed token cost
O(number of nested withRaw scopes) time and allocation, and the
state was updated even when no withRaw was pending. Replace the
IntMap with a single accumulated list plus a count and a counter of
active scopes: satisfyTok now conses at most once per token (and
does nothing when no scope is active); withRaw takes its slice of
the accumulated list by count difference.
Co-Authored-By: Claude <noreply@anthropic.com>
`tokenize pos` already produces tokens positioned relative to pos,
but the result was then shifted again by pos's line and column,
roughly doubling the reported positions of tokens retokenized from
comments in URL arguments. Tokenize from the position just after
the '%' instead, and drop the extra shift.
Co-Authored-By: Claude <noreply@anthropic.com>
When tokenizing `##` not followed by digits, the continuation
restarted at column pos+1 instead of pos+2, shifting the source
positions of all subsequent tokens on the line one column left.
Co-Authored-By: Claude <noreply@anthropic.com>
Previously \newif\foobar would blindly do `T.drop 2` on the name,
defining a bogus \foobar -> \iffalse plus junk "obartrue"/"obarfalse"
macros; the \iffalse expansion could then swallow following text up
to \fi. Now newif checks the "if" prefix (as TeX requires) and
otherwise falls through to normal unknown-command handling.
Co-Authored-By: Claude <noreply@anthropic.com>
unescapeURL used T.splitOn "\\", which conflates consecutive
backslashes, so "\\" in a URL argument was left as two backslashes
even though '\' is in the escapable set. Rewrite as a direct scan:
a backslash followed by an escapable character is unescaped, and
"\\\\" now yields a single backslash.
Co-Authored-By: Claude <noreply@anthropic.com>
The string literal "\a0\x25FB" parses as BEL + '0' + WHITE MEDIUM
SQUARE; the intent (cf. qedSign in T.P.Readers.LaTeX.Math) was
"\xa0\x25FB", a non-breaking space before the square.
Co-Authored-By: Claude <noreply@anthropic.com>
Big performance improvements.
In addition, treatment of tables now aligns with asciidoctor;
the first line is only implicitly treated as a header if it
is on a line by itself, followed by a blank line.
The ODT reader was written in an idiosyncratic arrows-based style,
using a custom ArrowState arrow and dozens of special combinators.
Replace this with an ordinary monad:
type XMLConverter nsID extraState
= ExceptT () (State (XMLConverterState nsID extraState))
This preserves the original semantics (state changes persist through
failure; recovery via Alternative's <|>) while allowing all readers
to be written in plain do/applicative notation.
- Rewrite Generic/XMLConverter.hs around the new monad; replace the
parent-element stack with a single currentElement field, and make
matchContent matchers simple (namespace, name, reader) triples
whose results are mconcat'd in document order.
- Port StyleReader.hs and ContentReader.hs to monadic style.
- constructList now runs its child reader once instead of twice.
- Delete Arrows/State.hs, Arrows/Utils.hs, and Base.hs; trim
Generic/Fallible.hs and Generic/Utils.hs to what is still used.
Net -994 lines. No changes to test output.
Co-Authored-By: Claude <noreply@anthropic.com>
role="img" and an aria-label copied from alt were being added to
every element with a src-like attribute, not just inlined SVGs.
That is redundant on img elements, whose alt already provides the
accessible name, and actively wrong on elements like video, audio,
iframe, and embed, where role="img" misrepresents interactive
content as a static image to assistive technology.
The attributes are only needed when an img element is replaced by
an inline svg element, since the alt text loses its meaning there;
restrict them to that path. Inline SVG output is unchanged.
Co-Authored-By: Claude <noreply@anthropic.com>
