Instead, generate them manually as we do for floating tables.
This removes our dependency on footnotehyper, and solves a
compatibility problem with `endfloat`.
Closes#11857.
Link targets containing square brackets broke the bracket link
syntax. Escape targets the way Emacs' org-link-escape does:
backslash-escape brackets and double backslash runs occurring before
a bracket or at the end of the target. Link descriptions cannot
contain escapes; instead, like org-link-make-string, insert a
zero-width space between consecutive closing brackets and before a
closing bracket at the end of the description.
Co-Authored-By: Claude <noreply@anthropic.com>
The overall table width used `round` while individual column widths
(in colSpecListToHtml) used `truncate`, so the table width could
exceed the sum of the column widths. Update test goldens.
Co-Authored-By: Claude <noreply@anthropic.com>
Org has no escape mechanism inside verbatim text, so `=code with ==`
did not parse as verbatim. Fall back to the equivalent `~...~`
delimiter when the content contains `=` (and no `~`).
Co-Authored-By: Claude <noreply@anthropic.com>
`#+begin_html` was removed in Org 9.0 (2016); export blocks are the
supported syntax. The org reader already parses `#+begin_export html`
to a raw HTML block, so round-tripping is preserved.
Co-Authored-By: Claude <noreply@anthropic.com>
D.prefixed only emits its prefix on a line that starts at column 0.
When a BlockQuote is the first block of a list item, the item's
marker and the quote's content are joined onto the same line, so
the quote's Prefixed doc begins mid-line and its "| " bar is
dropped on that first line, while later wrapped lines (which do
start at column 0) render it correctly.
Force the blockquote onto its own line with D.cr before applying
the prefix. D.cr is a no-op when already at the start of a line,
so top-level and nested blockquotes are unaffected.
Fixes#11804
Citeproc marks CSL-generated bibliographies with a hanging-indent
class and line-spacing/entry-spacing attributes. The HTML, LaTeX,
ConTeXt, and Ms writers act on these hints; the docx writer ignored
them, so bibliographies (e.g. APA) lost their hanging indent and
line spacing in docx output.
Apply the hints as paragraph properties (`w:ind`, `w:spacing`) in the
docx writer's Div handling. Paragraph properties are now sorted with
a proper `EG_PPrBase` ordering (`pPrTagOrder`) instead of the run
property ordering, keeping w:spacing before `w:ind` as required.
Closes#11871.
Org footnote labels may contain word-constituent characters, hyphens
and underscores. `noteMarker` (used for references and definitions)
already accepts these, but the inline note parser only accepted
alphanumeric characters, so `[fn:my-label: text]` was parsed as plain
text instead of a footnote.
Co-Authored-By: Claude <noreply@anthropic.com>
The `exportSubSuperscripts` setting was only consulted in
`simpleSubOrSuperText`, so `#+OPTIONS: ^:nil` still parsed the braced
(`a^{b}`) and parenthesized (`a^(b)`) forms. Conversely, `^:{}` (only
interpret braced expressions) was only handled by accident, because
`elispBoolean` treats `{}` as false.
Replace the boolean setting with a three-valued `SubSupOption`
(mirroring `org-export-with-sub-superscripts`, values `t`, `{}`, and
`nil`) and dispatch on it in `subOrSuperExpr`:
- `t`: all forms are interpreted (default, unchanged),
- `{}`: only braced expressions are interpreted,
- `nil`: sub-/superscript markup is left as plain text.
Co-Authored-By: Claude <noreply@anthropic.com>
The `shyHyphen` parser used `string "\\-"` without `try`, so any
backslash not followed by `-` that reached `specialStrings` (e.g.
`a \ b`, or an unclosed `\begin{...}` when the LaTeX fragment parser
gives up) consumed input on partial match and aborted the entire
document parse with an error.
Wrap the parser in `try` so other alternatives can still fire.
Co-Authored-By: Claude <noreply@anthropic.com>
Remove old ad hoc `paraToPlain` at the table cell level.
This should ensure that we don't get tables that mix
Plain and Para. Such tables tend to look funny when
rendered in docx and other formats.
Closes#11864.
Previously a paragraph that was, e.g. center-aligned would be
missing its default Body Text style.
Also ensure that any block sequence (list items, table cells,
block quote, div), First Paragraph is set for the first paragraph
in the item.
Closes#11867.
further tweaks
The friendly mediaPath was derived by percent-unescaping the key, so
distinct keys like "a%20b.png" and "a b.png" produced the same
mediaPath ("a b.png") and silently clobbered each other on
extraction (and inside docx/epub archives). Now the original name
is only kept if the key contains no percent sign, so mediaPath
equals the key and distinct keys yield distinct paths; anything
percent-encoded gets a content-hash name. Hashed names can only
coincide for identical contents, which is harmless.
Co-Authored-By: Claude <noreply@anthropic.com>
normalise does not remove redundant path components, so img/../a.png
and a.png were distinct keys: the same resource could be stored
twice, and a lookup by one spelling missed an insert by the other.
Use makeCanonical (as PandocPure's FileTree already does for its
path-indexed map), which also handles duplicate and trailing
slashes, replacing backslashes with slashes first. As a side
effect, paths with a collapsible ".." now keep a friendly name
instead of being renamed to a content hash.
Co-Authored-By: Claude <noreply@anthropic.com>
The insertMedia check used isInfixOf, so a harmless name like
foo..bar.png was silently renamed to its content hash. Check for an
actual ".." path component instead, treating both / and \ as
separators since the unescaped path may contain backslashes from
percent-encoding. Added tests.
Co-Authored-By: Claude <noreply@anthropic.com>
URI schemes are case-insensitive (RFC 3986, section 3.1), but several
places matched only lowercase "data:" or "file:":
- Text.Pandoc.URI.pBase64DataURI (also used by isURI)
- downloadOrRead: the fast-path prefix check and the parseURI
fallback for data: and file: URIs
- Text.Pandoc.App.Input.readSource: file:, http:, and https: URIs
given as input sources
- Text.Pandoc.MediaBag: canonicalize's fast path and insertMedia's
data-URI branch
Previously, e.g. DATA:image/gif;base64,... would be handed to openURL
and the image dropped, and a large uppercase data URI would also force
a full parseURI in MediaBag.canonicalize. Now such URIs are handled
the same as their lowercase equivalents. Added a test case.
Co-Authored-By: Claude <noreply@anthropic.com>
With raw_html disabled (the default for the HTML reader), an inline
<style> element is now ignored rather than passed through as a
RawInline; the raw_html-enabled behavior is still tested.
Co-Authored-By: Claude <noreply@anthropic.com>
The parser for \iftrue, \iffalse, and \ifmmode stopped at the first
\else or \fi it saw, so a nested conditional inside a skipped branch
(e.g. an unsupported primitive like \ifdim) made the outer
conditional end early and leaked tokens from the skipped branch.
Replace the sub-parse with a pure token scan that tracks nesting
depth, recognizing TeX's primitive conditionals as well as
\newif-defined ones, and (like TeX) does not expand tokens while
skipping a branch.
Co-Authored-By: Claude <noreply@anthropic.com>
The `opt` parsers in LaTeX.hs and Citation.hs parsed the bracketed
tokens in a separate runParserT call, discarding any state changes
(macro definitions, labels, notes) made while parsing them. Since
an optional argument is not a TeX group, such changes should persist.
Use parseFromToks, which runs the sub-parse in the same parser run.
This also removes a per-optional-argument state copy.
Co-Authored-By: Claude <noreply@anthropic.com>
Commit 47610fe3d (#10781) disabled the `--`, `---`, and `"` ligatures
inside `\texttt`, but single `` ` `` and `'` characters were still
converted to curly quotes, so `\texttt{it's}` produced code containing
a Unicode right single quotation mark. Gate those conversions on the
ligatures flag too, so quote characters inside `\texttt` stay as
typed. Update test/command/3958.md accordingly.
Co-Authored-By: Claude <noreply@anthropic.com>
`tokenize pos` already produces tokens positioned relative to pos,
but the result was then shifted again by pos's line and column,
roughly doubling the reported positions of tokens retokenized from
comments in URL arguments. Tokenize from the position just after
the '%' instead, and drop the extra shift.
Co-Authored-By: Claude <noreply@anthropic.com>
When tokenizing `##` not followed by digits, the continuation
restarted at column pos+1 instead of pos+2, shifting the source
positions of all subsequent tokens on the line one column left.
Co-Authored-By: Claude <noreply@anthropic.com>
Previously \newif\foobar would blindly do `T.drop 2` on the name,
defining a bogus \foobar -> \iffalse plus junk "obartrue"/"obarfalse"
macros; the \iffalse expansion could then swallow following text up
to \fi. Now newif checks the "if" prefix (as TeX requires) and
otherwise falls through to normal unknown-command handling.
Co-Authored-By: Claude <noreply@anthropic.com>
unescapeURL used T.splitOn "\\", which conflates consecutive
backslashes, so "\\" in a URL argument was left as two backslashes
even though '\' is in the escapable set. Rewrite as a direct scan:
a backslash followed by an escapable character is unescaped, and
"\\\\" now yields a single backslash.
Co-Authored-By: Claude <noreply@anthropic.com>
The string literal "\a0\x25FB" parses as BEL + '0' + WHITE MEDIUM
SQUARE; the intent (cf. qedSign in T.P.Readers.LaTeX.Math) was
"\xa0\x25FB", a non-breaking space before the square.
Co-Authored-By: Claude <noreply@anthropic.com>
Big performance improvements.
In addition, treatment of tables now aligns with asciidoctor;
the first line is only implicitly treated as a header if it
is on a line by itself, followed by a blank line.
role="img" and an aria-label copied from alt were being added to
every element with a src-like attribute, not just inlined SVGs.
That is redundant on img elements, whose alt already provides the
accessible name, and actively wrong on elements like video, audio,
iframe, and embed, where role="img" misrepresents interactive
content as a static image to assistive technology.
The attributes are only needed when an img element is replaced by
an inline svg element, since the alt text loses its meaning there;
restrict them to that path. Inline SVG output is unchanged.
Co-Authored-By: Claude <noreply@anthropic.com>
Rewrite the guard-isJust/fromJust combinations in pWebpSize as
case expressions, removing the partial fromJust calls, and drop
the now-unused Data.Maybe imports.
Also make checkDpi default to 72 for negative dpi values, not
just 0; a negative dpi would produce negative dimensions in
sizeInPoints.
Co-Authored-By: Claude <noreply@anthropic.com>
The ISO BMFF box parser treated the 32-bit size field as the
literal box size, but size == 1 means a 64-bit "largesize" field
follows the box type, and size == 0 means the box extends to the
end of the file. A new getBoxHeader helper handles all three
forms and is used everywhere box headers are read, fixing both
the header parsing and the size accounting in
searchAvifBoxesInRange, which subtracted the raw size field.
Also remove a double skip in searchAvifBoxes: tryParseDimensions
already skips the unconsumed contents of a box, so skipping
contentSize again caused parse failures whenever an unknown
top-level box preceded the meta box.
Co-Authored-By: Claude <noreply@anthropic.com>
AVIF files carry no resolution information in the boxes we parse, so
use the dpi from the writer options, as we already do for WebP and
SVG. With the default options this changes the assumed resolution
from 72 to 96 dpi.
Co-Authored-By: Claude <noreply@anthropic.com>
As with the earlier JPEG change: JuicyPixels' decodeImageWithMetadata
fully decodes paletted PNGs before returning any metadata (other PNG
color types are decoded lazily, but still allocate). Instead, read
the dimensions from the IHDR chunk and the resolution from the pHYs
chunk, using the same unit conversion as JuicyPixels, so results are
unchanged. If the header scan fails, we still fall back to the full
decoder.
Co-Authored-By: Claude <noreply@anthropic.com>
Instead of two full isInfixOf scans (one for "<svg", one for
"<SVG"), do a single pass that uses elemIndex (memchr) to jump
between '<' characters and checks both spellings at each one.
On a 26 MB non-SVG XML file this is 2.4x faster; on files with few
'<' characters, 60x.
Co-Authored-By: Claude <noreply@anthropic.com>
pPdfSize read the body of a compressed object stream with
`manyTill (satisfy (const True)) ...`, which builds a list with one
cons cell per byte and re-attempts the terminator parser at every
position. For a 4 MB stream this allocated about 3 GB, and larger
streams could exhaust memory. Replace it with a chunked scan for
the "endstream" keyword.
The trailing end-of-line before "endstream" is now passed along with
the stream data, which is harmless: zlib decompression stops at the
end of the compressed stream and ignores leftover input.
Co-Authored-By: Claude <noreply@anthropic.com>
Previously JPEG sizes were determined with JuicyPixels'
decodeImageWithMetadata, which fully decodes the image data even
though only the metadata is needed (for JPEG, both the metadata and
the Right/Left verdict depend on running the whole decoder). For a
4000x3000 photo this allocated over 1 GB.
Instead, scan the marker segments preceding the entropy-coded data:
the start-of-frame segment gives the dimensions, and the JFIF APP0
and Exif APP1 segments give the resolution. The dpi computation
follows JuicyPixels' metadata extraction (including Exif taking
precedence over JFIF), so results are unchanged; all JPEG files in
the test suite yield identical sizes. If the header scan fails, we
still fall back to the full decoder.
Co-Authored-By: Claude <noreply@anthropic.com>
The new Tests.ImageSize module constructs minimal binary files
in-memory to exercise imageType and imageSize for EPS, PDF (including
compressed object streams), SVG, EMF, WebP (lossless, lossy, and
extended), and AVIF (ispe and tkhd paths), covering regressions for
the bugs fixed in the preceding commits, plus fixture-based tests for
JPEG, PNG, and GIF.
[API change] ImageType now derives Eq.
Co-Authored-By: Claude <noreply@anthropic.com>
Support the features described in the LaTeX usrguide
(https://www.latex-project.org/help/documentation/usrguide.pdf):
- `\New/Renew/Provide/DeclareDocumentCommand` and the corresponding
`...DocumentEnvironment` commands, with argument specifiers
`m o O d D s t r R v e E b` and argument processors (`>{...}`:
`\TrimSpaces`, `\ReverseBoolean`, `\SplitArgument`, `\SplitList`).
Environment end code can refer to the environment's arguments.
- `\ProcessList`, `\UseName`, and `\ExpandArgs` (c, n, N letters).
- The conditionals `\IfNoValueTF`, `\IfValueTF`, `\IfBooleanTF`,
`IfBlankTF` (and variants with just `T` or `F`).
- `\New/Renew/DeclareCommandCopy` and `...EnvironmentCopy`.
- Case changing: `\MakeUppercase`, `\MakeLowercase`, and
`\MakeTitlecase`, `\NoCaseChange`,
`\Declare{Upper,Lower,Title}caseExclusions`; `\CaseSwitch` and
`\Declare*caseMapping`.
- The expandable evaluators `\inteval` and `\fpeval` (a practical
arithmetic subset); `\dimeval` and `\skipeval` substitute their
expression unevaluated.
- `\expandableinput`.
- `\ExplSyntaxOn/Off` regions are tokenized with `:` and `_` as
letters, so expl3 code can be skipped cleanly; `\ShowCommand` and
the ltkeys commands (`\DeclareKeys`, `\SetKeys`,
`\ProcessKeyOptions`, ...) are parsed and ignored.
- Some constants, such as `c_ampersand_str`, are supported in
expl3 regions.
Adds new `ArgSpec` constructors to non-exported module Text.Pandoc.TeX.
New tests in `test/command/latex3-commands.md` and
`test/command/latex3-usrguide.md`. (The latter correspond to the
examples in the user guide and were confirmed against LaTeX output
when possible.)
Closes#7540.
Co-Authored-By: Claude <noreply@anthropic.com>
Previously only `form: "year"` was special-cased (to SuppressAuthor);
all other forms fell through to NormalCitation. As a result,
`#cite(<key>, form: "prose")` rendered as a parenthetical citation
"(Doe and Smith 2020)" instead of the narrative "Doe and Smith (2020)".
Typst's `author` and `full` forms have no pandoc citation mode
equivalent and still fall back to NormalCitation.
Adds reader test coverage for all four cite forms.
`stringify` does a full tree rebuild (`walk fixInlines`) before
querying, and discards `Note` contents only after rebuilding them.
The new `stringifyInlines :: Foldable t => t Inline -> Text` produces
the same result in a single pass with no rebuild; in a new benchmark
it is about 6x faster on a large inline sequence. Use it in
`inlineListToIdentifier`, `figureDiv`, and `formatCode`.
Also make `stringify` accumulate `[Text]` and concatenate once at the
end, instead of mappending `Text` at every node, and add a QuickCheck
property verifying that `stringifyInlines` agrees with `stringify`.
Co-Authored-By: Claude <noreply@anthropic.com>
Previously it was not looking in the right directory
and not recursing.
The function now alphabetizes its results so that test output
is the same with or without `embed_data_files`.
John MacFarlane