Commit Graph
1343 Commits
Author SHA1 Message Date
John MacFarlane 7cf4d41fe5 Use compactifyTable for tables produced by all readers.
Remove old ad hoc `paraToPlain` at the table cell level.

This should ensure that we don't get tables that mix
Plain and Para. Such tables tend to look funny when
rendered in docx and other formats.

Closes #11864.
2026-09-12 20:02:41 -07:00
John MacFarlane 0e1930aced rebase_relative_paths: recognize URLs with unknown schemes.
Closes #11858.
2026-09-10 22:14:26 -07:00
John MacFarlaneandClaude 0e399cc799 Update test for raw_html-aware inline <style> handling.
With raw_html disabled (the default for the HTML reader), an inline
<style> element is now ignored rather than passed through as a
RawInline; the raw_html-enabled behavior is still tested.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-09-09 03:12:53 +00:00
John MacFarlaneandClaude cd949780c6 LaTeX reader: don't discard state changes made in optional arguments.
The `opt` parsers in LaTeX.hs and Citation.hs parsed the bracketed
tokens in a separate runParserT call, discarding any state changes
(macro definitions, labels, notes) made while parsing them.  Since
an optional argument is not a TeX group, such changes should persist.
Use parseFromToks, which runs the sub-parse in the same parser run.
This also removes a per-optional-argument state copy.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-09-08 16:10:11 -07:00
John MacFarlaneandClaude 63a1710cd4 LaTeX reader: keep ASCII quotes when ligatures are disabled.
Commit 47610fe3d (#10781) disabled the `--`, `---`, and `"` ligatures
inside `\texttt`, but single `` ` `` and `'` characters were still
converted to curly quotes, so `\texttt{it's}` produced code containing
a Unicode right single quotation mark.  Gate those conversions on the
ligatures flag too, so quote characters inside `\texttt` stay as
typed.  Update test/command/3958.md accordingly.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-09-08 16:10:11 -07:00
John MacFarlaneandClaude 113568e6e2 LaTeX reader: fix doubled source positions in retokenizeComment.
`tokenize pos` already produces tokens positioned relative to pos,
but the result was then shifted again by pos's line and column,
roughly doubling the reported positions of tokens retokenized from
comments in URL arguments.  Tokenize from the position just after
the '%' instead, and drop the extra shift.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-09-08 16:10:11 -07:00
John MacFarlaneandClaude 1cffc95ceb LaTeX reader: fix position off-by-one after ## in tokenizer.
When tokenizing `##` not followed by digits, the continuation
restarted at column pos+1 instead of pos+2, shifting the source
positions of all subsequent tokens on the line one column left.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-09-08 16:10:11 -07:00
John MacFarlane c54ad9ef75 Typst writer: emit label for bare table if present.
Previously a label was only emitted if the table was figurized
(and not `.typst:no-figure`).

Closes #11849.
2026-09-08 08:34:30 -07:00
John MacFarlaneandClaude a93f2fdc7f SelfContained: only add role and aria-label when inlining SVGs.
role="img" and an aria-label copied from alt were being added to
every element with a src-like attribute, not just inlined SVGs.
That is redundant on img elements, whose alt already provides the
accessible name, and actively wrong on elements like video, audio,
iframe, and embed, where role="img" misrepresents interactive
content as a static image to assistive technology.

The attributes are only needed when an img element is replaced by
an inline svg element, since the alt text loses its meaning there;
restrict them to that path.  Inline SVG output is unchanged.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-09-07 15:41:18 +00:00
John MacFarlaneandClaude c585c71118 LaTeX reader: support LaTeX3 (xparse) document commands.
Support the features described in the LaTeX usrguide
(https://www.latex-project.org/help/documentation/usrguide.pdf):

- `\New/Renew/Provide/DeclareDocumentCommand` and the corresponding
  `...DocumentEnvironment` commands, with argument specifiers
  `m o O d D s t r R v e E b` and argument processors (`>{...}`:
  `\TrimSpaces`, `\ReverseBoolean`, `\SplitArgument`, `\SplitList`).
  Environment end code can refer to the environment's arguments.
- `\ProcessList`, `\UseName`, and `\ExpandArgs` (c, n, N letters).
- The conditionals `\IfNoValueTF`, `\IfValueTF`, `\IfBooleanTF`,
  `IfBlankTF` (and variants with just `T` or `F`).
- `\New/Renew/DeclareCommandCopy` and `...EnvironmentCopy`.
- Case changing: `\MakeUppercase`, `\MakeLowercase`, and
  `\MakeTitlecase`, `\NoCaseChange`,
  `\Declare{Upper,Lower,Title}caseExclusions`; `\CaseSwitch` and
  `\Declare*caseMapping`.
- The expandable evaluators `\inteval` and `\fpeval` (a practical
  arithmetic subset); `\dimeval` and `\skipeval` substitute their
  expression unevaluated.
- `\expandableinput`.
- `\ExplSyntaxOn/Off` regions are tokenized with `:` and `_` as
  letters, so expl3 code can be skipped cleanly; `\ShowCommand` and
  the ltkeys commands (`\DeclareKeys`, `\SetKeys`,
  `\ProcessKeyOptions`, ...) are parsed and ignored.
- Some constants, such as `c_ampersand_str`, are supported in
  expl3 regions.

Adds new `ArgSpec` constructors to non-exported module Text.Pandoc.TeX.

New tests in `test/command/latex3-commands.md` and
`test/command/latex3-usrguide.md`. (The latter correspond to the
examples in the user guide and were confirmed against LaTeX output
when possible.)

Closes #7540.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-09-06 13:28:54 -07:00
John MacFarlane 6accb0b9f7 Typst writer: omit blank line at end of block.
This is just a cosmetic change; it is not semantically
significant.

Closes #11844.
2026-09-04 09:37:08 -07:00
John MacFarlane 3b91fcdd2c Fix Text.Pandoc.Data.getDataFileNames with -embed_data_files.
Previously it was not looking in the right directory
and not recursing.

The function now alphabetizes its results so that test output
is the same with or without `embed_data_files`.
2026-09-01 22:45:21 -07:00
Hendrik Erz fa0dfdf0e1 Markdown reader: case-insensitive alert keywords (#11836)
The keywords (e.g. `TIP`) for GFM-style alerts need not be all caps.
This commit fixes that. It also adds the class `alert` to the produced
Divs to allow for namespacing.

This commit changes the emitted classes from the alert extension
of the basic Markdown reader to add the class `alert` to the
produced DIVs to allow for namespacing.
2026-08-29 17:25:41 -07:00
John MacFarlane 6708521943 Rename some command tests. 2026-08-28 09:30:20 -07:00
John MacFarlane 9dfad4005d MediaWiki writer: put nowiki around possible URLs.
Otherwise they get linkified automatically.
Closes #11834.
2026-08-27 19:55:16 -07:00
John MacFarlane 2f103b3c76 Docx reader: handle case where non-header rows come before header rows.
In this case we'll treat them all as the header in order to
preserve the order.

Added test case for #11833.
2026-08-27 14:51:13 -07:00
John MacFarlane 9260d17d21 Make mathml the default math-method.
Closes #11751.
2026-08-26 09:51:03 -07:00
John MacFarlane c73a35a2e7 Add --math-method option.
This replaces (now deprecated but still functional) options
`--mathml`, `--mathjax`, `--gladtex`, `--katex`, `--webtex`.

The `plain` style can now be specified explicitly.

In defaults files, `html-math-method` is now
`math-method` (though `html-math-method` will still work).

Text.Pandoc.App.Opt:

- In unexported type CompletionKind, add MathMethods.
- In Opt, change `optHTMLMathMethod` to `optMathMethod`. [API  change]

Text.Pandoc.Options:

- Rename HTMLMathMethod type to MathMethod. [API change]
- Rename `writerHTMLMathMethod` field of WriterOptions to
  `writerMathMethod`. [API change]

Text.Pandoc.App.Completion:

- Clean up code.
- Add support for `--math-method`.
2026-08-26 09:46:48 -07:00
John MacFarlane 7b866613f6 ODT/OpenDocument writers: support RTL text direction.
RTL is now properly handled in these cases:

- `dir: rtl` (or `ltr`) in document metadata
- an RTL `lang` in metadata (e.g. `he`, `ar`), unless overridden
  by `dir`
- a `dir` attribute on a Div

The OpenDocument writer tracks the active writing mode and, when it
is RTL, emits automatic paragraph styles with
`style:writing-mode="rl-tb"` derived from the usual named styles
(deduplicated per parent style).  `fo:text-align` is also used
to get the matching text alignment.

Code blocks remain ltr regardless.

Closes #11301.

With some help from Claude Fable.
2026-08-26 08:49:07 -07:00
John MacFarlane 5727e10827 Markdown reader: Add a syntax for resetting example list counter.
This is needed often at the beginning of a chapter.

Closes #10940.
2026-08-24 12:07:19 -07:00
wzy 0ce83fb133 Add --completion={bash,zsh,fish} (#11818)
Make `--bash-completion` an alias of `--completion=bash`.

Reorganize option parsing code using a new data structure OptionSpec,
which can encode the type of completion needed by each option.

Add new unexported module Text.Pandoc.Completion.

Unexported module Text.Pandoc.CommandLineOptions: export OptionSpec.

Closes #8542.
2026-08-20 13:46:25 -07:00
John MacFarlane 5f37c6781b Typst reader: 'block' should also be block-level.
See 185c29b4b2 which claims that
`block` can occur in inline contexts. I can no longer confirm
this and suspect that it may have been an unintended behavior
of earlier versions of typst, now fixed.

Closes #11814.
2026-08-17 10:01:58 -07:00
John MacFarlane dd970030ba HTML reader: handle pre without code.
We preserve whitespace as nonbreaking spaces.

Closes #11810.
2026-08-17 09:27:38 -07:00
John MacFarlane 6107a26945 JATS writer: fix illegal use of p inside p.
Closes #11809.
2026-08-16 10:44:18 -07:00
John MacFarlane 3acfe775e6 LaTeX writer: omit empty caption on unnumbered tables.
(Even if they have an identifier.)

Closes #11795.
2026-08-11 11:22:12 -07:00
John MacFarlane 7c35a540d1 Revert "Typst writer: fix escaping of quotes."
This reverts commit 04848401d5.

However, we retain the test for #11463, which didn't actually
need the reverted commit.

Closes #11788.
2026-08-11 09:35:17 -07:00
John MacFarlane 87bc462713 Typst writer: make writer sensitive to typst-label attribute.
If this attribute is set, the writer will output a label.
(This overrides labels based on identifiers, when identifiers
are present.)

Note that labels in typst need not be unique and may be used to
style elements using show rules.  For example, one can attach
an `excerpt` label to all Divs that are excerpts, and style this
using a show rule like `#show <excerpt>: set text(blue)`.

See discussion at #11794.
2026-08-07 10:39:33 -07:00
John MacFarlane 12a44fbb98 Markdown raw HTML: fix handling of void hr element.
Closes #11793.

Added `voidTags` to unexported module
Text.Pandoc.Readers.HTML.TagCategories.
2026-08-05 07:18:12 -07:00
John MacFarlane db8d64fe6f Markdown writer: Fix ordered list bug.
Previously ordered lists starting with a code block did not
round trip. Pandoc uses an indented code block, and according
to commonmark rules only one space is gobbled after the list
marker; however, pandoc added two.  This has now been fixed.
An incidental change fixes an oddity in markua list indentation.

Closes #11762.
2026-07-14 17:34:15 +02:00
ce37e24c6a PPTX writer: keep tables and surrounding text on same slide (#11411)
Previously, when a slide contained bullet points before and after a
table, content would be split across multiple slides or use a two-column
layout that didn't preserve the source order.

This change:

1. Modified `splitBlocks'` in Presentation.hs to continue accumulating
   content after tables instead of immediately splitting to a new slide.

2. When a slide has a table with surrounding text, use ContentSlide
   (single column) instead of ContentWithCaptionSlide (two columns)
   to preserve content order.

3. Added vertical stacking in Output.hs (shapesToElementsStacked) to
   properly position multiple shapes (tables + text) within a content
   area. Space is allocated proportionally based on content size
   (paragraphs and table rows).

Closes #11433.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Co-authored-by: Chris Callison-Burch <ccb+github@upenn.edu>
2026-07-12 12:51:11 +02:00
Chirag Dhamange 432414c236 Fix: GFM empty task list conversion to ballot (#11744)
Extend `taskListItemToAscii` to cover empty task lists.
Closes #11599.
2026-07-12 12:01:23 +02:00
John MacFarlane 780e605c95 Texinfo writer: fix escaping in links.
Previously we dropped periods, commas, etc. This was a regression
from 2024. All we really need to do is escape the comma.

Closes #11758.
2026-07-11 10:57:33 +02:00
John MacFarlane 3ecd95bf00 LaTeX writer: Provide a way to use table instead of longtable.
When the `float` class is attached to a table, a standard
floating `table` will be generated instead of a `longtable`.
Placement can be specified via the `latex-placement` attribute.
This will help, especially, those who use two-column layouts,
since `longtable` is incompatible with these.

Small change to the default LaTeX template: the `caption`
package is included and a default spacing set between the caption
and the table.

Closes #1023.
2026-07-10 16:38:03 +02:00
John MacFarlane f6edf03303 Shared: Improve makeSections.
Previously it would sometimes create doubled section divs.
By leaving the inner heading's id on the heading itself, and
then consolidating the inner and outer section divs, we can
avoid this undesirable result.

Closes #11745.
2026-07-07 20:58:33 +02:00
John MacFarlane db7efddf74 Texinfo writer: fix spacing bug around HorizontalRule.
Closes #11749.
2026-07-07 16:58:44 +02:00
Can H. Tartanoglu 57639391fc ODT writer: image anchoring, percent dimensions, text width resolution (#11746)
Inline images now emit `text:anchor-type=as-char` on their `draw:frame`,
anchoring the frame as a character so it flows with the surrounding text.
Without this, consumers default to paragraph anchoring and float the frame
beside (or after) the following paragraph.

When both width and height are given and the width is a percentage,
the width is now resolved against the reference document text area width,
mirroring how the docx writer derives its print width from the reference docx
section properties. The height follows the aspect ratio.
2026-07-04 18:52:14 +02:00
John MacFarlane 8547196b44 Markdown reader: ordered sublists must start with 1
(or equivalent, e.g. 'i').  This brings the markdown
parser in line with commonmark, and avoids a lot of
unintended list starts.  Closes #11735.

Note that ordered lists that are not sublists may
still start with numbers other than 1.
2026-07-01 11:17:21 +02:00
John MacFarlane 612e143fbe DokuWiki reader: skip non-cell content after table row.
Closes #11739.
2026-06-29 19:28:54 +02:00
John MacFarlane 1c22498269 LaTeX writer: fix footnotes in def list headings.
Closes #8240.
2026-06-29 11:40:30 +02:00
John MacFarlane 197ea8e8bd RST reader: treat undefined substitution references as plain text.
This mirrors rst2html behavior. Closes #11727.
2026-06-23 12:06:27 +02:00
John MacFarlane 58929eb70c Markdown writer: in YAML metadata, put double quotes around...
...strings that are numbers beginning with 0 or ending with 0 and
having a decimal point.  Otherwise they will read as YAML numbers
and potentially be modified (e.g. 3.10 -> 3.1).

It would be better to fix this on the reader side, but since
we use a standard YAML parser it's hard to see how.

Closes #11715.
2026-06-21 15:15:42 +02:00
John MacFarlaneandClaude ec4fb91049 Typst reader: support auto_identifiers extension.
Headings without an explicit label can now be assigned automatic
identifiers based on the heading text, making them linkable in a
generated table of contents.  The extension is available for the
typst reader but is off by default; enable it with
`-f typst+auto_identifiers`.  The related `gfm_auto_identifiers` and
`ascii_identifiers` extensions are also made available.

Closes #11041.

Text.Pandoc.Readers.Typst.Parsing: PState gains sOptions,
sIdentifiers, and sLogMessages fields, and now has
HasReaderOptions, HasIdentifierList, and HasLogMessages
instances, allowing reuse of the shared registerHeader. (Not an
API change.)

Co-Authored-By: Claude <noreply@anthropic.com>
2026-06-13 23:38:51 +02:00
John MacFarlane 8a52ac1493 Markdown writer: improve identification of code language class...
...by looking it up in the syntax map. (Previously we just used
the first, excepting `sourceCode`.)

Closes #11701 (together with previous commit).
2026-06-12 21:01:25 +02:00
John MacFarlane 888a0062f1 Fix test for #11689 (use native) 2026-06-09 08:42:09 +02:00
John MacFarlane 5566a6bf3c Docx reader: improve horizontal rule detection.
Previously we failed to detect horizontal rules when the
underlying XML contained text elements for semantically
insignificant whitespace. We also failed to handle the case
where a paragraph contains text and has a bottom border
property: this seems to be what Word does when you insert
a horizontal rule, e.g. by typing a series of hyphens or #
characters.

Closes #11689.
2026-06-08 23:49:22 +02:00
John MacFarlane aafcfa14b0 LaTeX reader: handle \hypertarget with one braced parameter.
This seems to work in LaTeX, no doubt because a single token
is consumed as the target text parameter.

Closes #7418.
2026-06-08 19:22:24 +02:00
John MacFarlane 16d66edb2a Typst writer: fold lang into pickTypstAttributes.
This ensures that `lang` attributes will be set for both
Divs and Spans. Closes #11673.
2026-06-06 22:49:45 +02:00
John MacFarlane 9d68bb83a7 T.P.Writers.Shared; htmlAttrs - don't add data- prefix for markdown.
This is necessary for the `markdown_attribute` extension to
work properly. Closes #11685.
2026-06-05 17:17:22 +02:00
John MacFarlaneandClaude 098c48d79f RTF reader: fix tables parsed as deeply nested tables.
Closes #11682.

Fixes two bugs that caused a simple table to be read as a cascade of
nested tables:

* `\plain` reset the entire property record (including the in-table
  flag) via `const def`.  When a cell paragraph used `\plain` after
  `\intbl`, the in-table flag was cleared, so the next `\cell` closed
  the partially-built table and embedded it inside a new cell.  Per the
  RTF spec, `\plain` should reset only character formatting, so it now
  preserves paragraph/context properties (in-table, list level, outline
  level, hyperlink, anchor).

* `\row` was ignored and only `\trowd` started a new row.  Real-world
  RTF often emits a single `\trowd` and separates rows with `\row`, so
  every cell ended up in one row.  Both `\trowd` and `\row` now begin a
  fresh row (only when the current one has cells), and empty trailing
  rows are dropped when closing the table.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-06-03 16:36:29 +02:00
Meher Chaitanya 1ffeb856ee Add auto_identifiers support to Man Reader (#11675)
Add support for the `auto_identifiers`, `gfm_auto_identifiers`, and
`ascii_identifiers` extensions in the man reader. Section headings
parsed from .SH and .SS macros now receive auto-generated id
attributes when the extension is enabled, enabling `--toc` to
produce working anchor links.

- Add `autoIdExtensions` to default man extensions [behavior change]
- Add `HasReaderOptions`, `HasLogMessages` and `HasIdentifierList` to
  `ManState` to run `registerHeader`

Closes #8852.
2026-05-30 19:11:14 +02:00