Commit Graph
762 Commits
Author SHA1 Message Date
Hikaru Ibayashi 3be253fb90 LaTeX writer: fix bug with big footnotes inside emphasis (#9168)
Closes #8982.
2023-12-10 09:29:45 -08:00
Stephan Meijer 2e8ecb3fba Docx reader: unwrap content of shaped textboxes...
* #9214 text in shape format test document

* #9214 support Text in Shape Format

* #9214 remove irrelevant code
2023-11-29 18:59:09 -08:00
Stephan Meijer 56f7f11473 ODT reader: Support attr text:continue-numbering (#8998)
Closes #8979.
2023-11-06 19:15:37 -08:00
John MacFarlane fbf9194e34 DokuWiki reader: parse <code> and <file> as block-level code.
Previously we treated them as inline code in some contexts,
but that is not how DokuWiki works.

Closes #9154.
2023-10-22 10:59:07 -07:00
Julia Diaz fa3513b104 JATS reader: fix handling of alt-text (#9134)
Previously we were looking for an attribute that doesn't exist in
JATS; alt-text is provided by a child element.

Closes #9130.
2023-10-17 09:16:00 -07:00
Amneesh Singh e4467ba10c Org reader: allow escaping commas in macro arguments
Signed-off-by: Amneesh Singh <natto@weirdnatto.in>
2023-08-26 17:16:34 -07:00
John MacFarlane 70329edcd7 LaTeX writer: improve internal links and targets.
We no longer wrap section headings in a `\hypertarget`.
This is unnecessary (hyperref creates an anchor based on the
label) and it interferes with tagging.

In addition, we now use `\hyperref` rather than `\hyperlink`
for internal links.

Currently `\hypertarget` is still being used for link anchors
not on headings.

Closes #8744. Thanks to @u-fischer.
2023-08-19 09:52:01 -07:00
Julia Diaz 6673f832ea Fix display of block elements in JATS reader (PR #8971)
A number of block elements, like disp-quote, list, and disp-formula, were always treated as inlines if appearing inside paragraphs, even if their usage granted a separate block. The function isElementBlock has been refined to prevent this, and a number of specific parse cases have been added to parseBlock.

Also, some minimal cleanup of the test file, in order for it to pass XML validation against the JATS DTD 1.3 (it was not compliant with the current or any previous versions of JATS).

Closes #8889.
2023-08-10 13:02:43 -07:00
John MacFarlane eddedbfc14 Fix new variant of the vulnerability in CVE-2023-35936.
Guilhem Moulin noticed that the fix to CVE-2023-35936 was incomplete.
An attacker could get around it by double-encoding the malicious
extension to create or override arbitrary files.

    $ echo '![](data://image/png;base64,cHJpbnQgImhlbGxvIgo=;.lua+%252f%252e%252e%252f%252e%252e%252fb%252elua)' >b.md
    $ .cabal/bin/pandoc b.md --extract-media=bar
    <p><img
    src="bar/2a0eaa89f43fada3e6c577beea4f2f8f53ab6a1d.lua+%2f%2e%2e%2f%2e%2e%2fb%2elua" /></p>
    $ cat b.lua
    print "hello"
    $ find bar
    bar/
    bar/2a0eaa89f43fada3e6c577beea4f2f8f53ab6a1d.lua+

This commit adds a test case for this more complex attack and fixes
the vulnerability.  (The fix is quite simple: if the URL-unescaped
filename or extension contains a '%', we just use the sha1 hash of the
contents as the canonical name, just as we do if the filename contains
'..'.)
2023-07-20 11:51:28 -07:00
John MacFarlane cfc1d6a7b9 JATS writer: don't use <code> for inline code.
See #8889. The Taylor and Francis guide to JATS says that
`<code>` is block level and not intended to be used inline
within standard text.
2023-07-18 17:09:45 -07:00
John MacFarlane a2be4ac62b Add typst reader tests.
Closes #8942.
2023-07-06 12:20:20 -07:00
John MacFarlane 2467277f07 Fix asciidoc tests 2023-07-05 10:48:44 -07:00
John MacFarlane adba9d460c Make modern AsciiDoc the target for asciidoc.
The AsciiDoc community now regards the dialect parsed by `asciidoctor`
as the official AsciiDoc syntax, so it should be the target of our
`asciidoc` format.

Closes #8936.

The `asciidoc` output format now behaves like `asciidoctor` used to.
`asciidoctor` is a deprecated synonynm. For the old `asciidoc` behavior
(targeting the Python script), use `asciidoc_legacy`.

The templates have been consolidated. Instead of separate
`default.asciidoctor` and `default.asciidoc` templates, there is
just `default.asciidoc`.

Text.Pandoc.Writers.AsciiDoc API changes:

- `writeAsciiDoc` now behaves like `writeAsciiDoctor` used to.
- `writeAsciiDoctor` is now a deprecated synonym for `writeAsciiDoc`.
- New exported function `writeAsciiDocLegacy` behaves like
  `writeAsciDoc` used to.
2023-07-05 09:10:36 -07:00
John MacFarlane 5246f02f0b Improve tests for fillMediaBag/extractMedia.
Ensure that the current directory is not changed up if a test fails,
and fix messages for the assertion failures.
2023-06-23 10:54:59 -07:00
John MacFarlane fe62da61df Add tests for fillMediaBag/extractMedia. 2023-06-23 10:36:33 -07:00
ech0 7b8b8f802d Retain image query parameters in dokuwiki reader (#8887)
While converting dokuwiki syntax to gfm, the query parameters of images were stripped from the output.
In general this makes sense, as the parameters' semantics are specific to dokuwiki. But it makes it impossible
to access the query in a filter. This change retains the query parameters of image urls in the dokuwiki reader,
by adding it as an extra query attribute.
2023-06-22 11:00:47 -07:00
Stephan Meijer 9e8e016076 Rename test/docx/block_quotes_parse_indent.native for consistency 2023-05-09 11:41:42 -07:00
Albert Krewinkel b8c7678fcf Org reader: treat #+NAME as synonym for #+LABEL.
Closes: #8578
2023-04-05 14:50:52 +02:00
John MacFarlane 9edf965712 Add typst writer.
See #8713.
2023-03-25 22:05:52 -07:00
Christian Christiansen fe720dd0c6 Org reader: Allow zero width space as an escape character
Allow the character U+200B to be used as an escape character as described
in the Org-mode documentation https://orgmode.org/manual/Escape-Character.html

Closes issue #8716.
2023-03-22 18:29:53 +01:00
John MacFarlane 1bfab1d11e HTML writer footnotes changes:
When `--reference-location=section` or `=block`, use an
`aside` element for the notes rather than a `section`.

When `--reference-location=section`, include the `aside`
element inside the section element, rather than outside.
(In slide shows, this option causes footnotes on a slide
to be displayed at the bottom of the slide.)

Closes #8695.
2023-03-14 21:49:05 -07:00
Albert Krewinkel d08d1c04c5 JATS writer: include alt-text in <graphic>, <inline-graphic> elements
Closes: #8631
2023-02-18 00:09:21 +01:00
Daniel Kessler 9dbb75dacb ODT reader: fix blockquote indent detection
The ODT reader is supposed to detect blockquotes by checking a paragraph
style's indentation level. But it's broken for two reasons:

* The parser fails on non-integers. So "1in" will get read as 25mm, but
  "1.0in" fails. By default, the Quotations style is "0.3937in".
* The reader doesn't check indentation levels of parent styles.
  In my test documents, LibreOffice often creates child styles for
  individual paragraphs, so it's important to check parents (ODT files
  created by the Pandoc ODT writer don't have this issue though).

I added a new test "blockquote2" whose ODT file is generated directly
from the corresponding Markdown file with pandoc.

Fixes #3437.
2023-01-29 16:40:32 -08:00
Albert Krewinkel ca6d4bfa49 Markdown, CommonMark: add support for wiki links. [API change]
Adds the Markdown/CommonMark extensions `wikilinks_title_after_pipe` and
`wikilinks_title_before_pipe`. The former enables links of style `[[Name
of page|Title]]` and the latter `[[Title|Name of page]]`. Titles are
optional in both variants, so this works for both:
`[[https://example.org]]`, `[[Name of page]]`.

The writer is modified to render links with title `wikilink` as a
wikilink if a respective extension is enabled.

Pandoc will use `wikilinks_title_after_pipe` if both extensions are
enabled.

Closes: #2923
2023-01-15 10:46:55 -08:00
Albert Krewinkel 8f394a17f2 Remove Null constructor from docs 2023-01-14 10:10:26 +01:00
Albert KrewinkelandAner Lucero 909ced5153 Support complex figures. [API change]
Thanks and credit go to Aner Lucero, who laid the groundwork for this
feature in the 2021 GSoC project. He contributed many changes, including
modifications to the readers for HTML, JATS, and LaTeX, and to the HTML
and JATS writers.

Shared (Albert Krewinkel):

- The new function `figureDiv`, exported from `Text.Pandoc.Shared`,
  offers a standardized way to convert a figure into a Div element.

Readers (Aner Lucero):

- HTML reader: `<figure>` elements are parsed as figures, with the
  caption taken from the respective `<figcaption>` elements.

- JATS reader: The `<fig>` and `<caption>` elements are parsed into
  figure elements, even if the contents is more complex.

- LaTeX reader: support for figures with non-image contents and for
  subfigures.

- Markdown reader: paragraphs containing just an image are treated as
  figures if the `implicit_figures` extension is enabled. The identifier
  is used as the figure's identifier and the image description is also
  used as figure caption; all other attributes are treated as belonging
  to the image.

Writers (Aner Lucero, Albert Krewinkel):

- DokuWiki, Haddock, Jira, Man, MediaWiki, Ms, Muse, PPTX, RTF, TEI,
  ZimWiki writers: Figures are rendered like Div elements.

- Asciidoc writer: The figure contents is unwrapped; each image in the
  the figure becomes a separate figure.

- Classic custom writers: Figures are passed to the global function
  `Figure(caption, contents, attr)`, where `caption` and `contents` are
  strings and `attr` is a table of key-value pairs.

- ConTeXt writer: Figures are wrapped in a "placefigure" environment
  with `\startplacefigure`/`\endplacefigure`, adding the features
  caption and listing title as properties. Subfigures are place in a
  single row with the `\startfloatcombination` environment.

- DocBook writer: Uses `mediaobject` elements, unless the figure contains
  subfigures or tables, in which case the figure content is unwrapped.

- Docx writer: figures with multiple content blocks are rendered as
  tables with style `FigureTable`; like before, single-image figures are
  still output as paragraphs with style `Figure` or `Captioned Figure`,
  depending on whether a caption is attached.

- DokuWiki writer: Caption and "alt-text" are no longer combined. The
  alt text of a figure will now be lost in the conversion.

- FB2 writer: The figure caption is added as alt text to the images in
  the figure; pre-existing alt texts are kept.

- ICML writer: Only single-image figures are supported. The contents of
  figures with additional elements gets unwrapped.

- HTML writer: the alt text is no longer constructed from the caption,
  as was the case with implicit figures. This reduces duplication, but
  comes at the risk of images that are missing alt texts. Authors should
  take care to provide alt texts for all images.

  Some readers, most notably the Markdown reader with the
  `implicit_figures` extension, add a caption that's identical to the
  image description. The writer checks for this and adds an
  `aria-hidden` attribute to the `<figcaption>` element in that case.

- JATS writer: The `<fig>` and `<caption>` elements are used write
  figures.

- LaTeX writer: complex figures, e.g. with non-image contents and
  subfigures, are supported. The `subfigure` template variable is set if
  the document contains subfigures, triggering the conditional loading
  of the *subcaption* package. Contants of figures that contain tables
  are become unwrapped, as longtable environments are not allowed within
  figures.

- Markdown writer: figures are output as implicit figures if possible,
  via HTML if the `raw_html` extension is enabled, and as Div elements
  otherwise.

- OpenDocument writer: A separate paragraph is generated for each block
  element in a figure, each with style `FigureWithCaption`. Behavior for
  single-image figures therefore remains unchanged.

- Org writer: Only the first element in a figure is given a caption;
  additional block elements in the figure are appended without any
  caption being added.

- RST writer: Single-image figures are supported as before; the contents
  of more complex images become nested in a container of type `float`.

- Texinfo writer: Figures are rendered as float with type `figure`.

- Textile writer: Figures are rendered with the help of HTML elements.

- XWiki: Figures are placed in a group.

Co-authored-by: Aner Lucero <4rgento@gmail.com>
2023-01-13 09:13:27 -08:00
lifeunleaded 4746d0c806 Store "unnumbered" class in DocBook role attribute (#8481)
Markdown allows marking a heading as unnumbered, which is stored
as a class token internally. This change will recognize this
particular class token and append it to the role attribute, or
create a role attribute with it if needed. This does not imply
any processing in DocBook but is intended to let customized
stylesheets identify these sections and act accordingly.

Closes #1402
2023-01-12 19:29:53 -08:00
Albert Krewinkel b55cd83a60 Update copyright years, it's 2023! 2023-01-10 20:28:27 +01:00
Albert Krewinkel 8f5af5a06c ConTeXt writer: always use \type for inline code
Inline codes that contained curly braces where previously rendered with
`\mono`; this led to unexpected results when the presentation of `\type`
was customized, as those changes would not have been applied to code
rendered with `\mono`.
2022-12-17 23:53:12 +01:00
John MacFarlane 144bf90ab9 LaTeX writer: Use soul instead of ulem for strikeout, underline.
This handles things like hyphenation, line breaks, and nonbreaking
spaces better.

Closes #8411.
2022-11-19 12:28:41 -08:00
Wout Gevaert e9d2ceb9bd Add tests for the new table features 2022-11-11 10:12:07 -08:00
John MacFarlane 368f3fd71f Rename T.P.Readers.Odt -> T.P.Readers.ODT.
For consistency with Writers.ODT.

Similarly, rename `readOdt` -> `readODT`.

[API change]
2022-10-24 22:58:03 -07:00
John MacFarlane e6866c1543 Rename T.P.Writers.Docbook -> T.P.Writers.DocBook.
Similarly, rename `writeDocbook` -> `writeDocBook`, for
consistency with the DocBook reader's naming.

[API change]
2022-10-24 22:49:25 -07:00
Amir Dekel 31aa660e65 Org reader: make #+pandoc-emphasis-pre work as expected. (#8360)
So far, `orgStateLastPreCharPos` wasn't updated appropriately after each
parsing to native Str (by the parser `str`). In addition to solving
this, the guard `notAfterString` in `emphasisStart` is removed to allow
emphasis after Str at the first place.
2022-10-10 12:20:58 +02:00
John MacFarlane d9004c9e5c Officially deprecate writeJATS...
in favor of writeJatsArchiving.
2022-10-02 11:21:06 -07:00
Albert Krewinkel 5be9052f5f [API Change] Extract Lua code into new package pandoc-lua-engine
The flag 'lua53` must now be used with that package if pandoc is to be
compiled against Lua 5.3.
2022-09-30 08:33:40 -07:00
Albert Krewinkel 78814f27e0 [API Change] Filter: Parameterize applyFilters over scripting engine.
Pass the scripting engine as a parameter so it can be swapped out.
2022-09-30 08:33:40 -07:00
Albert Krewinkel ddaadc88bc [API Change] Rename pandocVersion to pandocVersionText...
and add a new `pandocVersion` value with type `Version`. This is
consistent with the type used for `pandocTypesVersion` and allows to use
the value where a Version type is required.
2022-09-30 08:33:40 -07:00
Justin Wood 740396d1ed HTML Reader: Tests for 'tt' and 'code' elements (#8330).
tt is a deprecated element in HTML, but is still used in the wild in
some places, support reading it as just another 'code' element.

Commit 4abb9d0ad8 was originally
part of this PR as well.
2022-09-28 14:04:07 -07:00
John MacFarlane 2ff0b9a9ef Tests.Command: remove unused runTest. 2022-09-27 22:14:43 -07:00
John MacFarlane a27e10111a Fix small whitespace things. 2022-09-27 13:03:26 -07:00
Albert Krewinkel a31cf9a005 Org writer: keep code block attributes as header args.
This allows to keep more information in the resulting `src` blocks,
making it easier to roundtrip from or through Org. Org babel ignores
unknown header arguments.
2022-08-22 12:53:57 +02:00
Albert Krewinkel 1de40c62fc Org writer: add code block identifier as #+name to src blocks. 2022-08-22 12:53:50 +02:00
luz paz aff726b460 Fix typos
Found via `codespell -q 3 -S changelog.md -L bu,fo,ist,mke,multline,noes,ot,pard,pres,tabl,te,tothe`
2022-08-21 20:23:00 -07:00
John MacFarlane 5c3423f2e2 DokuWiki reader: support latex plugin and math.
The `tex_math_dollars` extension is now supported for `dokuwiki`
(but off by default).

Content inside `<latex>...</latex>` is parsed as raw LaTeX inline,
and inside `<LATEX>..</LATEX>` as raw LaTeX block.

In addition, this commit changes the behavior of `<php>...</php>` so
that instead of producing a code block, it produces raw HTML
with `<?php ... ?>`.

Closes #8178.
2022-07-28 11:28:02 -07:00
John MacFarlane 98fb8521a7 RST writer: Fix missing spaces with nested inlines.
Previously spaces around links inside italics were omitted.

Closes #8182.
2022-07-16 15:49:45 +02:00
John MacFarlane 5df94da831 Remove Muse reader round-trip tests.
These are nondeterministic and have repeatedly failed
on strange edge cases.

The Muse reader's maintainer has not been active, and
it isn't worth developer time to chase down these problems.
2022-07-03 01:19:40 +02:00
Albert Krewinkel f49bee5c31 ConTeXt writer: support complex table structures. (#8116)
The following table feature are now supported in ConTeXt:

- colspans,
- rowspans,
- multiple bodies,
- row headers, and
- multi-row table head and foot.

The wrapping `placetable` environment is also given a `reference` option
with the table identifier, enabling referencing of the table from within
the document.
2022-06-21 10:22:34 -07:00
Albert Krewinkel e5c41f11de ConTeXt writer: use "sectionlevel" environment for headings (#8106)
The document hierarchy is now conveyed using the
`\startsectionlevel`/`\stopsectionlevel` by default. This makes it easy
to include pandoc-generated snippets in documents at arbitrary levels.
The more semantic environments "chapter", "section", "subsection", etc.
are used if the `--top-level-division` command line parameter is set to
a non-default value.

Closes: #5539
2022-06-06 09:24:28 -07:00
Albert Krewinkel 7844300414 Muse tests: do no try to roundtrip strings containing \DEL char
Fixes: #8108
2022-06-06 16:14:07 +02:00