Commit Graph
8388 Commits
Author SHA1 Message Date
John MacFarlane da717de334 Fix toTextM for Windows.
We forgot to filter out CRs as we do in toText.
2023-06-27 11:41:12 -07:00
John MacFarlane e0bea476bf Use toTextM instead of toText when possible.
(That is, whenever we have the filename and are in a
PandocMonad instance.)

This will lead to more informative error messages for UTF8
encoding, indicating the file path and byte offset where the
error occurs. Closes #8884.
2023-06-27 11:15:31 -07:00
John MacFarlane 67970d244e Text.Pandoc.Class: add toTextM.
This is like `Text.Pandoc.UTF8.toText`, except:

- it takes a file path as first argument, in addition to
  bytestring contents
- it raises an informative error with source position if
  the contents are not UTF8-encoded

[API change]

This replaces `utf8ToText` in `Text.Pandoc.App.Input`.

See #8884.
2023-06-27 11:12:58 -07:00
John MacFarlane c80ebc4621 Require commonmark >= 0.2.3 and fix import warning. 2023-06-26 09:15:11 -07:00
John MacFarlane 44d9b492f9 Fix spacing issues. 2023-06-24 11:25:38 -07:00
John MacFarlane 3f06c7f0dd Make --number-sections work with JATS writer. 2023-06-24 11:22:11 -07:00
Ruqi f257c97170 Update TableBody RowHeadColumns caculation: change from max to min (#8634)
This change sets RowHeadColumns to the minimum value of each row, which
gives better results in cases where rows have different numbers of leading th tags.
2023-06-24 09:18:03 -07:00
John MacFarlane df4f13b262 More fixes to 5e381e3.
These changes recognize that parseURI does not unescape the path.

Another change is that the canonical form of the path used as the
MediaBag key retains percent-encoding, if present; we only unescape
the string when writing to a file.

See #8918.

Some tests are needed before the issue can be closed.
2023-06-23 09:27:04 -07:00
John MacFarlane 54561e9a66 Fix bug in 5e381e3878
In the new code a comma mysteriously turned into a period.
This would have prevented proper separation of the mime type and
content in data uris. Thanks to @hseg for catching this.
2023-06-23 09:00:56 -07:00
ech0 7b8b8f802d Retain image query parameters in dokuwiki reader (#8887)
While converting dokuwiki syntax to gfm, the query parameters of images were stripped from the output.
In general this makes sense, as the parameters' semantics are specific to dokuwiki. But it makes it impossible
to access the query in a filter. This change retains the query parameters of image urls in the dokuwiki reader,
by adding it as an extra query attribute.
2023-06-22 11:00:47 -07:00
Stephen Altamirano 83b69ead81 Textile reader: Add support for link references (#8706)
Textile supports what it calls "link alias", which are analogous to
Markdown's reference-style links.
2023-06-22 10:53:15 -07:00
Albert Krewinkel b07552c23f Logging: add new log message type ScriptingWarning [API change] 2023-06-22 10:48:33 -07:00
Albert Krewinkel cce347eaf5 LaTeX reader: support alt text on images. (#8745)
Closes: #8743
2023-06-22 10:46:56 -07:00
John MacFarlane 5e381e3878 Fix a security vulnerability in MediaBag and T.P.Class.IO.writeMedia.
This vulnerability, discovered by Entroy C, allows users to write
arbitrary files to any location by feeding pandoc a specially crafted
URL in an image element.  The vulnerability is serious for anyone
using pandoc to process untrusted input.  The vulnerability does
not affect pandoc when run with the `--sandbox` flag.
2023-06-20 13:50:13 -07:00
John MacFarlane 40dacad525 ChunkedHTML writer: Make math work in top-level page.
Closes #8915.
2023-06-20 11:59:25 -07:00
John MacFarlane 83fcd0031a Add Extracting log message for --extract-media.
This message will also be triggered when media is being
extracted to a temporary location, e.g. in PDF production.
2023-06-19 11:18:30 -07:00
John MacFarlane 725ec09a10 Make implicit_figures work again in commonmark reader.
Support for this (introduced in #6350) disappeared when we made an
architectural change.
2023-06-19 11:18:30 -07:00
Adelar da Silva Queiróz 3a20ea3aca fix: add highlight for F# language
Fix for this SO question: https://tex.stackexchange.com/questions/563778/how-to-get-f-syntax-highlighting
2023-06-19 12:28:40 -04:00
John MacFarlane 8194199c34 Allow epub-title-page to be used in defaults files.
This wasn't implemented before due to an oversight. Closes #8908.
2023-06-19 09:27:19 -07:00
John MacFarlane 913bd8c777 LaTeX writer: Fix escaping of & in \href and \url.
Closes #8903.
2023-06-16 18:52:59 -04:00
Noah Malmed 76952d0b06 Add footer and multiple body parsing to JATS table reader (#8795)
Closes #8765.
2023-06-08 09:58:28 -07:00
John MacFarlane 7b4eb40737 Parse references title from ref-list in JATS reader.
Closes #8365.
2023-06-07 13:36:58 -07:00
John MacFarlane 4f440583a8 JATS writer: include title in ref-list.
Previously the reference title ended up in a separate
section at the back of the body instead of in the ref-list
in the back matter.

Closes #8364.
2023-06-07 13:18:20 -07:00
John MacFarlane 0166aecaab Allow abstract-title to be specified in docx metadata.
Closes #8794.
2023-06-07 09:08:53 -07:00
John MacFarlane d04838b678 Fix localization of "Abstract" title in docx writer.
See #8702, which was only partially implemented before.
2023-06-07 08:58:21 -07:00
John MacFarlane 8bdb430c74 Fix whitespace. 2023-06-06 17:51:17 -07:00
John MacFarlane 46a4e24d6b Add typst reader.
New module Text.Pandoc.Readers.Typst [API change].
2023-06-06 16:08:14 -07:00
Noah Malmed 7f24c65b86 Improve title and label parsing in the JATS reader (#8840)
Closes #8718.
2023-06-06 11:03:52 -07:00
John MacFarlane 2f7b4032d5 ODT reader: allow lists in table cells. Closes #8892. 2023-06-05 17:01:59 -07:00
John MacFarlane 2c8de4c129 Typst writer: use #footnote for notes.
Closes #8893.
2023-06-05 16:07:36 -07:00
John MacFarlane 5539eecd0b ODT reader: allow frames inside spans.
Closes #8886.
2023-06-04 15:51:43 -07:00
John MacFarlane 0e0c00db51 RST redear: Fix sorting on anonymous keys.
This fixes a bug in RST documents that have anonymous links.
Closes #8877.
2023-05-29 13:05:34 -07:00
John MacFarlane 903d11b537 HTML reader: fix iframe with data URI of an image.
Closes #8856.

In this case we don't want to try to parse the data at the URL.
Instead, create an image inside a div.
2023-05-17 17:02:59 -07:00
John MacFarlane f1233888c5 Ms writer: coerce titles to inlines.
Block-level formatting is not allowed inside .TL.
Closes #8835.
2023-05-09 11:40:09 -07:00
John MacFarlane 05ad84a0c6 LaTeX writer: better fix for colspecs for multicolumn table.
Improves on the last commit; closes #8831.
2023-05-08 13:09:04 -07:00
John MacFarlane ca5940bf2c LaTeX writer: fix width for multicolumn simple table.
This should help with #8831.  I'm not sure this is the
best solution, but it's certainly better than what we had before!
2023-05-08 12:39:50 -07:00
Stephan Meijer c97db9af26 Introduce support for Intense Quote in Docx conversion
This commit introduces support for the Intense Quote in Docx Conversion.

Previously this was converted to a regular paragraph, but Intense Quote
should be interpreted as a Quote in conversion.
2023-05-08 08:44:29 -07:00
Albert Krewinkel a00fba794b Jira writer: use first code block class as highlighting language.
The writer no longer searches the list of classes for a known programming
language but always uses the first class in that list as the language
identifier.

Closes: #8814
2023-05-05 09:54:05 +02:00
Albert Krewinkel f9bbbf6f40 Markdown reader: disallow escaping of ~ and " in markdown_strict
This matches the behavior of the legacy `markdown.pl` as well as what is
described in the manual.

Fixes: #8777
2023-05-04 11:52:04 +02:00
Albert Krewinkel 05a23af76d LaTeX reader: ignore args to column type in \multicolumn.
The `\multicolumn` command takes the column type as the second argument.
Types like `p` take an additional argument, which is now ignored and no
longer causes the table parser to fail.

Fixes: #8789
2023-05-04 11:36:10 +02:00
Michael Stahl 737446f33d Writers.OpenDocument: handle row header column cells as header cells
The previous commit prevented header column cells from being dropped on
the floor, this one changes the paragraph style to "Table_20_Heading".

Note that for the test input, the result is not correct: the
AnnotatedTable type cannot represent the HTML input properly, as it only
has a concept of header rows and header columns, but HTML can have an
individual cell that is a header (not 100% sure but they way i read
https://html.spec.whatwg.org/#header-and-data-cell-semantics the <th>
cell here is both a row header cell and a column header cell while the
other cells in the row and column are not header cells), and header
cells may even appear "in the middle" of a table (see example in
https://html.spec.whatwg.org/#the-th-element).

So while this appears like it's the right thing to do for
Writer.OpenDocument, it's not clear if this is going to make things
better or worse overall.

Fixes: #8764
2023-05-01 20:55:37 -07:00
Michael Stahl 2881c12740 Writers.OpenDocument: handle row header column cells as ordinary cells
While ODF 1.3 part 3 does specify a 9.1.11 <table:table-header-columns>
element, in practice it's only implemented by spreadsheet applications,
not word processors.

So simply treat the row header columns as ordinary table columns, at
least they don't get lost then.

Fixes: #8764
2023-05-01 20:55:37 -07:00
Michael Stahl 013351f602 Writers.OpenDocument: fix invalid text:p inside text:p from meta
The only element in meta for which data/templates/default.opendocument
allows block element content appears to be "abstract", which is already
filtered out of meta' at this point, so simply convert all blocks to
linebreak-separated inlines.

Fixes: #8256
2023-05-01 20:39:47 -07:00
John MacFarlane a34a4d8e4a Commonmark writer: use shortcut reference links.
Commonmark supports these.
2023-04-28 22:03:01 -07:00
Michael Stahl 16be353077 Writers.ODT: don't add settings.xml
The way "ooo:configuration-settings" work is that if LibreOffice Writer
loads a file in which it exists, it assumes that an old version of
itself has produced it and it defaults any config-item that it knows but
that doesn't exist in the file to the "backward-compatible" value, which
usually means to enable some layout or text formatting bugs so that the
document looks the same as in the old LO version that created it.

If there is no "ooo:configuration-settings" then it will use the default
values for a new document, which should give the best formatting
results.

Pandoc's documents are obviously not created by LO, so it doesn't make
sense to produce ooo:configuration-settings.

Regarding the "ooo:view-settings", IIRC these mainly set to the
document view to the position where the document was last edited (by
default only if the user in meta.xml matches the user data in LO's
configuration), which is also irrelevant for Pandoc.

So just leave out the settings.xml; only content.xml/styles.xml are
required (see ODF 1.3, part 3, 2.2.1 OpenDocument Document, A.2).

(The settings.xml that is produced for formulas by the documentSettings
function hopefully isn't problematic because LO Math works a bit
differently and stores important formatting attributes in settings.xml
because MathML doesn't allow it in content.xml.)
2023-04-28 12:17:22 -07:00
Michael Stahl 60e72b2542 Writers.ODT: don't add Configurations2 dir
This empty file is some ancient OOo wart and it's quite pointless here.
2023-04-28 12:17:22 -07:00
Michael Stahl 54c0c494a2 Writers.ODT: don't add thumbnail
Quite pointless to add a hardcoded image that doesn't show the
document content.
2023-04-28 12:17:22 -07:00
Michael Stahl ac35c9a31c Writers.ODT: put manifest.version on directory file-entry
See ODF 1.3 part 2, 4.16.14.1 <manifest:file-entry>:

  The manifest:version attribute specifies the format version of a file
  entry. For documents that are composed from multiple files, this
  attribute is specified at the manifest entry that references the
  folder that contains these files.
2023-04-28 12:17:22 -07:00
Michael Stahl f3e1d33ea0 Writers.ODT: stop validator complaints by producing ODF 1.3
The ODF validator complains about an invalid attribute
style:contextual-spacing in styles.xml, ultimately an implementation
error in whatever old version of LibreOffice that produced the
data/odt/styles.xml (should have put it into an extension namespace).

Fortunately the attribute was added to ODF 1.3, which was released a
couple years ago.

So the easiest fix is to simply produce ODF 1.3 instead of 1.2; it's
supposed to be fully backward compatible.
2023-04-28 12:17:22 -07:00
John MacFarlane 16f28ef5e9 Remove redundant line in pattern match. 2023-04-20 13:33:02 -07:00