Commit Graph
8848 Commits
Author SHA1 Message Date
John MacFarlane b706cb0ec5 Chunks: autogenerate unique ids for sections missing them.
This is needed for TOC generation to work properly. We can't
create TOC links if there are no ids.

This fixes some EPUB validation issues we've been getting since
switching over to Chunks for chunking.

Closes #9383.
2024-01-30 08:29:21 -08:00
John MacFarlane ef69842312 Chunks: improve fixTOCTreePaths.
We weren't adding ids for section headings that don't head a
chunk, but these headings are needed for a TOC.
2024-01-29 23:04:45 -08:00
John MacFarlane c1f87fdb37 Typst writer: handle labels and citaiton ids with spaces...
and other special characters. In these cases, we produce an
explicit `label()` rather than using `<>` or `@`.

Cloess #9387.
2024-01-29 12:25:09 -08:00
John MacFarlane 97eb927339 Typst writer: avoid illegal labels.
Closes #9387.
2024-01-29 11:45:15 -08:00
John MacFarlane 75d013f73e LaTeX reader: generate relative widths for \linewidth, \textheight.
Closes #9388.
2024-01-29 11:11:45 -08:00
John MacFarlane b6ac32b116 Typst writer escaping improvements.
We no longer escape `(`. The reason we did this before (#9137)
has been addressed in another way (#9252).

We only escape `=`, `+`, `-` at the beginning of a line.

We now also escape `/` at the beginning of a line.

This should reduce unnecessary escapes.

Closes #9386.
2024-01-29 10:53:51 -08:00
Iacobus1983 b837d59fd5 EPUB writer: add ARA roles for accessibility (#9378)
This commit adds roles to some html element within epub in order to comply with accessibility best practices. Footnote references are given role "doc-noteref",footnote text gets "doc-footnote", and nav gets "doc-toc".
2024-01-27 11:10:58 -08:00
John MacFarlane bc1caab557 EPUB writer: change default accessibilityFeatures.
Don't include ARIA and MathML in general; they won't apply for
epub2 or documents without math.
2024-01-27 10:57:50 -08:00
John MacFarlane 8d09c4aff4 EPUB reader: don't put # characters in identifiers. 2024-01-26 23:58:42 -08:00
John MacFarlane e7b2b2bd8b EPUB writer: Add accessibility metadata to EPUB metadata.
accessModes, accessibilityFeatures, accessibilityHazards.

Reasonable default values are used to ensure that pandoc's
EPUBs conform to the EU Accessibilty Act requirements.

Closes #9372.
2024-01-26 10:45:16 -08:00
John MacFarlane 09b6db8c50 EPUB writer: Add xml:lang to package element.
See #9372.
2024-01-25 10:38:57 -08:00
John MacFarlane d9a70d2eaf Add some kerns where needed between quotes in LaTeX output.
Closes #9371.
2024-01-25 09:41:47 -08:00
Gokul Rajiv e9bd6285e3 add case for "variable" class 2024-01-23 14:13:51 -08:00
John MacFarlane 3e54d3a45d Use reference form (e.g. @jones2000[p. 30]) for citations.
Previously we were using `#cite` and trying to put the supplement
in brackets; but a bracketed supplement only works with the
reference form.
2024-01-22 22:09:01 -08:00
Albert Krewinkel 57774d79e8 Org writer: escape special lines in code blocks
Fixes: #9218
2024-01-22 18:13:34 +01:00
John MacFarlane 243bbf699c HTML writer: ensure that an alt attribute is added in EPUB output.
This seems to be required by iBooks; even an empty alt attribute
will satisfy it. Closes #9354.
2024-01-20 10:34:32 -08:00
John MacFarlane f43ffabd3c LaTeX writer: create valid table even when table is empty.
Closes #9350.
2024-01-20 10:19:32 -08:00
John MacFarlane df1944f774 Markdown writer: fix output for pipe tables...
...with a huge number of columns. Previously we got invalid pipe
tables when the number of table columns exceeded the setting of
`--columns`.

Closes #9346.
2024-01-18 08:35:21 -08:00
John MacFarlane 18f0b70d40 DocBook reader: better handling of <procedure> and <substeps>.
`<procedure>` now gets parsed as an ordered list, and
`<substeps>` as a sublist. Closes #9341.
2024-01-16 11:58:42 -08:00
George Stagg 032f7c832f Add readMan to Text.Pandoc.Readers exports 2024-01-16 11:24:13 -08:00
John MacFarlane 2dd98b967b T.P.PDF: reliably detect when TOC has changed.
Sometimes the TOC changes but there are no warnings: this happens
when no labels are present.  In this case we must rerun LaTeX.
So we now take the sha1 hash of the TOC file and rerun LaTeX if it
changes between runs.

Closes #9295.
2024-01-06 11:10:05 -08:00
John MacFarlane 9f58a55280 Typst reader: fix handling of \overline.
Due to a typo, it was being incorrectly rendered as an `\underset`.
Closes #9294.
2024-01-05 22:09:36 -08:00
John MacFarlane e24002ed75 T.P.PDF: increase max number of LaTeX runs to 4.
On some documents, 4 runs are needed (e.g. when a LastPage
reference is used). Closes #9299.
2024-01-05 21:35:55 -08:00
John MacFarlane 6d19f174d4 Whitespace cleanup. 2024-01-05 14:50:52 -08:00
John MacFarlane 5cff32c575 OPML writer: respect --wrap options & --columns in contents of notes.
Closes #9297.
2024-01-04 16:45:02 -07:00
John MacFarlane 59e28b82b6 PDF: expand list of environment variables to display in verbose output.
Closes #9303.
2024-01-04 08:50:46 -07:00
John MacFarlane 87533e2d04 Add command test for #9293. 2023-12-30 22:09:13 -07:00
John MacFarlane 9a8f27b0fe MediaWiki reader: handle multiline math in list items.
Closes #9293.
2023-12-30 21:53:57 -07:00
John MacFarlane 17af220442 Add some explicit imports...
to avoid warnings with some GHC versions.
2023-12-26 23:06:38 -08:00
Edwin Török 075cbb6419 fix(docx): support absolute header/footer paths
Header and footer references may be absolute in the reference.docx.
E.g. editing it with dotnet's Open-XML-SDK causes this error:

```
+ pandoc test.md -t docx --reference-doc referenceh.docx -o test.docx
word//word/header1.xml missing in reference docx
```

There was already code in pandoc to handle relative vs absolute paths in
references, so use it.

Signed-off-by: Edwin Török <edwin@etorok.net>
2023-12-26 22:51:28 -08:00
John MacFarlane b1a1f04168 ODT/opendocument writers: properly handle highlighting styles.
These styles were going into an office:styles element in content.xml,
but this is invalid.  Instead they must go in styles.xml. See #9287.

The variable `highlighting-styles` no longer has any effect on
the default opendocument template, and highlighting styles are
not included in opendocument output.
2023-12-26 22:12:44 -08:00
John MacFarlane 3c178690e3 T.P.PDF: Ensure that we find all the LaTeX warnings requiring rerun.
This should fix a regression from 3.1.9 that led to incorrect
alignments in tables (and possibly other issues).

Closes #9284.
2023-12-24 12:26:35 -08:00
John MacFarlane 4672f35143 T.P.Chunks: Fine tune makeChunks.
* Ensure that chunks not based on sections (those with the
  "preamble" class) get unique identifiers, by appending chunk number.
* This will also ensure that they get unique path names when
  the path is generated from the identifier.

Closes #9281.
2023-12-22 11:57:26 -08:00
John MacFarlane 81521cee0d Markdown writer: add table identifier at end of caption if present.
Suggested at #9279.
2023-12-22 11:14:09 -08:00
John MacFarlane 36d1ec3e45 Org reader: parse caption and label for grid tables.
Closes #9279.
2023-12-20 16:55:10 -08:00
Edwin Török 712d746320 fix(docx): sort inline elements in schema order
Fixes #9273

```
[
    {
        "Description": "The element has unexpected child element 'http://schemas.openxmlformats.org/wordprocessingml/2006/main:b'.",
        "Path": {
            "NamespacesDefinitions": [
                "xmlns:w=\"http://schemas.openxmlformats.org/wordprocessingml/2006/main\""
            ],
            "Namespaces": {

            },
            "XPath": "/w:document[1]/w:body[1]/w:p[1]/w:r[7]/w:rPr[1]",
            "PartUri": "/word/document.xml"
        },
        "Id": "Sch_UnexpectedElementContentExpectingComplex",
        "ErrorType": "Schema"
    }
]
```

Signed-off-by: Edwin Török <edwin@etorok.net>
2023-12-19 15:00:23 -08:00
John MacFarlane 0e86a84e33 LaTeX writer: omit superfluous page locator label...
when used with `--natbib` or `--biblatex`. These will treat a
bare number as a page locator, and they will be able to localize
it.

We borrow the code for stripping the locator label from the suffix
from Citeproc code. Note that the recognition of the locator label
is locale-sensitive; if `lang` is `de`, then `S. 33` is a page
reference, and `p. 33` is not!

Closes #9275.
2023-12-18 23:27:40 -08:00
John MacFarlane 2f6a66feb1 Docx reader: fix HYPERLINK with only switch and no argument.
The argument can apparently be omitted, and then we just have
a fragment URL.  Closes #9246.
2023-12-18 17:44:56 -08:00
Edwin Török dbd1019a97 fix(docx): fix validation error on endnotePr
Copying `endnotePr` causes validation errors, because it is now
referencing something that doesn't exist in the document:
```
  {
        "FilePath": "test/docx/golden/custom_style_reference.docx",
        "ValidationErrors": "[{\"Description\":\"Element 'w:endnote' referenced by 'endnote@http://schemas.openxmlformats.org/wordprocessingml/2006/main:id' does not exist in part '/MainDocumentPart/EndnotesPart'. The reference value is '0'.\",\"Path\":{\"NamespacesDefinitions\":[\"xmlns:w=\\\"http://schemas.openxmlformats.org/wordprocessingml/2006/main\\\"\"],\"Namespaces\":{},\"XPath\":\"/w:settings[1]/w:endnotePr[1]/w:endnote[2]\",\"PartUri\":\"/word/settings.xml\"},\"Id\":\"Sem_MissingReferenceElement\",\"ErrorType\":\"Semantic\"},{\"Description\":\"Element 'w:endnote' referenced by 'endnote@http://schemas.openxmlformats.org/wordprocessingml/2006/main:id' does not exist in part '/MainDocumentPart/EndnotesPart'. The reference value is '-1'.\",\"Path\":{\"NamespacesDefinitions\":[\"xmlns:w=\\\"http://schemas.openxmlformats.org/wordprocessingml/2006/main\\\"\"],\"Namespaces\":{},\"XPath\":\"/w:settings[1]/w:endnotePr[1]/w:endnote[1]\",\"PartUri\":\"/word/settings.xml\"},\"Id\":\"Sem_MissingReferenceElement\",\"ErrorType\":\"Semantic\"}]"
  }
```

For now don't copy this element, it wasn't copied before, and it doesn't
seem necessary to fix the ordering problems we had with settings.

Fixes: c9bf4da74 ("Docx writer: ensure that elements in settings are ordered correctly.")

Signed-off-by: Edwin Török <edwin@etorok.net>
2023-12-18 17:15:03 -08:00
Edwin Török 1e4775f069 fix(docx): fix validation error on w:tblHeader
```
    {
        "FilePath": "test/docx/golden/tables.docx",
        "ValidationErrors": "[{\"Description\":\"The attribute 'http://schemas.openxmlformats.org/wordprocessingml/2006/main:val' has invalid value 'true'. The Enumeration constraint failed.\",\"Path\":{\"NamespacesDefinitions\":[\"xmlns:w=\\\"http://schemas.openxmlformats.org/wordprocessingml/2006/main\\\"\"],\"Namespaces\":{},\"XPath\":\"/w:document[1]/w:body[1]/w:tbl[1]/w:tr[1]/w:trPr[1]/w:tblHeader[1]\",\"PartUri\":\"/word/document.xml\"},\"Id\":\"Sch_AttributeValueDataTypeDetailed\",\"ErrorType\":\"Schema\"}]"
    }
```

Although this one might actually be a bug in Open-XML-SDK similar to
this, or a subtle difference between standard versions:
https://github.com/dotnet/Open-XML-SDK/issues/780

Signed-off-by: Edwin Török <edwin@etorok.net>
2023-12-18 17:15:03 -08:00
Edwin Török fe8847bdaa fix(docx): use left vs start consistently
They are equivalent, but OOXML-Validator complains:
```
  {
        "FilePath": "test/docx/golden/tables_separated_with_rawblock.docx",
        "ValidationErrors": "[{\"Description\":\"The attribute 'http://schemas.openxmlformats.org/wordprocessingml/2006/main:val' has invalid value 'start'. The Enumeration constraint failed.\",\"Path\":{\"NamespacesDefinitions\":[\"xmlns:w=\\\"http://schemas.openxmlformats.org/wordprocessingml/2006/main\\\"\"],\"Namespaces\":{},\"XPath\":\"/w:document[1]/w:body[1]/w:tbl[2]/w:tblPr[1]/w:jc[1]\",\"PartUri\":\"/word/document.xml\"},\"Id\":\"Sch_AttributeValueDataTypeDetailed\",\"ErrorType\":\"Schema\"},{\"Description\":\"The attribute 'http://schemas.openxmlformats.org/wordprocessingml/2006/main:val' has invalid value 'start'. The Enumeration constraint failed.\",\"Path\":{\"NamespacesDefinitions\":[\"xmlns:w=\\\"http://schemas.openxmlformats.org/wordprocessingml/2006/main\\\"\"],\"Namespaces\":{},\"XPath\":\"/w:document[1]/w:body[1]/w:tbl[1]/w:tblPr[1]/w:jc[1]\",\"PartUri\":\"/word/document.xml\"},\"Id\":\"Sch_AttributeValueDataTypeDetailed\",\"ErrorType\":\"Schema\"}]"
    }
```

pandoc already uses 'left' elsewhere, so be consistent, we still produce
the transitional schema, not the strict one which would have the 'start'
attribute.

Signed-off-by: Edwin Török <edwin@etorok.net>
2023-12-18 17:15:03 -08:00
Edwin Török 95dde29c5e fix(docx): fix validation error on inline w:i/w:iCs order
From `make validate-docx-golden-tests2`:
```
  {
        "FilePath": "test/docx/golden/definition_list.docx",
        "ValidationErrors": "[{\"Description\":\"The element has unexpected child element 'http://schemas.openxmlformats.org/wordprocessingml/2006/main:i'.\",\"Path\":{\"NamespacesDefinitions\":[\"xmlns:w=\\\"http://schemas.openxmlformats.org/wordprocessingml/2006/main\\\"\"],\"Namespaces\":{},\"XPath\":\"/w:document[1]/w:body[1]/w:p[3]/w:r[3]/w:rPr[1]\",\"PartUri\":\"/word/document.xml\"},\"Id\":\"Sch_UnexpectedElementContentExpectingComplex\",\"ErrorType\":\"Schema\"}]"
    },
```

Signed-off-by: Edwin Török <edwin@etorok.net>
2023-12-18 17:15:03 -08:00
Edwin Török c5780857e5 fix(docx): fix OOXMLValidator error on KeywordTok output
xmllint doesn't warn about this (maybe because the tag is empty?), but
the order doesn't match wml.xsd:
```
    <w:rPr>
      <w:color w:val="007020"/>
      <w:b/>
    </w:rPr>
```

And OOXMLValidatorCLI does warn about it:
```
{
        "Description": "The element has unexpected child element 'http://schemas.openxmlformats.org/wordprocessingml/2006/main:b'.",
        "Path": {
            "NamespacesDefinitions": [
                "xmlns:w=\"http://schemas.openxmlformats.org/wordprocessingml/2006/main\""
            ],
            "Namespaces": {

            },
            "XPath": "/w:styles[1]/w:style[40]/w:rPr[1]",
            "PartUri": "/word/styles.xml"
        },
        "Id": "Sch_UnexpectedElementContentExpectingComplex",
        "ErrorType": "Schema"
    }
```

Signed-off-by: Edwin Török <edwin@etorok.net>
2023-12-18 17:15:03 -08:00
Edwin Török 333b0dd2ff fix(docx): fix validation error on w:annotationRef
annotationRef is not valid for `w:rPr`, only for `w:r` according to
wml.xsd.

See https://github.com/jgm/pandoc/issues/9269

Signed-off-by: Edwin Török <edwin@etorok.net>
2023-12-18 14:08:53 -08:00
Edwin Török ded8132d30 fix(docx): fix validation error in w:nsid
The length here seems to refer to length in bytes (so twice as long in
hex):
```
./tmp/numbering-pretty.xml:4: element nsid: Schemas validity error : Element '{http://schemas.openxmlformats.org/wordprocessingml/2006/main}nsid', attribute '{http://schemas.openxmlformats.org/wordprocessingml/2006/main}val': [facet 'length'] The value 'A990' has a length of '2'; this differs from the allowed length of '4'.
```

[This](https://learn.microsoft.com/en-us/dotnet/api/documentformat.openxml.wordprocessing.nsid?view=openxml-2.8.1) also documents the longer values.

Signed-off-by: Edwin Török <edwin@etorok.net>
2023-12-18 12:30:20 -08:00
John MacFarlane f7e31adc97 Docx writer: fixed validation errors in tables.
Closes #9266.
2023-12-18 10:18:57 -08:00
John MacFarlane 9c5e6aeedc Docx writer: fix validation error.
The elements in pPr in lists were not properly ordered.
This doesn't seem to cause problems for Word, but it makes
validation fail and may pose problems for other consumers of
docx.  Closes #9265.
2023-12-18 09:54:14 -08:00
John MacFarlane c9bf4da74f Docx writer: ensure that elements in settings are ordered correctly.
The elements must occur in a specific order. This was being
messed up when integrating a custom reference.docx. Closes #9264.
2023-12-17 17:03:14 -08:00
John MacFarlane 3abbf97d0b Docx writer: don't emit empty rows.
These (`<w:tr />`) seem to cause problems for word. Closes #9224.
2023-12-17 11:32:27 -08:00
John MacFarlane 9f43da7f81 Logging: add MakePDFWarning constructor to LogMessage.
[API change]

Use this to pass LaTeX warnings on to user as warnings.
2023-12-15 12:23:33 -08:00