John MacFarlaneandClaude
ce060e98e6
The dispatch maps (inlineCommands, blockCommands, environments) have types that mention the parser monad, so they are compiled as functions of the PandocMonad dictionary and were being rebuilt from scratch -- hundreds of Map entries -- every time a command or environment was dispatched. SPECIALIZE pragmas can't fix this on the CLI path, since the reader table abstracts over the monad dictionary. Instead, add a ReaderT layer to the LP monad carrying a LaTeXEnv record with the three dispatch maps, built once per parse in readLaTeX (and once per chunk in rawLaTeXParser's callers), and consult it via askEnv at the dispatch sites. Internal `lift $ runParserT` sub-parses share the environment automatically. Benchmark (1.8MB LaTeX file): allocations drop from 17.85GB to 9.97GB and wall time from ~4.7s to ~3.4s. On a stress test of 100k \emph commands, allocations drop from 36GB to 10.5GB. Co-Authored-By: Claude <noreply@anthropic.com>
Pandoc
The universal markup converter
Pandoc is a Haskell library for converting from one markup format to another, and a command-line tool that uses this library.
It can convert from
asciidoc(AsciiDoc markup)bibtex(BibTeX bibliography)biblatex(BibLaTeX bibliography)bits(BITS XML, alias forjats)commonmark(CommonMark Markdown)commonmark_x(CommonMark Markdown with extensions)creole(Creole 1.0)csljson(CSL JSON bibliography)csv(CSV table)tsv(TSV table)djot(Djot markup)docbook(DocBook)docx(Word docx)dokuwiki(DokuWiki markup)endnotexml(EndNote XML bibliography)epub(EPUB)fb2(FictionBook2 e-book)gfm(GitHub-Flavored Markdown), or the deprecated and less accuratemarkdown_github; usemarkdown_githubonly if you need extensions not supported ingfm.haddock(Haddock markup)html(HTML)ipynb(Jupyter notebook)jats(JATS XML)jira(Jira/Confluence wiki markup)json(JSON version of native AST)latex(LaTeX)markdown(Pandoc’s Markdown)markdown_mmd(MultiMarkdown)markdown_phpextra(PHP Markdown Extra)markdown_strict(original unextended Markdown)mediawiki(MediaWiki markup)man(roff man)mdoc(mdoc manual page markup)muse(Muse)native(native Haskell)odt(OpenDocument text document)opml(OPML)org(Emacs Org mode)pod(Perl’s Plain Old Documentation)pptx(PowerPoint)ris(RIS bibliography)rtf(Rich Text Format)rst(reStructuredText)t2t(txt2tags)textile(Textile)tikiwiki(TikiWiki markup)twiki(TWiki markup)typst(typst)vimwiki(Vimwiki)xlsx(Excel spreadsheet)xml(XML version of native AST)- the path of a custom Lua reader, see Custom readers and writers below
It can convert to
ansi(text with ANSI escape codes, for terminal viewing)asciidoc(modern AsciiDoc as interpreted by Asciidoctor)asciidoc_legacy(AsciiDoc as interpreted byasciidoc-py).asciidoctor(deprecated synonym forasciidoc)bbcodeBBCodebbcode_fluxbbBBCode (FluxBB)bbcode_phpbbBBCode (phpBB)bbcode_steamBBCode (Steam)bbcode_hubzillaBBCode (Hubzilla)bbcode_xenforoBBCode (xenForo)beamer(LaTeX beamer slide show)bibtex(BibTeX bibliography)biblatex(BibLaTeX bibliography)chunkedhtml(zip archive of multiple linked HTML files)commonmark(CommonMark Markdown)commonmark_x(CommonMark Markdown with extensions)context(ConTeXt)csljson(CSL JSON bibliography)djot(Djot markup)docbookordocbook4(DocBook 4)docbook5(DocBook 5)docx(Word docx)dokuwiki(DokuWiki markup)epuborepub3(EPUB v3 book)epub2(EPUB v2)fb2(FictionBook2 e-book)gfm(GitHub-Flavored Markdown), or the deprecated and less accuratemarkdown_github; usemarkdown_githubonly if you need extensions not supported ingfm.haddock(Haddock markup)htmlorhtml5(HTML, i.e. HTML5/XHTML polyglot markup)html4(XHTML 1.0 Transitional)icml(InDesign ICML)ipynb(Jupyter notebook)jats_archiving(JATS XML, Archiving and Interchange Tag Set)jats_articleauthoring(JATS XML, Article Authoring Tag Set)jats_publishing(JATS XML, Journal Publishing Tag Set)jats(alias forjats_archiving)jira(Jira/Confluence wiki markup)json(JSON version of native AST)latex(LaTeX)man(roff man)markdown(Pandoc’s Markdown)markdown_mmd(MultiMarkdown)markdown_phpextra(PHP Markdown Extra)markdown_strict(original unextended Markdown)markua(Markua)mediawiki(MediaWiki markup)ms(roff ms)muse(Muse)native(native Haskell)odt(OpenDocument text document)opml(OPML)opendocument(OpenDocument XML)org(Emacs Org mode)pdf(PDF)plain(plain text)pptx(PowerPoint slide show)rst(reStructuredText)rtf(Rich Text Format)texinfo(GNU Texinfo)textile(Textile)t2t(txt2tags)slideous(Slideous HTML and JavaScript slide show)slidy(Slidy HTML and JavaScript slide show)dzslides(DZSlides HTML5 + JavaScript slide show)revealjs(reveal.js HTML5 + JavaScript slide show)s5(S5 HTML and JavaScript slide show)tei(TEI Simple)typst(typst)vimdoc(Vimdoc)xml(XML version of native AST)xwiki(XWiki markup)zimwiki(ZimWiki markup)- the path of a custom Lua writer, see Custom readers and writers below
Pandoc can also produce PDF output via LaTeX, Groff ms, or HTML.
Pandoc’s enhanced version of Markdown includes syntax for tables, definition lists, metadata blocks, footnotes, citations, math, and much more. See the User’s Manual below under Pandoc’s Markdown.
Pandoc has a modular design: it consists of a set of readers, which parse text in a given format and produce a native representation of the document (an abstract syntax tree or AST), and a set of writers, which convert this native representation into a target format. Thus, adding an input or output format requires only adding a reader or writer. Users can also run custom pandoc filters to modify the intermediate AST (see the documentation for filters and Lua filters).
Because pandoc’s intermediate representation of a document is less expressive than many of the formats it converts between, one should not expect perfect conversions between every format and every other. Pandoc attempts to preserve the structural elements of a document, but not formatting details such as margin size. And some document elements, such as complex tables, may not fit into pandoc’s simple document model. While conversions from pandoc’s Markdown to all formats aspire to be perfect, conversions from formats more expressive than pandoc’s Markdown can be expected to be lossy.
Installing
Here’s how to install pandoc.
Documentation
Pandoc’s website contains a full User’s Guide. It is also available here as pandoc-flavored Markdown. The website also contains some examples of the use of pandoc, a limited online demo, and a WebAssembly-based online demo.
Contributing
Pull requests, bug reports, and feature requests are welcome. Please make sure to read the contributor guidelines before opening a new issue.
License
© 2006-2024 John MacFarlane (jgm@berkeley.edu). Released under the GPL, version 2 or greater. This software carries no warranty of any kind. (See COPYRIGHT for full copyright and warranty notices.)