jgm/pandoc 2.8

pandoc 2.8

on GitHub

Improvements in templates system (from doctemplates):

• Pandoc templates now support a number of new features that have been added in doctemplates: notably, elseif, it, partials, filters, and syntax to control nesting and reflowing of text. These changes make pandoc more suitable out of the box for generating plain-text documents from data in YAML metadata. It can create enumerated lists and even tabular structures.
• We now used templates parameterized on doclayout Doc types. The main impact of this change is better reflowing of content interpolated into templates. Previously, interpolated variables were rendered independently and intepolated as strings, which could lead to overly long lines. Now the templates interpolated as Doc values which may include breaking spaces, and reflowing occurs after template interpolation rather than before.
• Remove code from the LaTeX, Docbook, and JATS writers that looked in the template for strings to determine whether it is a book or an article, or whether csquotes is used. This was always kludgy and unreliable.
• Change template code to use new API for doctemplates.

Add --defaults/-d option. This adds the ability to specify a collection of default values for options in a YAML file. For example, one might define a set of defaults for letters, and then do pandoc -d letter myletter.md -o myletter.pdf. See the documentation of this feature in MANUAL.txt.

Raise error on unsupported extensions (#4338).

The --list-extensions[=FORMAT] option now lists only extensions that affect the given FORMAT.

Add -L option as shortcut for --lua-filter.

Add --shift-heading-level-by option and deprecate --base-heading-level (#5615). The new option does everything the old one does, but also allows negative shifts. It also promotes the document metadata (if not null) to a level-1 heading with a +1 shift, and demotes an initial level-1 heading to document metadata with a -1 shift. This supports converting documents that use an initial level-1 heading for the document title.

Allow --metadata-file to be used repeatedly to include multiple metadata files (Owen McGrath, #5702). Values in files specified first will be overridden by those in later files.

--ascii now uses numerical hex character references (#5718).

Allow PDF output to stdout (#5751). PDF output now behaves like other binary formats: it will not be output to the terminal, but can be sent to stdout using either -o - or a pipe. The intermediate format will be determined based on the setting of --pdf-engine.

Make some writers sensitive to ‘unlisted’ class on headings (#1762). If this is present on a heading with the ‘unnumbered’ class, the heading won’t appear in the TOC. This class has no effect if ‘unnumbered’ is not also specified. This affects HTML-based writers (including slide shows and EPUB), LateX (including beamer), RTF, and PowerPoint. Other writers do not yet support unlisted.

Fix gfm_auto_identifiers behavior with emojis (#5813). Note that we also now use emoji names for emojis when ascii_identifiers is enabled.

When --ipynb-output is used with the default “best” format, strip ANSI escape codes for non-ipynb output (#5633). These cause problems in many formats, including LaTeX.

Don’t look for template files remotely for remote input (#5579). Previously pandoc would look for the template at a remote URL when a URL was used for the input file, instead of taking it from the data directory.

Allow combining -Vheader-includes and --include-in-header (#5904). Previously header-includes set as a variable would be clobbered by material included using --include-in-header.

Change merge behavior for metadata. Previously, if a document contained two YAML metadata blocks that set the same field, the conflict would be resolved in favor of the first. Now it is resolved in favor of the second (due to a change in pandoc-types). This makes the behavior more uniform with other things in pandoc (such as reference links and --metadata-file).

Don’t add a newline to fragment output if there’s already one.

Change exit codes and document in MANUAL.txt:

• PandocAppError was 1, is now 4
• PandocOptionError was 2, is now 6
• PandocMakePDFError was 65, is now 66

Switch to new pandoc-types and use Text instead of String [API change]. (Christian Despres, #5884).

HTML reader:

• Better handling of <q> with cite attribute (#5798, Ole Martin Ruud). If a <q> tag has a cite attribute, we interpret it as a Quoted element with an inner Span.
• Add support for HTML <samp> element (#5792, Amogh Rathore). The <samp> element is parsed as Code with class sample.
• Add support for HTML <var> element (#5799, Amogh Rathore). The <var> element is parsed as Code with class variable.
• Add support for <mark> elements (Florian B, #5797). Parse <mark> elements from HTML as Spans with class mark.
• Add support for <kbd> elements, parsing them as Span with class kbd (Daniele D’Orazio, #5796).
• Add support for <dfn>, parsing this as a Span with class dfn (#5882, Florian Beeres).

Markdown reader:

• Headers: don’t parse content over newline boundary (#5714).
• Handle inline code more eagerly within lists (Brian Leung, #5627).
• Removed some needless lookaheads.
• Don’t parse footnote body unless extension enabled.
• Fix small super/subscript issue (#5878). Superscripts and subscripts cannot contain spaces, but newlines were previously allowed (unintentionally). This led to bad interactions in some cases with footnotes. With this change newlines are also not allowed inside super/subscripts.
• Use take1WhileP for str, table row. This yields a small but measurable performance improvement.

LaTeX reader:

• Fix parsing of optional arguments that contain braced text (#5740).
• Don’t try to parse includes if raw_tex is set (#5673). When the raw_tex extension is set, we just carry through \usepackage, \input, etc. verbatim as raw LaTeX.
• Properly handle optional arguments for macros (#5682).
• Fix \\ in \parbox inside a table cell (#5711).
• Improve withRaw so it can handle cases where the token string is modified by a parser (e.g. accent when it only takes part of a Word token) (#5686). This fixes a bug that caused the ends of certain documents to be dropped.
• Handle \passthrough macro used by latex writer (#5659).
• Support tex \tt command (#5654).
• Search for image with list of extensions like latex does, if an extension is not provided (#4933).
• Handle \looseness command values better (#4439).
• Add mbox and hbox handling (Vasily Alferov, #5586). When +raw_tex is enabled, these are passed through literally. Otherwise, they are handled in a way that emulates LaTeX’s behavior.
• Properly handle \providecommand and \provideenvironment (#5635). They are now ignored if the corresponding command or environment is already defined.
• Support epigraph command in LaTeX Reader (oquechy, #3523).
• Ensure that expanded macros in raw LaTeX end with a space if the original did (#4442).
• Treat ly environment from lilypond as verbatim (Urs Liska, #5671).
• Add tikzcd to list of special environments (Eigil Rischel). This allows it to be processed by filters, in the same way that one can do for tikzpicture.

Roff reader:

• Better support for while.
• More improvements in parsing conditionals.
• Fix problem parsing comments before macro.
• Improve handling of groups.
• Better parsing of groups (#5410). We now allow groups where the closing \\} isn’t at the beginning of a line.

RST reader:

• Keep name property in imgAttr (Brian Leung, #5619).
• Fixed parsing of indented blocks (#5753). We were requiring consistent indentation, but this isn’t required by RST.
• Use title, not admonition-title, for admonition title. This puts RST reader into alignment with docbook reader.
• Don’t strip final underscore from absolute URI (#5763).
• Avoid spurious warning when resolving links to internal anchors ending with _ (#5763).

Org reader:

• Accept ATTR_LATEX in block attributes (Albert Krewinkel, #5648). Attributes for LaTeX output are accepted as valid block attributes; however, their values are ignored.
• Modify handling of example blocks (Brian Leung, #5717).
• Allow the -i switch to ignore leading spaces (Brian Leung).
• Handle awkwardly-aligned code blocks within lists (Brian Leung). Code blocks in Org lists must have their #+BEGIN_ aligned in a reasonable way, but their other components can be positioned otherwise.
• Fix parsing of empty comment lines (#5856, Albert Krewinkel). Comment lines in Org-mode can be completely empty.

Muse reader (Alexander Krotov):

• Add RTL support (#5551).
• Do not allow closing asterisks to be followed by *.
• Do not split series of asterisks into symbols and emphasis (#5821).
• Do not terminate emphasis on * not followed by space.

DokuWiki reader:

• Parse markup inside monospace (’’) (#5916, Alexander Krotov).

Docx reader:

• Move style-parsing-specific code to a new unexported module, Text.Pandoc.Readers.Docx.Parse.Styles (Nikolay Yakimov).
• Move StyleMap to docx writer (Nikolay Yakimov).
• Only use LTR when it is overriding BiDi setting (#5723, Jesse Rosenthal). The left-to-right direction setting in docx is used in the spec only for overriding an explicit right-to-left setting. We only process it when it happens in a paragraph set with BiDi. This is especially important for docs exported from Google Docs, which explicitly (and unnecessarily) set rtl=0 for every paragraph.
• Fix list number resumption for sublists (#4324). The first list item of a sublist should not resume numbering from the number of the last sublist item of the same level, if that sublist was a sublist of a different list item.

Docbook reader:

• Richer parse for admonitions (Michael Peyton Jones, #1234). Instead of parsing admonitions as blockquotes, we now parse them as Divs with an appropriate class. We also handle titles for admonitions as a nested Div with the “title” class.
• Fix nesting of chapters and sections (#5864, Florian Klink, Félix Baylac-Jacqué).
• Fix bug with entities in mathphrase element (#5885).

MediaWiki reader:

• Skip optional {{table}} template (#5757).

LaTeX reader:

• Fix dollar-math parsing to ensure that space is left between a control sequence and a following letter (#5836).
• In untokenize, ensure space between control sequence and following letter (#5836).
• Don’t omit macro definitions defined in the preamble. These were formerly omitted (though they still affected macro resolution if latex_macros was set). Now they are included in the document body.
• Parse macro definitions as raw LaTeX when latex_macros is disabled. (When latex_macros is enabled, we omit them, since pandoc is applying the macros itself.)
• Fix a hang/memory leak in certain circumstances (#5845).
• Text.Pandoc.Readers.LaTeX.Parsing: add [Tok] parameter to rawLaTeXParser. This allows us to repeat retokenizing unnecessarily in e.g. rawLaTeXBlock.
• Add KOMA-Script metadata commands (#5910, Andrew Dunning). Add all titling commands to existing definition for \dedication.
• Parse \micro siunitx unit command (#5921, Jose Luis Duran).

Markdown writer:

• Ensure proper nesting when we have long ordered list markers (#5705).
• Make plain output plainer (#5741). Previously we used the following Project Gutenberg conventions for plain output: extra space before and after level 1 and 2 headings, all-caps for strong emphasis, underscores surrounding regular emphasis. Now these conventions are used only when the gutenberg extension is enabled. By default, Strong and Emph are rendered without special formatting, and headings are rendered without special formatting, and with only one blank line following. To restore the former behavior, use -t plain+gutenberg.
• Prefer using raw_attribute when enabled (#4311). The raw_attribute will be used to mark raw bits, even HTML and LaTeX, and even when raw_html and raw_tex are enabled, as they are by default. To get the old behavior, disable raw_attribute in the writer.
• Prefer pipe_tables to raw HTML even when we must lose width information (#2608, #4497).
• Improve escaping in list items (#5918).

AsciiDoc writer:

• Don’t include + in code blocks for regular asciidoc. This is asciidoctor-specific.
• Handle admonitions (#5690).

LaTeX writer:

• Add thin space when needed in LaTeX quote ligatures (#5684).
• Use \hspace{0pt} for 0-width space U+200B (#5756).
• Use cslreferences environment for csl bibliographies. This allows bibliographies to receive special formatting. The template now contains definition of this environment (enabled only when CSL is used). It also defines a \cslhangindent length. This is set to 2em by default when the bibliography style specifies a hanging indent. To override the length, you can use e.g. \setlength{\cslhangindent}{7em} in header-includes. See jgm/pandoc-citeproc#410.
• Strip off {} around locator for biblatex/natbib output (#5722).
• Fix line breaks at start of paragraph (#3324). Previously we just omitted these. Now we render them using \hfill\break instead of \\. This is a revision of a PR by @sabine (#5591) who should be credited with the idea.
• We no longer look in the template or header-includes to see if a book or article documentclass is used, or to see whether the csquotes package is used. To use csquotes for LaTeX, set csquotes in your variables or metadata. To specify a book style, use the documentclass variable or --top-level-division.
• Fix horizontal rule (#5801). We change to use 0.5pt rather than \linethickness, which apparently only ever worked “by accident” and no longer works with recent updates to texlive.

ConTeXt writer:

• Add option to include source files in ConTeXt PDFs (Tristan Stenner, #5578). The metadata field or variable (includesource) can be set to attach the source documents to the resulting PDF.
• Customizable type of PDF/A for the ConTeXt writer (Karl Pettersson, #5608). The pdfa variable may now be set in metadata. Also updated color profile settings in accordance with ConTeXt wiki, and made ICC profile and output intent for PDF/A customizable using pdfaiccprofile and pdfaintent.
• Unit tests: adjust code property to avoid an irrelevant failure involving inline code with two consecutive newlines.
• Set csl-hanging-indent variable if needed.
• Use special environment for CSL references.
• Use braces, not start/stop, for inline language tags. This prevents unwanted gobbling of spaces.

HTML writer:

• Use numeric character references with --ascii (#5718). Previously we used named character references with html5 output. But these aren’t valid XML, and we aim to produce html5 that is also valid XHTML (polyglot markup). (This is also needed for epub3.)
• Ensure that line numbers in code blocks get id-prefix (#5650).
• Ensure TeX formulas are rendered correctly (Philip Pesca, #5658). The web service passed in to --webtex may render formulas using inline or display style by default. Prefixing formulas with the appropriate command ensures they are rendered correctly.
• Render inline formulas correctly with --webtex (Philip Pesca, #5655). We add \textstyle to the beginning of the formula to ensure it will be rendered in inline style.
• Pass through aria- attributes to HTML5 (#5642).
• Render a Quoted element with an inner Span with cite attribute using a <q> tag (#5798, Ole Martin Ruud).
• Render a Span with class mark using the <mark> element (Florian Beeres, #5797).
• Render Span with class dfn using <dfn> element (Florian Beeres, #5882).
• Render Span with class kbd using <kbd> element (Daniele D’Orazio, #5796).
• Render Code with class variable using <var> element (Amogh Rathore, #5799).
• Render Code with class sample using <samp> element (Amogh Rathore, #5799).

EPUB writer:

• Improve splitting into chapters (#5761), using makeSection.
• Avoid issuing warning multiple times when title not set (see #5760).
• Use svg tag wrapper for cover image (#5638). In addition, the code generating the image has been moved to the template, to make it more customizable. NOTE: Those who use custom EPUB templates will need to adjust their templates, adding the code to generate the cover image. (Previously this was just inserted into ‘body’.)
• Improve toChapters, making it work better if there are Divs around sections.
• Add support for EPUB2 covers (blmage, #3992).
• Do not override existing “fileN” medias when writing to EPUB format (blmage, #4206).
• Ensure that lang variable is set on all chapters (so that it will add an xml:lang attribute on the html element).

RST writer:

• Removed remnants of admonition-title.
• Fix handling of :align: on figures and images (#4420). When the image has the align-right (etc.) class, we now use an :align: attribute.
• Improve spacing for tables with no width information (#5899). If a simple table would be too wide, we use a grid table.
• Fix backslash escaping after strings (Albert Krewinkel, #5906). The check whether a complex inline element following a string must be escaped, now depends on the last character of the string instead of the first.
• Ensure there’s a blank line before tables (#5898).

Dokuwiki writer:

• Handle mixed lists without HTML fallback (#5107).

XWiki writer:

• Fix multiline table (Zihang Chen, #5683).

Muse writer:

• Add RTL support (Alexander Krotov, #5551).

Man writer:

• Suppress non-absolute link URLs (#5770). Absolute URLs are still printed in parentheses following the link text, but relative URLs are suppressed (just as internal links starting with ‘#’ always have been).
• Improved definition list term output. Now we boldface code but not other things. This matches the most common style in man pages (particularly option lists).

Ms writer:

• Use .LP instead of .PP for line block (#5588).
• Use boldface for definition terms in DefinitionLists.

JATS writer:

• Do not emit empty <fn-group> (Mauro Bieg, #5595).
• Update template to v1.1dtd (#5632, Arfon Smith).
• Update data/jats.csl to avoid commas between editor name-part elements. (#5629)
• Add abstract to template (Mauro Bieg).

TEI writer:

• Don’t strip hash from internal links (#5922).

Jira writer:

• Remove escapeStringForJira for code blocks (Jan-Otto Kröpke).
• Remove extraneous newline after single-line block quotes (#5858, Albert Krewinkel).
• Improve escaping of special characters, using backslash escapes instead of HTML entities (#5858, Albert Krewinkel).

OpenDocument writer:

• Avoid duplicate attributes (#4634). We use the innermost attribute in nested cases.
• If native_numbering extension is set, use native OpenDocument enumeration for figures and tables (Nils Carlson).
• Place caption before table (#5681, Dmitry Pogodin).

ODT writer:

• Add a test for MathML formulas in ODT documents (blmage).
• Improve the parsing of frames in ODT documents (blmage).

Docx writer:

• Make handling of styles more robust in localized versions of Word (Nikolay Yakimov, #5523, #5052, #5074). We now use style names, not ids, for assigning semantic meaning, since the ids can change depending on the locale. Style name comparisons are case-insensitive, since those are case-insensitive in Word. Since docx style names can have spaces in them, and pandoc-markdown classes can’t, anywhere when style name is used as a class name, spaces are replaced with ASCII dashes -. Code styles, i.e. “Source Code” and “Verbatim Char” now honor style inheritance. Docx Reader now honours “Compact” style (used in Pandoc-generated docx). The side-effect is that “Compact” style no longer shows up in docx+styles output. Styles inherited from “Compact” will still show up.
• Re-use Readers.Docx.Parse for StyleMap (#5766, Nikolay Yakimov).
• Internal improvements and code simplification (Nikolay Yakimov).
• Preserve built-in styles in DOCX with custom style (Ben Steinberg, #5670). This change prevents custom styles on divs and spans from overriding styles on certain elements inside them, like headings, blockquotes, and links. On those elements, the “native” style is required for the element to display correctly. This change also allows nesting of custom styles; in order to do so, it removes the default “Compact” style applied to Plain blocks, except when inside a table.
• Add proofState to list of elements carried over from settings.xml in the reference.docx (Krystof Beuermann, #5703).
• Change order of ilvl and numId in document.xml (Agustín Martín Barbero, #5645). Also, make list para properties go first. This reordering of properties shouldn’t be necessary but it seems Word Online does not understand the docx correctly otherwise.

PowerPoint writer:

• Code formatting is now context dependent (Jeroen de Haas, #5573). This commit alters the way in which the Powerpoint writer treats inline code and code blocks. Inline code is now formatted at the same size as the surrounding text. Code blocks are now given a margin and font size according to their level. Furthermore this commit allows changing the font with which code is formatted via the monofont option.
• Start numbering at appopriate numbers (Jesse Rosenthal, #5709). Starting numbers for ordered lists were previously ignored. Now we specify the number if it is something other than 1.

Text.Pandoc.Parsing:

• Add manyChar, many1Char, manyTillChar, many1TillChar, many1Till, manyUntil, mantyUntilChar: these are like their unsuffixed counterparts but pack some or all of their output (Christian Despres, #5884).
• Add stateAllowLineBreaks to ParserState [API change].
• Fix inline parsing in grid table cells (#5708).
• Change type of setLastStrPos so it takes a Maybe SourcePos rather than a SourcePos [API change].
• Make parseFromString' and gridTableWith and gridTableWith' polymorphic in the parser state, constraining it with HasLastStrPosition [API change].
• parseFromString': reset stateLastStrPos to Nothing before parse.
• Rename takeWhileP -> take1WhileP and clean it up. (It doesn’t match the empty sequence.)

Text.Pandoc.PDF:

• For PDFs via HTML, ensure temp file is deleted even if the pdf program is not found (#5720).
• Better detection of a Cygwin environment (#5451).
• Don’t assume tex log file is UTF8-encoded (#5872). Fall back to latin1 if it can’t be read as UTF-8.

Text.Pandoc.Extensions:

• Export new function getAllExtensions, which returns the extensions that affect a given format (whether enabled by default or not) [API change].
• Change type of parseFormatSpec from Either ParseError (String, Extensions -> Extensions) to Either ParseError (String, [Extension], [Extension]) [API change].
• Add Ext_gutenberg constructor to Extension [API change].
• Add Ext_native_numbering constructor to Extension [API change] (Nils Carlson).

Text.Pandoc.Readers, Text.Pandoc.Writers:

• Change type of getReader and getWriter so they return a value in the PandocMonad instance rather than an Either [API change]. Exceptions for unknown formats and unsupported extensions are now raised by these functions.

Text.Pandoc.App

• Change optMetadataFile type from Maybe FilePath to [FilePath] (Owen McGrath, #5702) [API change].

Text.Pandoc.Logging:

• Add CouldNotDeduceFormat constructor to LogMessage [API change]. Issue this warning when we’re falling back to markdown or html because we don’t recognize the extension of the input or output files.
• Clarify warning for missing title (#5760).
• Add UnusualConversion constructor to LogMessage [API change] (Mauro Bieg, #5736). Emit warning on -f latex -o out.pdf.

Lua filters:

• Improve function documentation (Albert Krewkinkel).

• Traverse nested blocks and inlines in correct order (Albert Krewinkel, #5667). Traversal methods are updated to use the new Walk module so that sequences with nested Inline (or Block) elements are traversed in the order in which they appear in the linearized document.

• New unexported module Text.Pandoc.Lua.Walk (Albert Krewinkel). Lua filters must be able to traverse sequences of AST elements and to replace elements by splicing sequences back in their place. Special Walkable instances can be used for this; those are provided in a new module Text.Pandoc.Lua.Walk.

• Attr values can now be given as normal Lua tables (Albert Krewinkel, #5744). This can be used as a convenient alternative to constructing Attr values with pandoc.Attr. Identifiers are taken from the id field, classes must be given as space separated words in the class field. All remaining fields are included as attributes. With this change, the following lines now create equal elements:

  pandoc.Span('test', {id = 'test', class = 'a b', check = 1})
  pandoc.Span('test', pandoc.Attr('test', {'a','b'}, {check = 1}))
  

  This also works when using the attr setter:

  local span = pandoc.Span 'text'
  span.attr = {id = 'test', class = 'a b', check = 1}
  

  Furthermore, the attributes field of AST elements can now be a plain key-value table even when using the attributes accessor:

  local span = pandoc.Span 'test'
  span.attributes = {check = 1}   -- works as expected now
  

• Export make_sections, remove hierarchicalize. Lua filters that use hierarchicalize will need to be rewritten to use make_sections.

• Add a clone() method to all AST elements (Albert Krewinkel, #5568).

• Fix Lua function names in pandoc.system (niszet). Change get_current_directory to get_working_directory and with_temp_directory to with_temporary_directory, to conform to the manual.

Text.Pandoc.Error:

• Add constructors PandocUnknownReaderError, PandocUnknownWriterError, PandocUnsupportedExtensionError. [API change].
• Better message for PandocShouldNeverHappenError.
• Better message for PandocTemplateError.

Text.Pandoc.Emoji:

• Update emoji list (#5666). Done using new tools/emojis.hs, which uses the list from the gem GitHub uses. Future updates can be done with this tool.

Text.Pandoc.PDF:

• Pass value of --dpi to rsvg-convert when converting SVG to PDF in the process of creating a PDF (#5721).

Text.Pandoc.Shared:

• Replace Element and makeHierarchical with makeSections. Now that we have Divs, we can use them to represent the structure of sections, and we don’t need a special Element type. makeSections reorganizes a block list, adding Divs with class section around sections, and adding numbering if needed. This change also fixes some longstanding issues recognizing section structure when the document contains Divs (#3057, see also #997).
• Remove Element type [API change]
• Remove makeHierarchicalize [API change]
• Add makeSections [API change]
• Export deLink [API change]
• Make filterIpynbOutput strip ANSI escapes from code in output for non-ipynb formats, when the default “best” option is used with --ipynb-output (#5633).
• Fix camelCaseToHyphenated so it handles ABCDef better.
• Improve isTight (#5857). If a list has an empty item, this should not count against its being a tight list.
• Export htmlSpanLikeElements [API change] (Daniele D’Orazio, #5796). This is a mapping of HTML span-like elements that are internally represented as a Span with a single class.
• Change the implementation of htmlSpanLikeElements to retain classes and attributes (#5882, Florian Beeres).

Text.Pandoc.Slides: recognize content in Divs when determining slide level.

Text.Pandoc.SelfContained:

• Omit content-type on type attribute for <style> (#5725). It doesn’t seem to be valid for HTML5, and as a result Chrome ignores the style element.

Text.Pandoc.Pretty has been removed [API change]. We now use the new external doclayout module instead.

Text.Pandoc.Writers.Shared:

• Remove metaToJSON, metaToJSON' [API change].
• Modify addVariablesToContext, defField, setField, getField, resetField to work with Context rather than JSON values. [API change]
• Export new function endsWithPlain [API change].
• Change gridTables so it does better at keeping the widths of columns (#4320) and does better at figuring out column widths when no widths are given (#5899).

Text.Pandoc.Options

• Change type of writerTemplate to Maybe Template instead of Maybe String.
• Change To/FromJSON instances for HTMLMathMethod, CiteMethod, ObfuscationMethod, TrackChanges, WrapOption, TopLevelDivision, ReferenceLocation, HTMLSlideVariant (#5790). In each case we use lowercase (or hyphenated lowercase) for constructors to line up more closely with command-line option values. This is a breaking change for those who manually decode or encode JSON for these data types (e.g. for ReaderOptions or WriterOptions).

Text.Pandoc.Filters:

• Add FromYAML instance for Filter.
• applyFilters: Add and apply filters in order (not reversed) This changes applyFilters from Text.Pandoc.Filter so that it does a left fold rather than a right fold, applying the filters in the order listed.

Text.Pandoc.XML:

• Change toEntities to emit numerical hex character references (#5718).

Text.Pandoc.Highlighting:

• Add additional listings languages (Wandmalfarbe).

Text.Pandoc.MediaBag:

• Some of the types using Strings were switched to use FilePath instead (Christian Despres, #5884).

Text.Pandoc.Templates:

• Don’t import/export varListToJSON [API change]. It is removed in doctemplates >= 0.3.
• Change type of renderTemplate' [API change]. Return value is now Text rather than being polymorphic. This makes room for upcoming removal of the TemplateTarget class from doctemplates.

Text.Pandoc.App.Opt API changes:

• More convenient To/FromJSON instances. Make the field names like strip-empty-paragraphs rather than optStripEmptyParagraphs. Anyone who is using JSON serialization of Opt will need to adjust things accordingly.
• Change optHighlightStyle to a Maybe String instead of Maybe Style. Do the parsing/loading of themes later, after option parsing.
• Remove optBaseHeaderLevel from Opt. We now just use optShiftHeadingLevelBy, to avoid redundancy.
• Change optShiftHeadingLevel to optShiftHeadingLevelBy to match the option.
• Custom FromJSON instance for LineEnding, so either CRLF or crlf will work.
• Change optVariables from [(String, String)] to Context Text.
• Change optMetadata to Meta, to allow structured values. The current behavior of the --metadata option stays the same.
• Rename optReader, optWriter as optFrom, optTo.
• Add FromYAML instances to Opt and to all subsidiary types.
• Rename optMetadataFile to optMetadataFiles.
• Rename optPDFEngineArgs to optPDFEngineOpts.
• Rename optWrapText to optWrap.
• Add IpynbOutput enumerated type: use this instead of a string for optIpynbOutput.
• Change optInputFiles to a Maybe [FilePath] (#5888) [API change]. Nothing means: nothing specified. Just [] means: an empty list specified (e.g. in defaults).
• List fields in Opt so they aren’t reversed (#5881) [API change]. Previously optIncludeInHeader, etc. were in reverse order.
• The sourcefile variable is now always a list. It used to be sometimes a string, sometimes a list (when there was more than one).

Template changes:

• default.latex: added code for cslreferences environment, to be used for pandoc-citeproc references. A csl-hanging-indent variable (set automatically if there is a hanging-ident class on the references Div) controls whether contents of this environment receive a hanging indent.
• default.latex: Add space as default option for xeCJK, so that spaces between words are preserved (#5855, jeongminkim-islab). This is necessary for Korean.
• default.latex: Remove include of grffile (#5848). This package used to be needed for proper handling of image filenames containing periods (in addition to the period before the extension). It no longer works with the latest LaTeX kernel and graphicx, so we have removed it. Future versions of graphicx will handle these filenames without the need for grffile.
• default.context: add a saner default for page numbers. Previously they appeared centered at the top of the page; now we put them centered at the bottom, unless the pagenumbering variable is set.
• default.context: define a start-stop-pair cslreferences to allow for hanging indents in the bibliography (#5875, Denis Maier).
• default.ms: update defaults. Use Palatino font, use slightly wider interparagraph space, don’t indent paragraphs, and put page numbers on the bottom. This brings ms output closer to default LaTeX output.
• default.revealjs: add navigationMode (Mauro Bieg, #5657).
• default.muse: handle multiple authors better.
• docbook4, docbook5 templates: add indentation to body.
• HTML-based templates: use styles.html partial to avoid code duplication.
• HTML-based templates: change indentation of styles in template.

reference.docx (#5820):

• Change Block Text (block quote) style so that the same font is used as in the body text, and the block text is indented left and right.
• All headings now have a uniform color.
• Level-1 headings no longer set w:themeShade="B5".
• Level-2 headings are now 14 point rather than 16 point.
• Level-3 headings are now 12 point rather than 14 point.
• Level-4 headings are italic rather than bold.

epub.css: Add CSS for hanging-indent div to support pandoc-citeproc’s new hanging indents.

pandoc.cabal:

• Repeat ghc-options in all stanzas.
• Remove conditionals for ghc < 8 (Albert Krewinkel, #5693).
• Compile with -Wcpp-undef and -fhide-source-paths when possible (Albert Krewinkel).
• Add cabal.project to extra-source-files (Albert Krewinkel).
• Add dependency on skylighting-core (#5729). Even though it shouldn’t be needed, some tools require it.
• Require latest pandoc-types, texmath, skylighting, haddock-library.
• Ensure TemplateHaskell is added to other-extensions when it is used (Vanessa McHale, #5728).
• Remove derive_json_via_th flag; always use TH. This cuts down on code duplication and reduces the chance for errors. See #4083.

Makefile:

• Add ghci target.
• Add ghcid target.
• Remove references to obsolete flag (#5694).

Benchmarks: fix failure on ipynb.

Use MathJax 3 (zorkow).

KaTeX math: respect classoption=fleqn variable, bump KaTeX version to 0.11.1 (#5815, Mauro Bieg).

Fix redundant constraint compiler warnings (Pete Ryland, #5625).

Use throwError instead of fail when appropriate.

Use Prelude.fail to avoid ambiguity with fail from GHC.Base.

Add diff-zip.sh to tools (John MacFarlane, Agustín Martín Barbero). This is intended to make it easier to test differences in zip containers, like epub, docx, or odt.

Add .gitattributes (#5747). This ensures that the golden files in test/fb2/reader/ don’t have newlines converted. This should fix a test failure on GitHub CI with Windows.

Reorder options in --help.

Revise code for HsYAML-0.2.0.0 (@vijayphoenix, #5704).

Remove blank line in code example in Haddocks (Leif Metcalf, #5679).

Fix trypandoc with getReader/getWriter changes.

Allow building pandoc with GHC 8.8.

linux tarball: add architecture -amd64 to filename. Now it will be: pandoc-VERSION-linux-ARCH.tar.gz.

MANUAL.txt:

• Add section for exit codes.
• Document some pptx limitations in slide show section: No incremental display (#5689). No pause with . . . (#5701).
• reveal.js flags (Mauro Bieg, #5653).
• Document addition of data- prefix to unknown attributes in HTML5.
• Link to YAML spec (Mauro Bieg, #5687).
• Fix capitalization of “Linux” (#5859, Marcus Stollsteimer).
• Use a table for exit codes.
• Put all template variable docs into one section.
• Use ATX headers consistently.
• Add fuller documentation of templates (#5779), including new template syntax, partials, etc.
• Add documentation for the variable hyperrefoptions (Wandmalfarbe).
• Clarify when macro definitions are passed as raw latex. In Markdown input, they are always passed through. In LaTeX, only if latex_macros is disabled.
• Clarify that --dpi provides a default and doesn’t override dpi values specified in the images themselves (#5721).
• Document how to use custom writers with --standalone (#5866).
• Clarify --preserve-tabs default.

INSTALL.md:

• Fix instructions for libicu.
• Add Void Linux instructions (Volodymyr Kozachnyskyi).

CONTRIBUTING.md:

• Add information on tests (Agustín Martín Barbero, #5652).
• Add information about command test naming to CONTRIBUTING (Florian B).

Fix typos in changelog and comments (#5896, Brian Wignall).

doc/lua-filters.md:

• Fix mistakes in mediabag module docs (#5851, Albert Krewinkel).
• Improve metadata replacement example in lua-filters doc (#5851).
• Mention which Lua version is shipped with pandoc (Albert Krewinkel, #5892).