Theorem environments no longer work in LaTeX/PDF output with Pandoc >= 2.7.3

[yihui]

Because Pandoc converts <div class='foo'></div> to \begin{foo}\end{foo} now.

I'm a little surprised that no one has reported it so far. Perhaps this feature is rarely used by anyone... Anyway, this is a nasty bug and needs to be fixed soon.

https://bookdown.org/yihui/bookdown/markdown-extensions-by-bookdown.html#theorems

[yihui]

For anyone reading this issue and has the same problem, please:

remotes::install_github('yihui/knitr')

[tchevri]

Sorry to trouble you - I tried your fix and it did not work for me (just to confirm and make sure I did not screw up anything, my knitr version is 1.28.9 after running your command) . As per your earlier request, I posted as a comment back on SO.

[cderv]

Thanks for the report @tchevri. I can reproduce. As it bugged me, I spend all my spare time from the last two days trying to understand what is wrong with the fix.

After some times, I started again from the beginning not considering the current fix and I think I found the issue.

Because Pandoc converts <div class='foo'> to \begin{foo}\end{foo} now.

@yihui I did a few checks, and I think this comes from our recent latex-div.lua filter. That would mean it happens with every pandoc version.

See this reprex: the environment is created from a Div, only if I activate our filter.

test_in <- tempfile()
test_out <- tempfile(fileext = ".tex")
xfun::write_utf8(
  '<div class="definition">something</div>',
  test_in
)
# using last pandoc
rmarkdown::pandoc_version()
#> [1] '2.9.2.1'
# by default, pandoc does not create an environment from a Div
rmarkdown::pandoc_convert(test_in, to = 'latex', from = "html", 
                          output = test_out, verbose = TRUE)
#> "C:/Users/chris/scoop/shims/pandoc" +RTS -K512m -RTS "C:\Users\chris\AppData\Local\Temp\RtmpG6BPN2\file5e8052eb5c38" --to latex --from html --output "C:\Users\chris\AppData\Local\Temp\RtmpG6BPN2\file5e8089c27a9.tex"
xfun::read_utf8(test_out)
#> [1] "something"
# with our filter, an environment is created from a div
rmarkdown::pandoc_convert(test_in, to = 'latex', from = "html", 
                          output = test_out, verbose = TRUE,
                          options = rmarkdown:::pandoc_lua_filters("latex-div.lua"))
#> "C:/Users/chris/scoop/shims/pandoc" +RTS -K512m -RTS "C:\Users\chris\AppData\Local\Temp\RtmpG6BPN2\file5e8052eb5c38" --to latex --from html --output "C:\Users\chris\AppData\Local\Temp\RtmpG6BPN2\file5e8089c27a9.tex" --lua-filter "C:/Users/chris/Documents/R/win-library/3.6/rmarkdown/rmd/lua/latex-div.lua"
xfun::read_utf8(test_out)
#> [1] "\\begin{definition}" ""                    "something"          
#> [4] ""                    "\\end{definition}"
unlink(c(test_in, test_out))

^{Created on 2020-06-21 by the reprex package (v0.3.0.9001)}

Reprex using pandoc 2.6

test_in <- tempfile()
test_out <- tempfile(fileext = ".tex")
xfun::write_utf8(
  '<div class="definition">something</div>',
  test_in
)
# using last pandoc
rmarkdown::find_pandoc(version = "2.6")
#> $version
#> [1] '2.6'
#> 
#> $dir
#> [1] "C:/Users/chris/scoop/shims"
rmarkdown::pandoc_version()
#> [1] '2.6'
# by default, pandoc does not create an environment from a Div
rmarkdown::pandoc_convert(test_in, to = 'latex', from = "html", 
                          output = test_out, verbose = TRUE)
#> "C:/Users/chris/scoop/shims/pandoc" +RTS -K512m -RTS "C:\Users\chris\AppData\Local\Temp\RtmpwzE4uh\file3abc7b856d08" --to latex --from html --output "C:\Users\chris\AppData\Local\Temp\RtmpwzE4uh\file3abc420185c.tex"
xfun::read_utf8(test_out)
#> [1] "something"
# with our filter, an environment is created from a div
rmarkdown::pandoc_convert(test_in, to = 'latex', from = "html", 
                          output = test_out, verbose = TRUE,
                          options = rmarkdown:::pandoc_lua_filters("latex-div.lua"))
#> "C:/Users/chris/scoop/shims/pandoc" +RTS -K512m -RTS "C:\Users\chris\AppData\Local\Temp\RtmpwzE4uh\file3abc7b856d08" --to latex --from html --output "C:\Users\chris\AppData\Local\Temp\RtmpwzE4uh\file3abc420185c.tex" --lua-filter "C:/Users/chris/Documents/R/win-library/3.6/rmarkdown/rmd/lua/latex-div.lua"
xfun::read_utf8(test_out)
#> [1] "\\begin{definition}" ""                    "something"          
#> [4] ""                    "\\end{definition}"
unlink(c(test_in, test_out))

^{Created on 2020-06-21 by the reprex package (v0.3.0.9001)}

Reading the filter, I can confirm that the filter add a latex environment for each div with a class.
https://github.com/rstudio/rmarkdown/blob/master/inst/rmd/lua/latex-div.lua

I think the fix should be either

• in our filter so that we do really nothing if no data-latex attribute is provided.
• or we should apply your fix for all pandoc version and consider that rmarkdown will convert Div to latex environment. (What pandoc does not want to do I think (reading at fenced span/div for latex output jgm/pandoc#5880 (comment))

What do you think ?

[yihui]

@cderv I'm still hesitating on if the data-latex attribute should be mandatory (rstudio/rmarkdown#1779). Now I feel it probably should, since I have been surprised a few times that <div> is also converted to LaTeX environments (I was only expecting ::: {} to be converted). Anyway, yihui/knitr@3f08505 should fix @tchevri's problem. Please try:

remotes::install_github('yihui/knitr')

Thanks!

[cderv]

I am still searching if there is a way to apply lua filer only on fenced div, and not explicit <div>.
It seems we may need to make the data-latex mandatory, or make the filter latex-div optional so that people can choose to opt-in, with the known consequences. Currently it applies to each rendering with any tools that uses rmarkdown and convert to PDF.

[yihui]

I just reverted rstudio/rmarkdown@14127ac with rstudio/rmarkdown@b2cff70. I'll update the R Markdown Cookbook soon.

I am still searching if there is a way to apply lua filer only on fenced div, and not explicit <div>.

I feel this may not be possible. Pandoc's parser treats them as identical:

 % echo '<div class="foo">abc</div>' | pandoc -f markdown -t native
[Div ("",["foo"],[])
 [Plain [Str "abc"]]]

% echo '::: foo\nabc\n:::' | pandoc -f markdown -t native
[Div ("",["foo"],[])
 [Para [Str "abc"]]]

[cderv]

Shouldn't we also revert the change in knitr that closed this issue ?

<div class="environment">

won't be converted to latex environment anymore.

[RLesur]

I feel this may not be possible. Pandoc's parser treats them as identical:

Pandoc can treat them differently if one disables the native_divs extension (this extension is enabled by default for markdown input).

[yihui]

@cderv It doesn't matter. I mean yihui/knitr@6907b42 doesn't hurt, although it is no longer necessary.

@RLesur Thanks for the tip!

[github-actions]

This old thread has been automatically locked. If you think you have found something related to this, please open a new issue by following the issue guide (https://yihui.org/issue/), and link to this old issue if necessary.