library(tinkr)
The goal of tinkr is to convert (R)Markdown files to XML and back to allow their editing with xml2 (XPath!) instead of numerous complicated regular expressions. If these words mean nothing to you, see our list of resources to get started.
Possible applications are R scripts using tinkr, and XPath via xml2 to:
roweb2_headers.R
script and pull request #279 to roweb2;Only the body of the (R) Markdown file is cast to XML, using the Commonmark specification via the commonmark
package. YAML metadata could be edited using the yaml
package, which is not the goal of this package.
We have created an R6 class object called yarn to store the representation of both the YAML and the XML data, both of which are accessible through the $body
and $yaml
elements. In addition, the namespace prefix is set to “md” in the $ns
element.
You can perform XPath queries using the $body
and $ns
elements:
library("tinkr")
library("xml2")
<- system.file("extdata", "example1.md", package = "tinkr")
path head(readLines(path))
#> [1] "---"
#> [2] "title: \"What have these birds been studied for? Querying science outputs with R\""
#> [3] "slug: birds-science"
#> [4] "authors:"
#> [5] " - name: Maëlle Salmon"
#> [6] " url: https://masalmon.eu/"
<- tinkr::yarn$new(path)
ex1 # find all ropensci.org blog links
xml_find_all(
x = ex1$body,
xpath = ".//md:link[contains(@destination,'ropensci.org/blog')]",
ns = ex1$ns
)#> {xml_nodeset (7)}
#> [1] <link destination="https://ropensci.org/blog/2018/08/21/birds-radolfzell/ ...
#> [2] <link destination="https://ropensci.org/blog/2018/09/04/birds-taxo-traits ...
#> [3] <link destination="https://ropensci.org/blog/2018/08/21/birds-radolfzell/ ...
#> [4] <link destination="https://ropensci.org/blog/2018/08/14/where-to-bird/" t ...
#> [5] <link destination="https://ropensci.org/blog/2018/08/21/birds-radolfzell/ ...
#> [6] <link destination="https://ropensci.org/blog/2018/08/28/birds-ocr/" title ...
#> [7] <link destination="https://ropensci.org/blog/2018/09/04/birds-taxo-traits ...
This is a basic example. We read “example1.md”, change all headers 3 to headers 1, and save it back to md. Because the xml2 objects are passed by reference, manipulating them does not require reassignment.
library("magrittr")
library("tinkr")
# From Markdown to XML
<- system.file("extdata", "example1.md", package = "tinkr")
path # Level 3 header example:
cat(tail(readLines(path, 40)), sep = "\n")
#> ### Getting a list of 50 species from occurrence data
#>
#> For more details about the following code, refer to the [previous post
#> of the series](https://ropensci.org/blog/2018/08/21/birds-radolfzell/).
#> The single difference is our adding a step to keep only data for the
#> most recent years.
<- tinkr::yarn$new(path)
ex1 # transform level 3 headers into level 1 headers
$body %>%
ex1::xml_find_all(xpath = ".//md:heading[@level='3']", ex1$ns) %>%
xml2::xml_set_attr("level", 1)
xml2
# Back to Markdown
<- tempfile(fileext = "md")
tmp $write(tmp)
ex1# Level three headers are now Level one:
cat(tail(readLines(tmp, 40)), sep = "\n")
#> # Getting a list of 50 species from occurrence data
#>
#> For more details about the following code, refer to the [previous post
#> of the series](https://ropensci.org/blog/2018/08/21/birds-radolfzell/).
#> The single difference is our adding a step to keep only data for the
#> most recent years.
unlink(tmp)
For R Markdown files, to ease editing of chunk label and options, to_xml
munges the chunk info into different attributes. E.g. below you see that code_blocks
can have a language
, name
, echo
attributes.
<- system.file("extdata", "example2.Rmd", package = "tinkr")
path <- tinkr::yarn$new(path)
rmd $body
rmd#> {xml_document}
#> <document xmlns="http://commonmark.org/xml/1.0">
#> [1] <code_block xml:space="preserve" language="r" name="setup" include="FALS ...
#> [2] <heading level="2">\n <text xml:space="preserve">R Markdown</text>\n</h ...
#> [3] <paragraph>\n <text xml:space="preserve">This is an </text>\n <striket ...
#> [4] <paragraph>\n <text xml:space="preserve">When you click the </text>\n ...
#> [5] <code_block xml:space="preserve" language="r" name="" eval="TRUE" echo=" ...
#> [6] <heading level="2">\n <text xml:space="preserve">Including Plots</text> ...
#> [7] <paragraph>\n <text xml:space="preserve">You can also embed plots, for ...
#> [8] <code_block xml:space="preserve" language="python" name="" fig.cap="&quo ...
#> [9] <code_block xml:space="preserve" language="python" name="">plot(pressure ...
#> [10] <paragraph>\n <text xml:space="preserve">Non-RMarkdown blocks are also ...
#> [11] <code_block info="bash" xml:space="preserve" name="">echo "this is an un ...
#> [12] <code_block xml:space="preserve" name="">This is an ambiguous code block ...
#> [13] <paragraph>\n <text xml:space="preserve">Note that the </text>\n <code ...
#> [14] <table>\n <table_header>\n <table_cell align="left">\n <text xm ...
#> [15] <paragraph>\n <text xml:space="preserve">blabla</text>\n</paragraph>
Note that all of the features in tinkr work for both Markdown and R Markdown.
Inserting new nodes into the AST is surprisingly difficult if there is a default namespace, so we have provided a method in the yarn object that will take plain Markdown and translate it to XML nodes and insert them into the document for you. For example, you can add a new code block:
<- system.file("extdata", "example2.Rmd", package = "tinkr")
path <- tinkr::yarn$new(path)
rmd ::xml_find_first(rmd$body, ".//md:code_block", rmd$ns)
xml2#> {xml_node}
#> <code_block space="preserve" language="r" name="setup" include="FALSE" eval="TRUE">
<- c(
new_code "```{r xml-block, message = TRUE}",
"message(\"this is a new chunk from {tinkr}\")",
"```")
<- data.frame(
new_table package = c("xml2", "xslt", "commonmark", "tinkr"),
cool = TRUE
)# Add chunk into document after the first chunk
$add_md(new_code, where = 1L)
rmd# Add a table after the second chunk:
$add_md(knitr::kable(new_table), where = 2L)
rmd# show the first 21 lines of modified document
$head(21)
rmd#> ---
#> title: "Untitled"
#> author: "M. Salmon"
#> date: "September 6, 2018"
#> output: html_document
#> ---
#>
#> ```{r setup, include=FALSE, eval=TRUE}
#> knitr::opts_chunk$set(echo = TRUE)
#> ```
#>
#> ```{r xml-block, message=TRUE}
#> message("this is a new chunk from {tinkr}")
#> ```
#>
#> | package | cool |
#> | :------------------------- | :------------------ |
#> | xml2 | TRUE |
#> | xslt | TRUE |
#> | commonmark | TRUE |
#> | tinkr | TRUE |
If you are not closely following one of the examples provided, what background knowledge do you need before using tinkr?
The (R)md to XML to (R)md loop on which tinkr
is based is slightly lossy because of Markdown syntax redundancy, so the loop from (R)md to R(md) via to_xml
and to_md
will be a bit lossy. For instance
lists can be created with either “+”, “-” or "*“. When using tinkr
, the (R)md after editing will only use”-" for lists.
Links built like [word][smallref]
with a bottom anchor [smallref]: URL
will have the anchor moved to the bottom of the document.
Characters are escaped (e.g. “[” when not for a link).
GitHub tickboxes are preserved (only for yarn
objects)
Block quotes lines all get “>” whereas in the input only the first could have a “>” at the beginning of the first line.
For tables see the next subsection.
Such losses make your (R)md different, and the git diff a bit harder to parse, but should not change the documents your (R)md is rendered to. If it does, report a bug in the issue tracker!
A solution to not loose your Markdown style, e.g. your preferring “*” over “-” for lists is to tweak our XSL stylesheet and provide its filepath as stylesheet_path
argument to to_md
.
to_xml
+ to_md
. If you notice something amiss, e.g. too much space compared to what you were expecting, please open an issue.While Markdown parsers like pandoc know what LaTeX is, commonmark does not, and that means LaTeX equations will end up with extra markup due to commonmark’s desire to escape characters.
However, if you have LaTeX equations that use either $
or $$
to delimit them, you can protect them from formatting changes with the $protect_math()
method (for users of the yarn
object) or the protect_math()
function (for those using the output of to_xml()
). Below is a demonstration using the yarn
object:
<- system.file("extdata", "math-example.md", package = "tinkr")
path <- tinkr::yarn$new(path)
math $tail() # malformed
math#>
#> $$
#> Q\_{N(norm)}=\\frac{C\_N +C\_{N-1}}2\\times
#> \\frac{\\sum *{i=N-n}^{N}Q\_i} {\\sum*{j=N-n}^{N}{(\\frac{C\_j+C\_{j-1}}2)}}
#> $$
$protect_math()$tail() # success!
math#>
#> $$
#> Q_{N(norm)}=\frac{C_N +C_{N-1}}2\times
#> \frac{\sum _{i=N-n}^{N}Q_i} {\sum_{j=N-n}^{N}{(\frac{C_j+C_{j-1}}2)}}
#> $$
Note, however, that there are a few caveats for this:
The dollar notation for inline math must be adjacent to the text. E.G. $\alpha$
is valid, but $ \alpha$
and $\alpha $
are not valid.
We do not currently have support for bracket notation
If you use a postfix dollar sign in your prose (e.g. BASIC commands or a Burroughs-Wheeler Transformation demonstration), you must be sure to either use punctuation after the trailing dollar sign OR format the text as code. (i.e. `INKEY$`
is good, but INKEY$
by itself is not good and will be interpreted as LaTeX code, throwing an error:
<- system.file("extdata", "basic-math.md", package = "tinkr")
path <- tinkr::yarn$new(path)
math $head(15) # malformed
math#> ---
#> title: basic math
#> ---
#>
#> BASIC programming can make things weird:
#>
#> - Give you $2 to tell me what INKEY$ means.
#> - Give you $2 to *show* me what INKEY$ means.
#> - Give you $2 to *show* me what `INKEY$` means.
#>
#> Postfix dollars mixed with prefixed dollars can make things weird:
#>
#> - We write $2 but say 2$ verbally.
#> - We write $2 but *say* 2$ verbally.
$protect_math() #error
math#> Error: Inline math delimiters are not balanced.
#>
#> HINT: If you are writing BASIC code, make sure you wrap variable
#> names and code in backtics like so: `INKEY$`.
#>
#> Below are the pairs that were found:
#> start...end
#> -----...---
#> Give you $2 to ... me what INKEY$ means.
#> Give you $2 to ... 2$ verbally.
#> We write $2 but ...