Filter Divs along with its notes.

[urosmikanovic]

Hi,

I keep markdown sources with divs with attributes that I want to selectively filter out depending on the target. Example:

družbeni in psihološki razsežnosti. A prava vprašanja se po Beechu
tičejo produkcijskih razmerij v umetnikovem ateljeju, razmerja med
umetnico in njenimi produkcijskimi sredstvi, produktivnosti oziroma
neproduktivnosti umetnikovega dela -- tu očitna ekonomska izjemnost
umetnosti ostaja neteoretizirana. [@beech2015art, 239-240]

::: comment
Pantič razširi vprašanje: "Ne le odnos umetnine do kapitala, temveč
odnos umetnostne produkcije do celotne družbene presežne vrednosti."
[@pantic2019umetnost, 182]
:::

Strožja ločitev ekonomske in družbene problematike, ki jo zahteva
Beech, razpravo o statusu umetnosti v razmerju do kapitalističnega
produkcijskega načina zaostri v okvire razprave o tranziciji
iz fevdalizma v kapitalizem. Glede tega je jasen v uvodu, kjer
kapitalistični produkcijski način opredeli kot sistem, ki je glavno
določen s produkcijskim razmerjem med razredom razlastninjenih
delavcev in razredom brezdelnih lastnikov. Kot takega ga ne določata
ne produkcija za trg oziroma blagovna produkcija ne določena stopnja
razvoja produktivnih sil. Vprašanje razmerja med umetnostjo in
kapitalističnim produkcijskim načinom je po Beechu vprašanje, ali
je umetnost prešla tranzicijo iz fevdalizma v kapitalizem oziroma ali
umetnost uteleša produkcijska razmerja med mezdnim delom in kapitalom.
[@beech2015art, 6-9]

For some targets I want to filter out all Divs with comment attribute. I guess with a filter like this:

SCRIPT_NAME = "strip_comments.lua"
os.setlocale("C")

return {
	{ Div = function (div)
		if div.classes:includes("comment") then
			return {}
		end
	end }
}

The problem is that citeproc somehow takes the citations inside the removed Divs into account and doesn't style the citations correctly (chicago fullnote with ibid for example). The processed citations should be styled something like this:

[1]: Dave Beech, *Art and Value* (Place: Publisher, 2015), 239-240.
[2]: Ibid, 6-9.

Instead the second note is styled as a short note, as citeproc seems to think they are not consecutive references to the same work. I tried setting the citeproc filter in the filter list after strip_comments.lua and tried setting the traversal order to topdown but I can't seem to achieve the desired note forms. Any suggestions how to strip Divs including their notes and citations? Thanks!

[fiapps]

I think your problem is that your filter runs after citeproc instead of before it. The order of the arguments on the command line determines the order of processing, so you need to give the filter filename with -L before --citeproc. That way, by the time citeproc sees the AST, the .comment divs will no longer be part of it.

[jgm]

Try again. If the Lua filter is first, the comment div will not be present in the AST when the citeproc filter is called. I just verified this experimentally.

[urosmikanovic]

Try again. If the Lua filter is first, the comment div will not be present in the AST when the citeproc filter is called. I just verified this experimentally.

Sadly, I can't reproduce. For clarity, I'm trying to verify by running this command:

pandoc -L strip_comments.lua --citeproc --csl chicago-fullnote-bibliography-with-ibid.csl -o output.md input.md

input.md:

Should be a full note. [@beech2015art, 239-240]

::: comment
Should be skipped. [@pantic2019umetnost, 182]
:::

Should be ibid. [@beech2015art, 6-9]

Should be ibid. [@beech2015art, 10-12]

---
references:
- type: book
  id: beech2015art
  author:
  - family: Beech
    given: Dave
  title: "Art and value: art's economic exceptionalism in classical,
    neoclassical and Marxist economics"
  title-short: "Art and value"
  publisher-place: Leiden
  publisher: Brill
  issued: 2015
  language: en
- type: book
  id: pantic2019umetnost
  author:
  - family: Pantić
    given: Rade
  title: "Umetnost skozi teorijo: historičnomaterialistične analize"
  title-short: "Umetnost skozi teorijo"
  translator:
  - family: Baskar
    given: Nil
  publisher-place: Ljubljana
  publisher: /*cf.
  issued: 2019
  language: sl
...

output.md

Should be a full note.[^1]

Should be ibid.[^2]

Should be ibid.[^3]

:::: {#refs .references .csl-bib-body .hanging-indent entry-spacing="0"}
::: {#ref-beech2015art .csl-entry}
Beech, Dave. *Art and Value: Art's Economic Exceptionalism in Classical,
Neoclassical and Marxist Economics*. Leiden: Brill, 2015.
:::
::::

[^1]: Dave Beech, *Art and Value: Art's Economic Exceptionalism in
    Classical, Neoclassical and Marxist Economics* (Leiden: Brill,
    2015), 239--40.

[^2]: Beech, *Art and Value*, 6--9.

[^3]: Ibid., 10--12.

I would expect the second note to be Ibid., 6--9. When I don't reference anything in the comment, it is styled correctly as Ibid., 6--9.

Notice, however, that the reference in the stripped comment is correctly not listed in the bibliography. The div is not present in the AST, but whatever is keeping track of subsequence takes count of it.

[urosmikanovic]

Attached json output. The div is not there, but the citationNoteNums count 1, 3, 4.

output_pretty.json

[jgm]

Okay, that's true, omitted a Div won't change other aspects of the AST, like citationNoteNum.
You could have your filter adjust these too, I suppose.

[urosmikanovic]

Thanks. So I would need to walk the divs? Is there no way to skip processing some references? When does citeproc even see the reference if it's being called after the filter?

[jgm]

It doesn't see the reference. When parsing Mardown, pandoc creates Citation elements, and these contain a citationNoteNum field that records the number of the note in which they appear. Citeproc will then process these elements. If you remove some of these Citation elements, that doesn't change the citationNoteNum fields that were assigned to the others.

[urosmikanovic]

Thank you for walking me through this. Amending the filter as below gives the expected result. Let me know if there is an immediately more obvious, more idiomatic way.

SCRIPT_NAME = "strip_comments.lua"
os.setlocale("C")

local note_num = 1

return {
	{ Div = function (div)
		if div.classes:includes("comment") then return {} end
	end },
	{ Cite = function (cite)
		for i,c in ipairs(cite.citations) do
			c.note_num = note_num
		end
		note_num = note_num + 1
		return cite
	end },
}

[jgm]

This should work as long as all of your footnotes are from ordinary citations (and you don't have manual notes with citations inside them).

You could add a clause for Note that incremented the note number for non-citations.