pandoc-style citations

pandoc-style citations

[Alex Fenton]

Hello,

I see that there were several extensive and fruitful discussions on this 
list last year on citation syntax. There seemed to be a reasonable 
degree of consensus that pandoc-style citation syntax was at the least a 
good model.

I'd like to know if there are any implementations out there of elisp to 
parse pandoc citation syntax and turn it into latex \cites. My question 
is not so much "when/if this will be in org mode" but rather whether 
there's something I can drop in now (likely as a link type).

I have a lot of longish citations with multiple references each with 
their own pre- and post- ("'blah blah blah @ref1, p.23, also @ref2, for 
a contrary view see @ref3 pp148-152") that end up as \cites. However my 
home-brew link solution, stuffing the multiple pre- and posts- with 
separator into the link description is unwieldy - difficult to write, 
hard to read and easy to get wrong or breaking output.

thanks
alex

Re: pandoc-style citations

[Richard Lawrence]

Hi Alex,

Alex Fenton <alex.fenton@pressure.to> writes:

> I see that there were several extensive and fruitful discussions on this 
> list last year on citation syntax. There seemed to be a reasonable 
> degree of consensus that pandoc-style citation syntax was at the least a 
> good model.
>
> I'd like to know if there are any implementations out there of elisp to 
> parse pandoc citation syntax and turn it into latex \cites. My question 
> is not so much "when/if this will be in org mode" but rather whether 
> there's something I can drop in now (likely as a link type).

As far as Elisp implementations go, I know of no specific parser for
Pandoc citation syntax.  But there is support for a Pandoc-like syntax
(discussed in the threads you read) in the wip-cite branch of Org's
repository.  This provides support for multi-cite citations in Org
syntax, but it isn't hooked up to the export framework at all.

Export is where efforts stalled last year.  Aaron Ecay, Vaidheeswaran C,
and I all worked on different proof-of-concept implementations to hook
up the citation parser to a citation processor and the various
exporters.  This is a non-trivial problem, and it seems that not many
people have a lot of time to work on it (including me), so if you want
to help, that would be great!  Aaron's work is in the Org repository
(see the wip-cite-awe branch).  Vaidheeswaran's is elsewhere; I don't
know where exactly, but you can search the list for a link.  My own is
here, in the wip-cite-rebase branch:

https://github.com/wyleyr/org-mode/.

There is also Pandoc itself, which can read (some) Org syntax.
Depending on what your document looks like, you might have good luck
just using Pandoc to convert it to LaTeX.

> I have a lot of longish citations with multiple references each with 
> their own pre- and post- ("'blah blah blah @ref1, p.23, also @ref2, for 
> a contrary view see @ref3 pp148-152") that end up as \cites. However my 
> home-brew link solution, stuffing the multiple pre- and posts- with 
> separator into the link description is unwieldy - difficult to write, 
> hard to read and easy to get wrong or breaking output.

You may also want to look at John Kitchin's org-ref, which I believe
works similar to your homebrew link solution, but has a lot of features
and may provide a better interface for what you're trying to do:

https://github.com/jkitchin/org-ref
 
Best,
Richard

Re: pandoc-style citations

[John Kitchin]

you might also checkout helm-bibtex. A recent note indicated "Support
for pre- and postnotes for pandoc-citeproc citations."

I haven't tried it, so I am not sure how it works for org-mode. org-ref
supports [[cite:somekey][pre-text::post-text]] syntax, without folding
the link on itself so you can see it.

I think your example would look like:

blah blah blah [[cite:ref1][::, p.23]], also cite:ref2, for a contrary
view see [[cite:ref3][:: pp148-152]]

this exports out of the org-ref box to:
blah blah blah \cite[][, p.23]{ref1}, also \cite{ref2}, for a contrary
view see \cite[][ pp148-152]{ref3}.

I don't use this style of citations, so I don't know if it is right.

Alex Fenton writes:

> Hello,
>
> I see that there were several extensive and fruitful discussions on this
> list last year on citation syntax. There seemed to be a reasonable
> degree of consensus that pandoc-style citation syntax was at the least a
> good model.
>
> I'd like to know if there are any implementations out there of elisp to
> parse pandoc citation syntax and turn it into latex \cites. My question
> is not so much "when/if this will be in org mode" but rather whether
> there's something I can drop in now (likely as a link type).
>
> I have a lot of longish citations with multiple references each with
> their own pre- and post- ("'blah blah blah @ref1, p.23, also @ref2, for
> a contrary view see @ref3 pp148-152") that end up as \cites. However my
> home-brew link solution, stuffing the multiple pre- and posts- with
> separator into the link description is unwieldy - difficult to write,
> hard to read and easy to get wrong or breaking output.
>
> thanks
> alex


--
Professor John Kitchin
Doherty Hall A207F
Department of Chemical Engineering
Carnegie Mellon University
Pittsburgh, PA 15213
412-268-7803
@johnkitchin
http://kitchingroup.cheme.cmu.edu

Re: pandoc-style citations

[Alex Fenton]

Hi Richard

On 27/04/16 16:09, Richard Lawrence wrote:
> As far as Elisp implementations go, I know of no specific parser for
> Pandoc citation syntax.  But there is support for a Pandoc-like syntax
> (discussed in the threads you read) in the wip-cite branch of Org's
> repository.

> https://github.com/wyleyr/org-mode/

Thanks for your reply, and for all your work on specifying and 
developing this. It looks like that for now I can adapt some of the 
parsing in org-element.el (org-element-citation-reference-parser and 
org-element-citation-parser etc) to get something that works for now in 
standard mainline org.


> Export is where efforts stalled last year.

That's understandable, given that, as you say, it's a complex problem 
given the range of citation styles and output formats. It's still a 
shame given the work that you (pl.) have put into integrating citations 
into the org parser & element tree so they are first class objects.

I don't know whether it's conceivable that the data structures and 
parsing could be integrated into org, with the (presumably) relatively 
easy latex output, which I suspect is the commonest use case, and then 
with some kind of "adequate" output for other targets (html, text, odt) 
- perhaps an output that would require further post-processing by a 
third-party tool such as citeproc or pandoc.

Latex-outputters would be better off and other targets no worse off than 
present, and it might act as a spur to solve the other target formats 
one by one way. But I can see that this has been to some extent 
considered and can also see the arguments against.

>
> You may also want to look at John Kitchin's org-ref, which I believe
> works similar to your homebrew link solution, but has a lot of features
> and may provide a better interface for what you're trying to do:

Org-ref does a lot of nice things, but on this particular point is not 
very helpful, and has nothing to add for the case of multiple cites with 
overall and individual pre/posts. The package states that this is low 
priority b/c this style of citation is rare in scientific publishing (by 
which I guess is meant natural sciences; footnotes with multiple 
citations and linking text are of common in humanities and some social 
sciences).

thanks again and best wishes
alex

Re: pandoc-style citations

[Alex Fenton]

On 27/04/16 21:41, John Kitchin wrote:
> you might also checkout helm-bibtex. A recent note indicated "Support
> for pre- and postnotes for pandoc-citeproc citations."

Thanks, I already use helm-bibtex together with org-ref so will have a look.
> this exports out of the org-ref box to:
> blah blah blah \cite[][, p.23]{ref1}, also \cite{ref2}, for a contrary
> view see \cite[][ pp148-152]{ref3}.
>
> I don't use this style of citations, so I don't know if it is right.

Not quite unfortunately, this would mean that in the two styles I use, 
Harvard (common in social science) referencing, the references appear in 
separate parentheses, or footnote (common in e.g. history) in separate 
footnotes, when they should be together in a single one. I appreciate 
that the style in medicine and many natural sciences is often just an 
author/year abbreviation or number, which works much better plopped into 
normal running text.

best
alex

Re: pandoc-style citations

[John Kitchin]

>
> Org-ref does a lot of nice things, but on this particular point is not
> very helpful, and has nothing to add for the case of multiple cites with
> overall and individual pre/posts. The package states that this is low
> priority b/c this style of citation is rare in scientific publishing (by
> which I guess is meant natural sciences; footnotes with multiple
> citations and linking text are of common in humanities and some social
> sciences).

Fair enough. I don't think the standard link syntax is rich enough for this.

I worked out the basis for an alternative approach here:
http://kitchingroup.cheme.cmu.edu/blog/2015/02/05/Extending-the-org-mode-link-syntax-with-attributes/
that might also be adaptable for your use case. It basically gets
towards the cite syntax but doesn't require adding new elements and
other things.


I updated the org-ref docs on why we don't currently do more than the
simple pre/post text. I don't have anything against supporting more, I
just don't know how to do it, or when it would be right ;)

>
> thanks again and best wishes
> alex


--
Professor John Kitchin
Doherty Hall A207F
Department of Chemical Engineering
Carnegie Mellon University
Pittsburgh, PA 15213
412-268-7803
@johnkitchin
http://kitchingroup.cheme.cmu.edu

Re: pandoc-style citations

[Richard Lawrence]

Hi Alex,

Alex Fenton <alex.fenton@pressure.to> writes:

>> Export is where efforts stalled last year.
>
> That's understandable, given that, as you say, it's a complex problem 
> given the range of citation styles and output formats. It's still a 
> shame given the work that you (pl.) have put into integrating citations 
> into the org parser & element tree so they are first class objects.
>
> I don't know whether it's conceivable that the data structures and 
> parsing could be integrated into org, with the (presumably) relatively 
> easy latex output, which I suspect is the commonest use case, and then 
> with some kind of "adequate" output for other targets (html, text, odt) 
> - perhaps an output that would require further post-processing by a 
> third-party tool such as citeproc or pandoc.
>
> Latex-outputters would be better off and other targets no worse off than 
> present, and it might act as a spur to solve the other target formats 
> one by one way. But I can see that this has been to some extent 
> considered and can also see the arguments against.

Unfortunately, even the LaTeX case is not so easy unless you assume
users are managing the bibliography database, and the mapping of
citation keys into that database, themselves.  It's easy to map the Org
syntax to the LaTeX syntax if you can just pass the keys through, but
that isn't a great assumption to make.  And things are a lot more
complicated once you have to worry about looking up the keys somewhere,
generating an appropriate .bib file, etc. We will have to do this stuff
for other backends anyway, so it seems sensible to do it for LaTeX too,
but that's what makes the project so big (especially if we can't just
pick one database format to support).

Maybe you're right that a good first step would just be to generate
LaTeX syntax, and just let users manage the bibliography database in
whatever way works for them, without trying to interface with it.  This
could be done quickly -- indeed, I think it is already most of the way
done.  And as you say, that would go a long way toward meeting many
people's needs.

I hope I will have some time next week to look at this.  I'll report
back!

Best,
Richard