[wip-cite-new] Merging tomorrow?

[wip-cite-new] Merging tomorrow?

[Nicolas Goaziou]

Hello,

I think the "wip-cite-new" branch is in good shape now. As
a consequence, I'd like to merge it tomorrow.

It is documented, but the documentation is scattered across the various
"oc" libraries, and some threads in the mailing list. I'll do a summary
here, from a user point of view.

--8<---------------cut here---------------start------------->8---
Basically, in order to use it, you need to first set-up a bibliography,
using one or more "bibliography" keywords. <C-c '> on such a keyword
visits the related file. Out of the box, Org supports JSON-CSL and
BibTeX (or biblatex) bibliographies.

Then, citations can be inserted with the following syntax:

  [cite/style:common prefix ;prefix @key suffix; ... ; common suffix]

Spaces are meaningful except those after the initial colon and before
the closing bracket.

Every part of the syntax is optional, except the brackets, "cite" and
the colon. Also the citation must contain at least a key. So its minimal
form is:

  [cite:@key]

The "style" part is detailed below, in the part related to export.

Org can insert or edit citations with <C-c C-x @> (and delete them with
<C-u C-c C-x @>), follow them with <C-c C-o>, fontify them, and export
them. These four actions (insert, follow, activate, and export) are
called capabilities.  Libraries responsible for these capabilities are
called citation processors.

You can select one citation processor for each capability, independently
on the others, through the following variables:

- org-cite-activate-processor
- org-cite-export-processors
- org-cite-follow-processor
- org-cite-insert-processor

Out of the box, Org provides the "basic" (in "oc-basic.el") processor
for all of these tasks. It also boasts processors dedicated for export:
"csl", "natbib" and "biblatex".

During export, output for citations is controlled by their style, which
is an Org label that the export processor may recognize and associate to
a specific display, or fall-back to a default style (called "nil"). For
example, most processors support "noauthor" and "text" styles. 

Some styles can accept a variant, with the syntax "style/variant".
Again, it's up to the processor to associate it to a specific display.
Common variants include "bare", "caps" or "full".  They also accept
short-hands, like "b", "c" and "f".  Please refer to the export
processors' libraries ("oc-basic.el", "oc-csl.el", …) for more information.

It is possible to define a default style for a whole document (with
"cite_export"), or for all documents (with `org-cite-export-processors').

References are displayed with the "print_bibliography" keyword. It is
possible to add parameters to its value, as some export processors could
make use of them.
--8<---------------cut here---------------end--------------->8---

Please let me know if there are any objections to the merge.

Regards,
-- 
Nicolas Goaziou

Re: [wip-cite-new] Merging tomorrow?

[William Denton]

On 8 July 2021, Nicolas Goaziou wrote:

> Please let me know if there are any objections to the merge.

I do not object---I am eager to try it!  I haven't experimented with the work as 
it was being done, but I was very impressed with and grateful for all the work 
that everyone did on this.  I'm a librarian, so I know how ridiculous and 
bizarre citation formats are, but I still learned a lot about how complex it is 
to write code to handle them.

My thanks to everyone involved for doing such great work to add a wonderful new 
feature to Org.

Bill

--
William Denton
https://www.miskatonic.org/
Librarian, artist and licensed private investigator.

Re: [wip-cite-new] Merging tomorrow?

[Thomas S. Dye]

Aloha Nicolas,

Good news! I'm looking forward to using this facility.

Thanks to all the contributors.

All the best,
Tom

Nicolas Goaziou <mail@nicolasgoaziou.fr> writes:

> Hello,
>
> I think the "wip-cite-new" branch is in good shape now. As
> a consequence, I'd like to merge it tomorrow.
>
> It is documented, but the documentation is scattered across the 
> various
> "oc" libraries, and some threads in the mailing list. I'll do a 
> summary
> here, from a user point of view.
>
> --8<---------------cut 
> here---------------start------------->8---
> Basically, in order to use it, you need to first set-up a 
> bibliography,
> using one or more "bibliography" keywords. <C-c '> on such a 
> keyword
> visits the related file. Out of the box, Org supports JSON-CSL 
> and
> BibTeX (or biblatex) bibliographies.
>
> Then, citations can be inserted with the following syntax:
>
>   [cite/style:common prefix ;prefix @key suffix; ... ; common 
>   suffix]
>
> Spaces are meaningful except those after the initial colon and 
> before
> the closing bracket.
>
> Every part of the syntax is optional, except the brackets, 
> "cite" and
> the colon. Also the citation must contain at least a key. So its 
> minimal
> form is:
>
>   [cite:@key]
>
> The "style" part is detailed below, in the part related to 
> export.
>
> Org can insert or edit citations with <C-c C-x @> (and delete 
> them with
> <C-u C-c C-x @>), follow them with <C-c C-o>, fontify them, and 
> export
> them. These four actions (insert, follow, activate, and export) 
> are
> called capabilities.  Libraries responsible for these 
> capabilities are
> called citation processors.
>
> You can select one citation processor for each capability, 
> independently
> on the others, through the following variables:
>
> - org-cite-activate-processor
> - org-cite-export-processors
> - org-cite-follow-processor
> - org-cite-insert-processor
>
> Out of the box, Org provides the "basic" (in "oc-basic.el") 
> processor
> for all of these tasks. It also boasts processors dedicated for 
> export:
> "csl", "natbib" and "biblatex".
>
> During export, output for citations is controlled by their 
> style, which
> is an Org label that the export processor may recognize and 
> associate to
> a specific display, or fall-back to a default style (called 
> "nil"). For
> example, most processors support "noauthor" and "text" styles.
>
> Some styles can accept a variant, with the syntax 
> "style/variant".
> Again, it's up to the processor to associate it to a specific 
> display.
> Common variants include "bare", "caps" or "full".  They also 
> accept
> short-hands, like "b", "c" and "f".  Please refer to the export
> processors' libraries ("oc-basic.el", "oc-csl.el", …) for more 
> information.
>
> It is possible to define a default style for a whole document 
> (with
> "cite_export"), or for all documents (with 
> `org-cite-export-processors').
>
> References are displayed with the "print_bibliography" keyword. 
> It is
> possible to add parameters to its value, as some export 
> processors could
> make use of them.
> --8<---------------cut 
> here---------------end--------------->8---
>
> Please let me know if there are any objections to the merge.
>
> Regards,


--
Thomas S. Dye
https://tsdye.online/tsdye

Re: [wip-cite-new] Merging tomorrow?

[Matt Price]

[-- Attachment #1: Type: text/plain, Size: 4104 bytes --]

I cannot believe this is finally happening, and I am so stoked and excited
about it. I've been using ~wip-cite-new~ in my classes this week, and these
new tools are absolutely transformative.

Thank you so much for the immense amount of work you put into this.  And
also to Bruce for championing it, and Andras and Denis and others for their
contributions. Really, I feel like there should be a parade.

Matt

On Wed, Jul 7, 2021 at 10:23 PM Thomas S. Dye <tsd@tsdye.online> wrote:

> Aloha Nicolas,
>
> Good news! I'm looking forward to using this facility.
>
> Thanks to all the contributors.
>
> All the best,
> Tom
>
> Nicolas Goaziou <mail@nicolasgoaziou.fr> writes:
>
> > Hello,
> >
> > I think the "wip-cite-new" branch is in good shape now. As
> > a consequence, I'd like to merge it tomorrow.
> >
> > It is documented, but the documentation is scattered across the
> > various
> > "oc" libraries, and some threads in the mailing list. I'll do a
> > summary
> > here, from a user point of view.
> >
> > --8<---------------cut
> > here---------------start------------->8---
> > Basically, in order to use it, you need to first set-up a
> > bibliography,
> > using one or more "bibliography" keywords. <C-c '> on such a
> > keyword
> > visits the related file. Out of the box, Org supports JSON-CSL
> > and
> > BibTeX (or biblatex) bibliographies.
> >
> > Then, citations can be inserted with the following syntax:
> >
> >   [cite/style:common prefix ;prefix @key suffix; ... ; common
> >   suffix]
> >
> > Spaces are meaningful except those after the initial colon and
> > before
> > the closing bracket.
> >
> > Every part of the syntax is optional, except the brackets,
> > "cite" and
> > the colon. Also the citation must contain at least a key. So its
> > minimal
> > form is:
> >
> >   [cite:@key]
> >
> > The "style" part is detailed below, in the part related to
> > export.
> >
> > Org can insert or edit citations with <C-c C-x @> (and delete
> > them with
> > <C-u C-c C-x @>), follow them with <C-c C-o>, fontify them, and
> > export
> > them. These four actions (insert, follow, activate, and export)
> > are
> > called capabilities.  Libraries responsible for these
> > capabilities are
> > called citation processors.
> >
> > You can select one citation processor for each capability,
> > independently
> > on the others, through the following variables:
> >
> > - org-cite-activate-processor
> > - org-cite-export-processors
> > - org-cite-follow-processor
> > - org-cite-insert-processor
> >
> > Out of the box, Org provides the "basic" (in "oc-basic.el")
> > processor
> > for all of these tasks. It also boasts processors dedicated for
> > export:
> > "csl", "natbib" and "biblatex".
> >
> > During export, output for citations is controlled by their
> > style, which
> > is an Org label that the export processor may recognize and
> > associate to
> > a specific display, or fall-back to a default style (called
> > "nil"). For
> > example, most processors support "noauthor" and "text" styles.
> >
> > Some styles can accept a variant, with the syntax
> > "style/variant".
> > Again, it's up to the processor to associate it to a specific
> > display.
> > Common variants include "bare", "caps" or "full".  They also
> > accept
> > short-hands, like "b", "c" and "f".  Please refer to the export
> > processors' libraries ("oc-basic.el", "oc-csl.el", …) for more
> > information.
> >
> > It is possible to define a default style for a whole document
> > (with
> > "cite_export"), or for all documents (with
> > `org-cite-export-processors').
> >
> > References are displayed with the "print_bibliography" keyword.
> > It is
> > possible to add parameters to its value, as some export
> > processors could
> > make use of them.
> > --8<---------------cut
> > here---------------end--------------->8---
> >
> > Please let me know if there are any objections to the merge.
> >
> > Regards,
>
>
> --
> Thomas S. Dye
> https://tsdye.online/tsdye
>
>

[-- Attachment #2: Type: text/html, Size: 5430 bytes --]

Re: [wip-cite-new] Merging tomorrow?

[Timothy]

Nicolas Goaziou <mail@nicolasgoaziou.fr> writes:

> I think the "wip-cite-new" branch is in good shape now. As
> a consequence, I'd like to merge it tomorrow.

This may be much too late to raise this (sorry), but I've got a query.

At the moment org-ref allows for:
+ citing from a bibliography
+ referencing elements within the document

wip-cite-new deals with citing from bibliographies, but I don't think it
deals with within-document referencing --- should it?

In case it helps, here's a small example of referencing elements with
org-ref:

#+begin_src org
,#+caption: Wow, look a me. label:some-f
[[file:some-file.png]]

Have you seen cref:some-f ?
#+end_src

Exported LaTeX:

#+begin_src LaTeX
\begin{figure}[htbp]
\centering
\includegraphics[width=.9\linewidth]{some-file.png}
\caption{Wow, look a me. \label{some-f}}
\end{figure}

Have you seen \cref{some-f} ?
#+end_src

I know that we can get a label added with #+name, but I don't know that
there's an easy way to reference it without org-ref. I feel like ideally
this should be something Org provides.

-- 
Timothy

Re: [wip-cite-new] Merging tomorrow?

[Bruce D'Arcus]

On Wed, Jul 7, 2021 at 11:48 PM Timothy <tecosaur@gmail.com> wrote:

> wip-cite-new deals with citing from bibliographies, but I don't think it
> deals with within-document referencing --- should it?

It doesn't now.

I guess to break down the second question further:

1. Should it?
2. Can it? Could the design be extended to include internal referencing?
2. If yes to both, should that hold back merger now?

I've not thought about this a lot, but my tentative view ...

1. Maybe.
2. I think so. You'd just need a way to include internal targets in
addition to the citation-references (keys); for illustration,
something like [cite:#some-if].
3. No.

I say no in part because while it's possible it's fairly
straightforward to add this, it will take some thought, and there's
probably details to sort out.

And the code is ready, I think, based on the requirements that have
been the focus the past year+.

OTOH, perhaps this basic requirements question was raised before I
joined the discussion, and it was already rejected?

Either way, I don't think this should hold back merger now. It can be
added later if it makes sense.

Bruce

Re: [wip-cite-new] Merging tomorrow?

[Joost Kremers]

On Thu, Jul 08 2021, Bruce D'Arcus wrote:
> On Wed, Jul 7, 2021 at 11:48 PM Timothy <tecosaur@gmail.com> wrote:
>
>> wip-cite-new deals with citing from bibliographies, but I don't think it
>> deals with within-document referencing --- should it?
>
> It doesn't now.
>
> I guess to break down the second question further:
>
> 1. Should it?

One thing to keep in mind here: these are two different sets of functionality
and a tool designed to handle one isn't necessarily right for handling the
other.

Org-cite provides four capabilities: insert, follow, activate and export. And
while they may be very similar conceptually for a user, a provider would need to
handle each of these very differently for citations and in-document references.

And that's the point: while it makes sense for Ebib to provide insert and follow
capabilities for citations, there is really no point in Ebib providing those for
in-document references as well. It doesn't have the infrastructure for it, nor
is Ebib the first (or even second or third) option that comes to mind when you
think about inserting and following in-document references.

I do think it makes sense if such a hypothetical org-new-ref has a very similar
conceptual design to org-cite and a very similar user interface (e.g., the same
keybindings), and perhaps a part of the code can be shared, it should be
possible to register different provides for them.

-- 
Joost Kremers
Life has its moments

Re: [wip-cite-new] Merging tomorrow?

[Jens Neuhalfen]

Hi Nicolas,

first: thank you for all the work, especially for thinking of documentation for end users. My biggest struggle with the current org (and emacs) documentation is the lack of end-to-end examples. This makes it incredibly difficult to get things working. It often is like „this is a big puzzle, some pieces are in other boxes and we don’t provide a picture of how the final puzzle will look like“. The documentation often makes sense once I get it running, though . But this is to late.

This being the motivation, I would greatly appreciate the following:

——— snip ———-
Consider the following minimal viable example where an org file with one bibtex file is rendered to pdf and html.

This is the BibTex file
#+begin_example bibtex
….
#+end_example

This is the org file
#+begin_example org
….
#+end_example

and this is the rendered html/Latex 

- picture 1
- screenshot 2

This has been achieved with the following minimal configuration:

#+begin_example elisp 
….
#+end_example

As you can see in line 42 the … etc, etc

If you would like to use two bibtex files you would need to change …

#+begin_example elisp 
….
; change line 14 in the sample for one file like this:
(…)
…
#+end_example

——- snap ——-


That kind of documentation would have made many, many hours of frustrating „search in google/github for a solution someone else has found“ and replace it with „wow that was actually quite beginner friendly!“. Pictures also make it much easier to visualize what is actually achieved.

Cheers
Jens

> On 8. Jul 2021, at 02:18, Nicolas Goaziou <mail@nicolasgoaziou.fr> wrote:
> 
> Hello,
> 
> I think the "wip-cite-new" branch is in good shape now. As
> a consequence, I'd like to merge it tomorrow.
> 
> It is documented, but the documentation is scattered across the various
> "oc" libraries, and some threads in the mailing list. I'll do a summary
> here, from a user point of view.
> 
> --8<---------------cut here---------------start------------->8---
> Basically, in order to use it, you need to first set-up a bibliography,
> using one or more "bibliography" keywords. <C-c '> on such a keyword
> visits the related file. Out of the box, Org supports JSON-CSL and
> BibTeX (or biblatex) bibliographies.
> 
> Then, citations can be inserted with the following syntax:
> 
>  [cite/style:common prefix ;prefix @key suffix; ... ; common suffix]
> 
> Spaces are meaningful except those after the initial colon and before
> the closing bracket.
> 
> Every part of the syntax is optional, except the brackets, "cite" and
> the colon. Also the citation must contain at least a key. So its minimal
> form is:
> 
>  [cite:@key]
> 
> The "style" part is detailed below, in the part related to export.
> 
> Org can insert or edit citations with <C-c C-x @> (and delete them with
> <C-u C-c C-x @>), follow them with <C-c C-o>, fontify them, and export
> them. These four actions (insert, follow, activate, and export) are
> called capabilities.  Libraries responsible for these capabilities are
> called citation processors.
> 
> You can select one citation processor for each capability, independently
> on the others, through the following variables:
> 
> - org-cite-activate-processor
> - org-cite-export-processors
> - org-cite-follow-processor
> - org-cite-insert-processor
> 
> Out of the box, Org provides the "basic" (in "oc-basic.el") processor
> for all of these tasks. It also boasts processors dedicated for export:
> "csl", "natbib" and "biblatex".
> 
> During export, output for citations is controlled by their style, which
> is an Org label that the export processor may recognize and associate to
> a specific display, or fall-back to a default style (called "nil"). For
> example, most processors support "noauthor" and "text" styles. 
> 
> Some styles can accept a variant, with the syntax "style/variant".
> Again, it's up to the processor to associate it to a specific display.
> Common variants include "bare", "caps" or "full".  They also accept
> short-hands, like "b", "c" and "f".  Please refer to the export
> processors' libraries ("oc-basic.el", "oc-csl.el", …) for more information.
> 
> It is possible to define a default style for a whole document (with
> "cite_export"), or for all documents (with `org-cite-export-processors').
> 
> References are displayed with the "print_bibliography" keyword. It is
> possible to add parameters to its value, as some export processors could
> make use of them.
> --8<---------------cut here---------------end--------------->8---
> 
> Please let me know if there are any objections to the merge.
> 
> Regards,
> -- 
> Nicolas Goaziou
> 

Re: [wip-cite-new] Merging tomorrow?

[Greg Minshall]

Matt Price <moptop99@gmail.com> wrote:

> Really, I feel like there should be a parade.

+1

Re: [wip-cite-new] Merging tomorrow?

[Timothy]

Matt Price <moptop99@gmail.com> writes:

> Really, I feel like there should be a parade.

There will be one in the next edition of This Month in Org 😉.

-- 
Timothy

Re: [wip-cite-new] Merging tomorrow?

[Timothy]

Bruce D'Arcus <bdarcus@gmail.com> writes:

>> wip-cite-new deals with citing from bibliographies, but I don't think it
>> deals with within-document referencing --- should it?

> 1. Should it?
> 1. Maybe.

I feel like it would fit. With everything that's been done for
citations, this feels like it may be a rather minor addition (or at
least this is what I hope).

> 2. Can it? Could the design be extended to include internal referencing?
> 2. I think so. You'd just need a way to include internal targets in
> addition to the citation-references (keys); for illustration,
> something like [cite:#some-if].

I can't claim to have thought about this that much either, but something
like [cite:#some-fig] would seem to fit.

> 3. If yes to both, should that hold back merger now?
> 3. No.

I don't think this should hold up the merge either, but it's relevant in
the overall nature of the feature and perhaps could be shoehorned in
following the merge? I feel like this is one small quite simple case and
most of the thinking required has already been done. I'm not sure
though, I'd go with whatever Nic's thought are on this.

Whichever way this diversion goes, I'm excited to see wip-cite-new merged!

-- 
Timothy

Re: [wip-cite-new] Merging tomorrow?

[Nicolas Goaziou]

Hello,

Timothy <tecosaur@gmail.com> writes:

> Bruce D'Arcus <bdarcus@gmail.com> writes:
>
>>> wip-cite-new deals with citing from bibliographies, but I don't think it
>>> deals with within-document referencing --- should it?
>
>> 1. Should it?
>> 1. Maybe.
>
> I feel like it would fit. With everything that's been done for
> citations, this feels like it may be a rather minor addition (or at
> least this is what I hope).
>
>> 2. Can it? Could the design be extended to include internal referencing?
>> 2. I think so. You'd just need a way to include internal targets in
>> addition to the citation-references (keys); for illustration,
>> something like [cite:#some-if].
>
> I can't claim to have thought about this that much either, but something
> like [cite:#some-fig] would seem to fit.
>
>> 3. If yes to both, should that hold back merger now?
>> 3. No.
>
> I don't think this should hold up the merge either, but it's relevant in
> the overall nature of the feature and perhaps could be shoehorned in
> following the merge? I feel like this is one small quite simple case and
> most of the thinking required has already been done. I'm not sure
> though, I'd go with whatever Nic's thought are on this.

At this point, I don't have enough understanding of the problem to have
an opinion. IIUC, your example does not even mention citations. How
should it be used, what should be the output in LaTeX, and in UTF-8
export? This is not clear to me.

What can I say however is: if this feature implies to change, or extend,
syntax, then it is /de facto/ a blocker for the merge, and needs to be
sorted out.

Regards,
-- 
Nicolas Goaziou

Re: [wip-cite-new] Merging tomorrow?

[Bruce D'Arcus]

On Thu, Jul 8, 2021 at 6:26 AM Nicolas Goaziou <mail@nicolasgoaziou.fr> wrote:
>
> Hello,
>
> Timothy <tecosaur@gmail.com> writes:
>
> > Bruce D'Arcus <bdarcus@gmail.com> writes:
> >
> >>> wip-cite-new deals with citing from bibliographies, but I don't think it
> >>> deals with within-document referencing --- should it?
> >
> >> 1. Should it?
> >> 1. Maybe.
> >
> > I feel like it would fit. With everything that's been done for
> > citations, this feels like it may be a rather minor addition (or at
> > least this is what I hope).
> >
> >> 2. Can it? Could the design be extended to include internal referencing?
> >> 2. I think so. You'd just need a way to include internal targets in
> >> addition to the citation-references (keys); for illustration,
> >> something like [cite:#some-if].
> >
> > I can't claim to have thought about this that much either, but something
> > like [cite:#some-fig] would seem to fit.
> >
> >> 3. If yes to both, should that hold back merger now?
> >> 3. No.
> >
> > I don't think this should hold up the merge either, but it's relevant in
> > the overall nature of the feature and perhaps could be shoehorned in
> > following the merge? I feel like this is one small quite simple case and
> > most of the thinking required has already been done. I'm not sure
> > though, I'd go with whatever Nic's thought are on this.
>
> At this point, I don't have enough understanding of the problem to have
> an opinion. IIUC, your example does not even mention citations. How
> should it be used, what should be the output in LaTeX, and in UTF-8
> export? This is not clear to me.
>
> What can I say however is: if this feature implies to change, or extend,
> syntax, then it is /de facto/ a blocker for the merge, and needs to be
> sorted out.

As I was hinting, I don't know this area well either.

I think the first question is the "should" one; whether this is
in-scope of this module.

I wasn't sure, so said "maybe".

Joost says "no."

In latex, such internal references are not citations though; they use
a different mechanism.

Does that not suggest, Timothy, that this might be out-of-scope for
this module; that Joost is right?

Bruce

Re: [wip-cite-new] Merging tomorrow?

[John Kitchin]

[-- Attachment #1: Type: text/plain, Size: 2826 bytes --]

My intuition is that crossrefs are separate from the citations. In org-ref,
they are separate link types like ref:xxx, pageref:xxx. eqref:xxx, etc.
They also use a different source of candidates than cites do.

John

-----------------------------------
Professor John Kitchin (he/him/his)
Doherty Hall A207F
Department of Chemical Engineering
Carnegie Mellon University
Pittsburgh, PA 15213
412-268-7803
@johnkitchin
http://kitchingroup.cheme.cmu.edu



On Thu, Jul 8, 2021 at 7:16 AM Bruce D'Arcus <bdarcus@gmail.com> wrote:

> On Thu, Jul 8, 2021 at 6:26 AM Nicolas Goaziou <mail@nicolasgoaziou.fr>
> wrote:
> >
> > Hello,
> >
> > Timothy <tecosaur@gmail.com> writes:
> >
> > > Bruce D'Arcus <bdarcus@gmail.com> writes:
> > >
> > >>> wip-cite-new deals with citing from bibliographies, but I don't
> think it
> > >>> deals with within-document referencing --- should it?
> > >
> > >> 1. Should it?
> > >> 1. Maybe.
> > >
> > > I feel like it would fit. With everything that's been done for
> > > citations, this feels like it may be a rather minor addition (or at
> > > least this is what I hope).
> > >
> > >> 2. Can it? Could the design be extended to include internal
> referencing?
> > >> 2. I think so. You'd just need a way to include internal targets in
> > >> addition to the citation-references (keys); for illustration,
> > >> something like [cite:#some-if].
> > >
> > > I can't claim to have thought about this that much either, but
> something
> > > like [cite:#some-fig] would seem to fit.
> > >
> > >> 3. If yes to both, should that hold back merger now?
> > >> 3. No.
> > >
> > > I don't think this should hold up the merge either, but it's relevant
> in
> > > the overall nature of the feature and perhaps could be shoehorned in
> > > following the merge? I feel like this is one small quite simple case
> and
> > > most of the thinking required has already been done. I'm not sure
> > > though, I'd go with whatever Nic's thought are on this.
> >
> > At this point, I don't have enough understanding of the problem to have
> > an opinion. IIUC, your example does not even mention citations. How
> > should it be used, what should be the output in LaTeX, and in UTF-8
> > export? This is not clear to me.
> >
> > What can I say however is: if this feature implies to change, or extend,
> > syntax, then it is /de facto/ a blocker for the merge, and needs to be
> > sorted out.
>
> As I was hinting, I don't know this area well either.
>
> I think the first question is the "should" one; whether this is
> in-scope of this module.
>
> I wasn't sure, so said "maybe".
>
> Joost says "no."
>
> In latex, such internal references are not citations though; they use
> a different mechanism.
>
> Does that not suggest, Timothy, that this might be out-of-scope for
> this module; that Joost is right?
>
> Bruce
>
>

[-- Attachment #2: Type: text/html, Size: 4089 bytes --]

Re: [wip-cite-new] Merging tomorrow?

[Eric S Fraga]

On Thursday,  8 Jul 2021 at 11:47, Timothy wrote:
> wip-cite-new deals with citing from bibliographies, but I don't think it
> deals with within-document referencing --- should it?

Are these not orthogonal activities?  Doesn't org already support
in-document references?  I may be missing something but what you have
described has nothing to do with citations.

-- 
: Eric S Fraga via Emacs 28.0.50, Org release_9.4.6-579-gfdb98a
: Latest paper written in org: https://arxiv.org/abs/2106.05096

Re: [wip-cite-new] Merging tomorrow?

[Eric S Fraga]

On Thursday,  8 Jul 2021 at 02:17, Nicolas Goaziou wrote:
> I think the "wip-cite-new" branch is in good shape now. As
> a consequence, I'd like to merge it tomorrow.

Yes please!  I've been using it on and off (having to switch branches)
for some time now and it is working very well.  It needs to be in master
*now*!

One point, however:

> Basically, in order to use it, you need to first set-up a bibliography,
> using one or more "bibliography" keywords. <C-c '> on such a keyword
> visits the related file. 

Why C-c ' and not C-c C-o to be consistent with the rest of org?  For
me, it seems that in the rest of org, C-c ' is for editing something;
C-c C-o is for opening/visiting/following.

Not a show-stopper, more of a puzzling choice (to me).

Thank you for all the work on this new feature, in any case!

-- 
: Eric S Fraga via Emacs 28.0.50, Org release_9.4.6-579-gfdb98a
: Latest paper written in org: https://arxiv.org/abs/2106.05096

Re: [wip-cite-new] Merging tomorrow?

[Nicolas Goaziou]

Hello,

Eric S Fraga <e.fraga@ucl.ac.uk> writes:

> Why C-c ' and not C-c C-o to be consistent with the rest of org?  For
> me, it seems that in the rest of org, C-c ' is for editing something;
> C-c C-o is for opening/visiting/following.

Good question.

<C-c '> is "remote editing", <C-c C-o> is "follow link".

In this situation, I think

  #+bibliography: somefile.bib

is closer from

  #+include: somefile.org

or

  #+setupfile: config.org

than it is from

  [[file:somefile.org]]

So, I do think this is consistent with the rest of Org, actually.

Regards,
-- 
Nicolas Goaziou

Re: [wip-cite-new] Merging tomorrow?

[Eric S Fraga]

Okay, thank you.  I'll have to train myself to use C-c ' in more
situations (as I didn't know it would work on #+include etc. lines).

-- 
: Eric S Fraga via Emacs 28.0.50, Org release_9.4.6-579-gfdb98a
: Latest paper written in org: https://arxiv.org/abs/2106.05096

Re: [wip-cite-new] Merging tomorrow?

[Timothy]

[-- Attachment #1: Type: text/plain, Size: 1762 bytes --]

Hi Nicolas,

> At this point, I don’t have enough understanding of the problem to have
> an opinion. IIUC, your example does not even mention citations. How
> should it be used, what should be the output in LaTeX, and in UTF-8
> export? This is not clear to me.
>
> What can I say however is: if this feature implies to change, or extend,
> syntax, then it is /de facto/ a blocker for the merge, and needs to be
> sorted out.

This very much depends on how you view references vs citations. I personally
think of references as in-text citations (i.e. you’re citing other bits of the
very same document), but I doubt this is a common view (as suggested by other
replies).

To try to lay out what one may expect with references, I’d think some support in
Org (without org-ref et. al) would be good (at least for exporting) — but I’m
not sure what a good for would be.

I think it could be treated similarly to citations, given a variant syntax like
[<ref:#name>] or even just be added as a way of exporting links to named
figures/tables (i.e. ).

It’s a bit late to bring this up, but in case this should come under the
citation umbrella I thought I should.

Lastly, an example of what I’d expect when exporting to ascii (with three
example syntaxes):

┌────
│ #+name: sometab
│ #+caption: Some table
│ | a | b |
│ | c | d |
│ 
│ Hey, look at [[sometab]]. (or)
│ Hey, look at [cite:#sometab]. (or)
│ Hey, look at [ref:sometab].
└────

┌────
│ ━━━━━━
│  a  b 
│  c  d 
│ ━━━━━━
│ Table 1: Some table
│ 
│ Hey, look at Table 1.
└────

I hope this clears up what I was thinking.

All the best,
*Timothy*

Re: [wip-cite-new] Merging tomorrow?

[Bruce D'Arcus]

On Thu, Jul 8, 2021 at 9:43 AM Timothy <tecosaur@gmail.com> wrote:
>
> Hi Nicolas,
>
> > At this point, I don’t have enough understanding of the problem to have
> > an opinion. IIUC, your example does not even mention citations. How
> > should it be used, what should be the output in LaTeX, and in UTF-8
> > export? This is not clear to me.
> >
> > What can I say however is: if this feature implies to change, or extend,
> > syntax, then it is /de facto/ a blocker for the merge, and needs to be
> > sorted out.
>
> This very much depends on how you view references vs citations. I personally
> think of references as in-text citations (i.e. you’re citing other bits of the
> very same document), but I doubt this is a common view (as suggested by other
> replies).

I'm glad you raised the question, even if it is a bit late. But I
indeed don't think your view is common :-)

I decided to take a look at what a canonical guide like Chicago Manual
of Style says about this.

They call these "text references" (though confusingly, they use the
word "references" when discussing citations too).

For example, there are headings like "3.9: Text references to numbered
illustrations". or "2.30: Formatting text references and callouts to
tables and illustrations".

And then descriptions like "A text reference is addressed to the
reader (“see table 5,” or “fig. 3.2”) and will appear in the published
version."

Here's some stuff from APA, though they don't go into references much.

https://apastyle.apa.org/style-grammar-guidelines/tables-figures/figures

And finally, FWIW, MS Word and Open/LibreOffice implement support for
this via "cross-references":

https://guides.lib.umich.edu/c.php?g=283073&p=1888264

https://wiki.openoffice.org/wiki/Documentation/OOo3_User_Guides/Writer_Guide/Using_automatic_cross-references

So they may be similar, but are not the same.

And the implementation challenges raised by John Kitchin and Joost
Kremers (namely the candidate list is different) make this better to
deal with using a different mechanism.

As a package developer that supports org-cite, I really don't want to
be worrying about internal references and such.

And as a user, I don't see the benefit of doing so.

Effectively, these seem more like org internal links.

Bruce

Re: [wip-cite-new] Merging tomorrow?

[Nicolas Goaziou]

Timothy <tecosaur@gmail.com> writes:

> Lastly, an example of what I’d expect when exporting to ascii (with three
> example syntaxes):
>
> ┌────
> │ #+name: sometab
> │ #+caption: Some table
> │ | a | b |
> │ | c | d |
> │ 
> │ Hey, look at [[sometab]]. (or)
> │ Hey, look at [cite:#sometab]. (or)
> │ Hey, look at [ref:sometab].
> └────
>
> ┌────
> │ ━━━━━━
> │  a  b 
> │  c  d 
> │ ━━━━━━
> │ Table 1: Some table
> │ 
> │ Hey, look at Table 1.
> └────

I'm still lost, sorry.

--8<---------------cut here---------------start------------->8---
#+name: sometab
#+caption: Some table
| a | b |
| c | d |

Hey, look at Table [[sometab]].
--8<---------------cut here---------------end--------------->8---

is already exported as

--8<---------------cut here---------------start------------->8---
━━━━━━
 a  b 
 c  d 
━━━━━━
Table 1: Some table

Hey, look at Table 1.
--8<---------------cut here---------------end--------------->8---

Could you explain what you would like to see, in addition to what is
already possible?

I think, however, that it is not directly related to citations, unless
you want to be able to somehow link to a cite. Then we may have
a problem, because there is currently no way to name a cite. However, if
that ever makes sense, it is still possible to add a target next to it:

   <<@key>>[cite:@key]

Re: [wip-cite-new] Merging tomorrow?

[William Denton]

On 8 July 2021, Bruce D'Arcus wrote:

> And the implementation challenges raised by John Kitchin and Joost
> Kremers (namely the candidate list is different) make this better to
> deal with using a different mechanism.
>
> As a package developer that supports org-cite, I really don't want to
> be worrying about internal references and such.
>
> And as a user, I don't see the benefit of doing so.
>
> Effectively, these seem more like org internal links.

I agree with all this.  Personally, I'd never expect a citation system like this 
to handle internal references.  The way librarians think about citations and 
referencing, these are different.


Bill

--
William Denton
https://www.miskatonic.org/
Librarian, artist and licensed private investigator.

Re: [wip-cite-new] Merging tomorrow?

[Timothy]

Hi Nicolas,

In light of all the thoughts expressed on referencing, I no longer think it's a
good idea to have referencing capabilities in wip-cite-new.

I think referencing should get a bit of attention, as citation has here, but a
much smaller separate effort now appears more appropriate to me. This could be
as simple as a way of handling links to named images/tables/etc. when
exporting.

Update: I see wip-cite-new has been merged 🎉. Still sending this
because I think my sentiment is worth sharing and I'd like to prompt a
discussion on references (I'll probably give this a bit of a think then
start a new thread in a few days).

--
Timothy

William Denton <wtd@pobox.com> writes:

> On 8 July 2021, Bruce D'Arcus wrote:
>
>> And the implementation challenges raised by John Kitchin and Joost
>> Kremers (namely the candidate list is different) make this better to
>> deal with using a different mechanism.
>>
>> As a package developer that supports org-cite, I really don't want to
>> be worrying about internal references and such.
>>
>> And as a user, I don't see the benefit of doing so.
>>
>> Effectively, these seem more like org internal links.
>
> I agree with all this.  Personally, I'd never expect a citation system like this
> to handle internal references.  The way librarians think about citations and
> referencing, these are different.
>
>
> Bill


--
Timothy

Re: [wip-cite-new] Merging tomorrow?

[Eric S Fraga]

On Friday,  9 Jul 2021 at 15:58, Timothy wrote:
> This could be as simple as a way of handling links to named
> images/tables/etc. when exporting.

Maybe start a new thread, with a clear indication of what is missing in
the current version with respect to referencing.  I use internal
references all the time (to figures, tables, equations) and it works
great, at least for LaTeX export.  Is it that other export engines don't
do what is needed?

-- 
: Eric S Fraga via Emacs 28.0.50, Org release_9.4.6-579-gfdb98a
: Latest paper written in org: https://arxiv.org/abs/2106.05096

Re: [wip-cite-new] Merging tomorrow?

[William Denton]

Is the citation work big enough to move the version number for the next full 
release to 10?

Bill

--
William Denton
https://www.miskatonic.org/
Librarian, artist and licensed private investigator.

Re: [wip-cite-new] Merging tomorrow?

[Eric S Fraga]

On Friday,  9 Jul 2021 at 09:36, William Denton wrote:
> Is the citation work big enough to move the version number for the
> next full release to 10?

I guess it doesn't break anything (i.e. fully backwards compatible) so
no real need to bump the version number?

-- 
: Eric S Fraga via Emacs 28.0.50, Org release_9.4.6-579-gfdb98a
: Latest paper written in org: https://arxiv.org/abs/2106.05096

Re: [wip-cite-new] Merging tomorrow?

[John Kitchin]

[-- Attachment #1: Type: text/plain, Size: 3738 bytes --]

Is there a timeline for when this will be available in orgmode.org/elpa or
other package repository? I tried it today, but it doesn't seem to be there
yet, at least not in this version:

Org mode version 9.4.6 (9.4.6-10-gee652a-elpaplus @
/Users/jkitchin/Dropbox/emacs/scimax/elpa/org-plus-contrib-20210712/)

John

-----------------------------------
Professor John Kitchin (he/him/his)
Doherty Hall A207F
Department of Chemical Engineering
Carnegie Mellon University
Pittsburgh, PA 15213
412-268-7803
@johnkitchin
http://kitchingroup.cheme.cmu.edu



On Wed, Jul 7, 2021 at 8:18 PM Nicolas Goaziou <mail@nicolasgoaziou.fr>
wrote:

> Hello,
>
> I think the "wip-cite-new" branch is in good shape now. As
> a consequence, I'd like to merge it tomorrow.
>
> It is documented, but the documentation is scattered across the various
> "oc" libraries, and some threads in the mailing list. I'll do a summary
> here, from a user point of view.
>
> --8<---------------cut here---------------start------------->8---
> Basically, in order to use it, you need to first set-up a bibliography,
> using one or more "bibliography" keywords. <C-c '> on such a keyword
> visits the related file. Out of the box, Org supports JSON-CSL and
> BibTeX (or biblatex) bibliographies.
>
> Then, citations can be inserted with the following syntax:
>
>   [cite/style:common prefix ;prefix @key suffix; ... ; common suffix]
>
> Spaces are meaningful except those after the initial colon and before
> the closing bracket.
>
> Every part of the syntax is optional, except the brackets, "cite" and
> the colon. Also the citation must contain at least a key. So its minimal
> form is:
>
>   [cite:@key]
>
> The "style" part is detailed below, in the part related to export.
>
> Org can insert or edit citations with <C-c C-x @> (and delete them with
> <C-u C-c C-x @>), follow them with <C-c C-o>, fontify them, and export
> them. These four actions (insert, follow, activate, and export) are
> called capabilities.  Libraries responsible for these capabilities are
> called citation processors.
>
> You can select one citation processor for each capability, independently
> on the others, through the following variables:
>
> - org-cite-activate-processor
> - org-cite-export-processors
> - org-cite-follow-processor
> - org-cite-insert-processor
>
> Out of the box, Org provides the "basic" (in "oc-basic.el") processor
> for all of these tasks. It also boasts processors dedicated for export:
> "csl", "natbib" and "biblatex".
>
> During export, output for citations is controlled by their style, which
> is an Org label that the export processor may recognize and associate to
> a specific display, or fall-back to a default style (called "nil"). For
> example, most processors support "noauthor" and "text" styles.
>
> Some styles can accept a variant, with the syntax "style/variant".
> Again, it's up to the processor to associate it to a specific display.
> Common variants include "bare", "caps" or "full".  They also accept
> short-hands, like "b", "c" and "f".  Please refer to the export
> processors' libraries ("oc-basic.el", "oc-csl.el", …) for more information.
>
> It is possible to define a default style for a whole document (with
> "cite_export"), or for all documents (with `org-cite-export-processors').
>
> References are displayed with the "print_bibliography" keyword. It is
> possible to add parameters to its value, as some export processors could
> make use of them.
> --8<---------------cut here---------------end--------------->8---
>
> Please let me know if there are any objections to the merge.
>
> Regards,
> --
> Nicolas Goaziou
>
>

[-- Attachment #2: Type: text/html, Size: 4821 bytes --]

Re: [wip-cite-new] Merging tomorrow?

[Eric S Fraga]

Hello all,

A quick question: I am exporting to ODT and am happy with the basic cite
exporter.  I want numeric citations.  Does oc-basic allow me to somehow
specify that the reference (in the text) be numeric by default and not
authors and year?  I can use [@cite/nb:...] with

#+cite-export: basic numeric

but if I forget the /nb, I get (author year) in the text.

Am I missing something obvious?  The documentation is obviously still
lacking (not a complaint, just reality).

Thank you,
eric

PS - I normally export to LaTeX which is working just fine, by the way!

-- 
: Eric S Fraga via Emacs 28.0.50, Org release_9.4.6-625-ge7454c
: Latest paper written in org: https://arxiv.org/abs/2106.05096

Re: [wip-cite-new] Merging tomorrow?

[Eric S Fraga]

Ignore my question.  The following works (had to look at the source,
which I should have done in the first place, of course <blush>):

#+cite_export: basic numeric nb

Thank you and back to your normal programme ... ;-)

-- 
: Eric S Fraga via Emacs 28.0.50, Org release_9.4.6-625-ge7454c
: Latest paper written in org: https://arxiv.org/abs/2106.05096

Re: [wip-cite-new] Merging tomorrow?

[Bruce D'Arcus]

Though for ODT, you might want to look into oc-csl; citeproc-el just
got improved ODF support.

https://github.com/andras-simonyi/citeproc-el/pull/45

On Fri, Aug 20, 2021 at 9:46 AM Eric S Fraga <e.fraga@ucl.ac.uk> wrote:
>
> Ignore my question.  The following works (had to look at the source,
> which I should have done in the first place, of course <blush>):
>
> #+cite_export: basic numeric nb
>
> Thank you and back to your normal programme ... ;-)
>
> --
> : Eric S Fraga via Emacs 28.0.50, Org release_9.4.6-625-ge7454c
> : Latest paper written in org: https://arxiv.org/abs/2106.05096
>

Re: [wip-cite-new] Merging tomorrow?

[Eric S Fraga]

On Friday, 20 Aug 2021 at 09:47, Bruce D'Arcus wrote:
> Though for ODT, you might want to look into oc-csl; citeproc-el just
> got improved ODF support.

Thank you for the heads up on this.  Looks promising for those that
regularly export to ODT!

In my case, I just need a quick and dirty draft so that I can get the
core of the text into a Word document being worked on by several
people.  I hate writing in a word processor (my fingers just know Emacs
so I often end up deleting whole sections of text when I type C-x... not
to mention C-q ;-)) so prefer doing the bulk of the writing in
org.  Once in the Word document, references etc. will be re-formatted
manually by others.

Thanks again,
eric

-- 
: Eric S Fraga via Emacs 28.0.50, Org release_9.4.6-625-ge7454c
: Latest paper written in org: https://arxiv.org/abs/2106.05096

Re: [wip-cite-new] Merging tomorrow?

[Bruce D'Arcus]

I think so.

I actually just started the process of removing the bibtex-completion
dependency.

Give it a try in any case. It should work well with org-cite, and you
can even use pieces of org-ref-cite if you prefer that to my
(currently) more minimal embark-based approach.

On Fri, Aug 20, 2021 at 10:53 AM Eric S Fraga <e.fraga@ucl.ac.uk> wrote:
>
> Bruce,
>
> off-list, I notice that bibtex-actions has been updated.  I have been
> using bibtex-completions but as I now longer use either ivy or helm,
> having switched to selectrum, I should probably switch to your
> package.  Would this be a correct view/conclusion?
>
> Thank you,
> eric
>
> --
> : Professor Eric S Fraga
> : Fresa black box optimization: doi:10.5281/zenodo.5045812
> : PGP/GnuPG key: 8F5C 279D 3907 E14A 5C29  570D C891 93D8 FFFC F67D
>
> Latest publications: doi:10.3303/CET2186011 & arxiv:2106.05096

Re: [wip-cite-new] Merging tomorrow?

[John Kitchin]

[-- Attachment #1: Type: text/plain, Size: 1224 bytes --]

I think bibtex-completion is agnostic of ivy or helm, and doesn't require
either of them to work. You can use it for candidates to selectrum if you
want, and the other many features it offers for notes, pdf, etc...


On Fri, Aug 20, 2021 at 11:07 AM Bruce D'Arcus <bdarcus@gmail.com> wrote:

> I think so.
>
> I actually just started the process of removing the bibtex-completion
> dependency.
>
> Give it a try in any case. It should work well with org-cite, and you
> can even use pieces of org-ref-cite if you prefer that to my
> (currently) more minimal embark-based approach.
>
> On Fri, Aug 20, 2021 at 10:53 AM Eric S Fraga <e.fraga@ucl.ac.uk> wrote:
> >
> > Bruce,
> >
> > off-list, I notice that bibtex-actions has been updated.  I have been
> > using bibtex-completions but as I now longer use either ivy or helm,
> > having switched to selectrum, I should probably switch to your
> > package.  Would this be a correct view/conclusion?
> >
> > Thank you,
> > eric
> >
> > --
> > : Professor Eric S Fraga
> > : Fresa black box optimization: doi:10.5281/zenodo.5045812
> > : PGP/GnuPG key: 8F5C 279D 3907 E14A 5C29  570D C891 93D8 FFFC F67D
> >
> > Latest publications: doi:10.3303/CET2186011 & arxiv:2106.05096
>
>

[-- Attachment #2: Type: text/html, Size: 1896 bytes --]

Re: [wip-cite-new] Merging tomorrow?

[Bruce D'Arcus]

On Fri, Aug 20, 2021 at 11:21 AM John Kitchin <jkitchin@andrew.cmu.edu> wrote:

> I think bibtex-completion is agnostic of ivy or helm, and doesn't require either of them to work. You can use it for candidates to selectrum if you want, and the other many features it offers for notes, pdf, etc...

Sorry; meant to send that off-list.

But yes, to clarify: bibtex-completion is independent of the frontend.

I was mostly removing my dependency on it to get some of the benefits
of the new parsebib release (namely CSL JSON support), and because I
had some ideas I wanted to explore.

But in any case, bibtex-completion provides no front-end; you still
need a completing-read function for that, and I think bibtex-actions
provides a good one. Also, you need to wrap it's functions to get
interactive commands.

Re: [wip-cite-new] Merging tomorrow?

[Eric S Fraga]

On Friday, 20 Aug 2021 at 09:47, Bruce D'Arcus wrote:
> Though for ODT, you might want to look into oc-csl; citeproc-el just
> got improved ODF support.

So I decided to try this out.  Not working for me, but early days in
debugging.  For some reason, when I export to ODT, the "format" for the
export is "org" and not "org-odt" in citeproc, specifically in the call
to =citeproc-formatter-for-format=.

I haven't figured out how this format is set; early days, as I
said.  It's now the weekend and I need a break but if I learn anything,
I'll update on the list.

-- 
: Eric S Fraga via Emacs 28.0.50, Org release_9.4.6-628-g366444
: Latest paper written in org: https://arxiv.org/abs/2106.05096