[PATCH] oc-csl: New custom option `org-cite-csl-sentence-case-bibtex-titles'

[PATCH] oc-csl: New custom option `org-cite-csl-sentence-case-bibtex-titles'

[András Simonyi]

[-- Attachment #1: Type: text/plain, Size: 880 bytes --]

Dear All,

since bibtex and biblatex requires title fields to be in title case
but CSL assumes that they are in sentence-case, citeproc-el converts
title fields in bib(la)tex bibliography databases into sentence-case
before processing them except for entries with an explicit non-English
langid value. Although this seems to be reasonable behaviour, there
were several requests in the last couple of years to make it possible
to turn this conversion off (see, e.g., citeproc issues  #119 and
#142). The attached patch introduces a new custom option to configure
when to perform the conversion.

I'm a bit unsure about naming the option:
Perhaps `org-cite-csl-sentence-case-bibtex-titles-without-langid'
would be more precise but I found it absurdly long and technical, as
most users are probably unaware of the existence of langid fields.

best wishes,
András

[-- Attachment #2: 0001-oc-csl-New-custom-option-org-cite-csl-sentence-case-.patch --]
[-- Type: text/x-patch, Size: 2771 bytes --]

From 031428611c18bb4d97bbdbd11a7549ca2b96ccec Mon Sep 17 00:00:00 2001
From: Andras Simonyi <andras.simonyi@gmail.com>
Date: Sat, 11 May 2024 11:20:41 +0200
Subject: [PATCH] oc-csl: New custom option
 `org-cite-csl-sentence-case-bibtex-titles'

* lisp/oc-csl.el (org-cite-csl-sentence-case-bibtex-titles): New
variable.
(org-cite-csl--processor): Create the itemgetter using the new option.
* etc/ORG-NEWS (New option
~org-cite-csl-sentence-case-bibtex-titles~): Announce the change.
---
 etc/ORG-NEWS   |  9 +++++++++
 lisp/oc-csl.el | 18 +++++++++++++++++-
 2 files changed, 26 insertions(+), 1 deletion(-)

diff --git a/etc/ORG-NEWS b/etc/ORG-NEWS
index 36eeddda1..9ca71d0a2 100644
--- a/etc/ORG-NEWS
+++ b/etc/ORG-NEWS
@@ -1128,6 +1128,15 @@ blocks that do not specify any ~:formatter~ parameter. Its default
 value (the new function ~org-columns-dblock-write-default~) yields the
 previous (fixed) formatting behaviour.
 
+*** New option ~org-cite-csl-sentence-case-bibtex-titles~
+
+When this option is non-nil then title fields in bibtex bibliography
+entries are converted to sentence-case before being formatted
+according to a CSL style, except for entries with a =langid= field
+specifying a non-English language.  When nil, this conversion is
+limited to entries having a =langid= field specifying a variant of
+English.  The defult value is ~t~.
+
 ** New features
 *** =ob-lua=: Support all types and multiple values in results
 
diff --git a/lisp/oc-csl.el b/lisp/oc-csl.el
index 9bbe5e29d..7234174d0 100644
--- a/lisp/oc-csl.el
+++ b/lisp/oc-csl.el
@@ -321,6 +321,21 @@ in the bibliography measured in characters."
   :type 'string
   :package-version '(Org . "9.7"))
 
+(defcustom org-cite-csl-sentence-case-bibtex-titles t
+  "Convert bibtex title fields to sentence-case by default.
+
+When non-nil, title fields in bibtex bibliography entries are
+converted to sentence-case before being formatted according to a
+CSL style, except for entries with a `langid' field specifying a
+non-English language.
+
+When nil, this conversion is limited to entries having a `langid'
+field specifying a variant of English."
+  :group 'org-cite
+  :package-version '(Org . "9.7")
+  :type 'boolean
+  :safe #'booleanp)
+
 \f
 ;;; Internal variables
 (defconst org-cite-csl--etc-dir
@@ -584,7 +599,8 @@ property in INFO."
              (processor
               (citeproc-create
                (org-cite-csl--style-file info)
-               (citeproc-hash-itemgetter-from-any bibliography)
+               (citeproc-hash-itemgetter-from-any
+                bibliography (not org-cite-csl-sentence-case-bibtex-titles))
                (org-cite-csl--locale-getter)
                locale)))
         (plist-put info :cite-citeproc-processor processor)
-- 
2.34.1

Re: [PATCH] oc-csl: New custom option `org-cite-csl-sentence-case-bibtex-titles'

[Ihor Radchenko]

András Simonyi <andras.simonyi@gmail.com> writes:

> since bibtex and biblatex requires title fields to be in title case
> ...

Are you sure? AFAIK, bibtex and biblatex (depending on the bibstyle)
does not care about capitalization and instead applies its own, unless
the title explicitly protects the capitalization/case with {Curly
BracketS}.

See https://texfaq.org/FAQ-capbibtex and
https://tex.stackexchange.com/questions/20335/proper-casing-in-citation-bibliography-titles-using-biblatex-biber

> ... but CSL assumes that they are in sentence-case, citeproc-el converts
> title fields in bib(la)tex bibliography databases into sentence-case
> before processing them except for entries with an explicit non-English
> langid value...

AFAIU, the general recommendation is to use sentence case in the bib
files. Both for Bib(La)Tex (because it converts into title case if
necessary by itself) and for CSL, according to
https://citationstyles.org/authors/

Of course, not every real-life bibliography follows such suggestion,
but, as stated in https://citationstyles.org/authors/, converting from
title case to sentence case is error-prone:

>> For this reason, we recommend that you store all titles in your
>> reference database in sentence case. Our repository CSL styles that
>> need sentence case will generally just print titles as is, whereas
>> styles that need title case will use an automatic title-case
>> conversion.

Looking at https://github.com/andras-simonyi/citeproc-el/issues/119 and
https://github.com/andras-simonyi/citeproc-el/issues/142, it appears to
me that citeproc.el does not stick to the above guideline from CSL
website - unless title case is requested by the bibliography converting
to sentence case should not be done by default. Also, such conversion
should only happen in titles, AFAIU; not in other fields.

> Subject: [PATCH] oc-csl: New custom option
> `org-cite-csl-sentence-case-bibtex-titles'

I see no problem with the new feature, but I'd consider flipping the
default to nil.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: [PATCH] oc-csl: New custom option `org-cite-csl-sentence-case-bibtex-titles'

[András Simonyi]

Dear All,

On Sat, 11 May 2024 at 19:24, Ihor Radchenko <yantar92@posteo.net> wrote:

>> > since bibtex and biblatex requires title fields to be in title case
> Are you sure? AFAIK, bibtex and biblatex (depending on the bibstyle)
> does not care about capitalization and instead applies its own, unless
> the title explicitly protects the capitalization/case with {Curly
> BracketS}.

Yes, I'm pretty sure that the expected casing in .bib bibliography
databases for the title fields is by default title case, plus
protective braces around texts whose case shouldn't be touched during
formatting (of course, formatting itself can produce both sentence and
title case from this input depending on the used style). This
requirement was already clearly stated in Lamport's original LaTeX
book. To quote the relevant part of "The Bibliography Database"
chapter  (2nd edition, p. 158):

> The bibliography style determines whether or not a title is capitalized; the
> titles of books usually are, the titles of articles usually are not. You type
> a title the way it should appear if it is capitalized. You should capitalize
> the first word of the title, the first word after a colon, and all other words
> except articles and unstressed conjunctions and prepositions. BiBTeX will
> change uppercase letters to lowercase if appropriate. Uppercase letters that
> should not be changed are enclosed in braces.

These requirements haven't changed since then and also hold for
biblatex, see, e.g.,
https://tex.stackexchange.com/questions/439440/what-is-the-proper-casing-to-use-when-storing-titles-in-the-bibliography-database.
If you are interested, you can also look at the discussion concerning
the citeproc-el conversion implementation at
https://github.com/andras-simonyi/citeproc-el/issues/71, see also the
"Capitalization in titles" section in
Pandoc's User’s Guide.

The proposed default makes it possible to use a .bib bibliography
database that conforms to the standard title field format requirements
(title case etc.) and get the intended output corresponding to the
used citation style both with bib(la)tex and CSL styles.  Moreover, it
matches Pandoc's behaviour, see
https://github.com/jgm/pandoc-citeproc/issues/269.

best wishes,
András

Re: [PATCH] oc-csl: New custom option `org-cite-csl-sentence-case-bibtex-titles'

[Ihor Radchenko]

András Simonyi <andras.simonyi@gmail.com> writes:

> On Sat, 11 May 2024 at 19:24, Ihor Radchenko <yantar92@posteo.net> wrote:
>
>>> > since bibtex and biblatex requires title fields to be in title case
>> Are you sure? AFAIK, bibtex and biblatex (depending on the bibstyle)
>> does not care about capitalization and instead applies its own, unless
>> the title explicitly protects the capitalization/case with {Curly
>> BracketS}.
>
> Yes, I'm pretty sure that the expected casing in .bib bibliography
> databases for the title fields is by default title case, plus
> protective braces around texts whose case shouldn't be touched during
> formatting (of course, formatting itself can produce both sentence and
> title case from this input depending on the used style). This
> requirement was already clearly stated in Lamport's original LaTeX
> book. To quote the relevant part of "The Bibliography Database"
> chapter  (2nd edition, p. 158):
> ...
> These requirements haven't changed since then and also hold for
> biblatex, see, e.g.,
> ...

Thanks for the explanation.
I also cross-checked with the examples given in
https://ctan.org/pkg/bibtex - all consistent.

> I'm a bit unsure about naming the option:
> Perhaps `org-cite-csl-sentence-case-bibtex-titles-without-langid'
> would be more precise but I found it absurdly long and technical, as
> most users are probably unaware of the existence of langid fields.

Maybe org-cite-csl-sentence-case-english-titles?

Also, it would be nice to point out the CSL and Bibtex have different
conventions for the title field.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: [PATCH] oc-csl: New custom option `org-cite-csl-sentence-case-bibtex-titles'

[Ihor Radchenko]

Ihor Radchenko <yantar92@posteo.net> writes:

>> I'm a bit unsure about naming the option:
>> Perhaps `org-cite-csl-sentence-case-bibtex-titles-without-langid'
>> would be more precise but I found it absurdly long and technical, as
>> most users are probably unaware of the existence of langid fields.
>
> Maybe org-cite-csl-sentence-case-english-titles?
>
> Also, it would be nice to point out the CSL and Bibtex have different
> conventions for the title field.

Andras, it has been a month since the last message in this thread.
May I know about the status of the patch?

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: [PATCH] oc-csl: New custom option `org-cite-csl-sentence-case-bibtex-titles'

[András Simonyi]

Dear Ihor,
thanks for asking -- unfortunately, I've been absurdly busy in the
last few weeks but it's getting better now, so I'm going to respond
shortly with a new, hopefully final, version.
best wishes,
András

On Mon, 17 Jun 2024 at 13:37, Ihor Radchenko <yantar92@posteo.net> wrote:
>
> Ihor Radchenko <yantar92@posteo.net> writes:
>
> >> I'm a bit unsure about naming the option:
> >> Perhaps `org-cite-csl-sentence-case-bibtex-titles-without-langid'
> >> would be more precise but I found it absurdly long and technical, as
> >> most users are probably unaware of the existence of langid fields.
> >
> > Maybe org-cite-csl-sentence-case-english-titles?
> >
> > Also, it would be nice to point out the CSL and Bibtex have different
> > conventions for the title field.
>
> Andras, it has been a month since the last message in this thread.
> May I know about the status of the patch?
>
> --
> Ihor Radchenko // yantar92,
> Org mode contributor,
> Learn more about Org mode at <https://orgmode.org/>.
> Support Org development at <https://liberapay.com/org-mode>,
> or support my work at <https://liberapay.com/yantar92>

Re: [PATCH] oc-csl: New custom option `org-cite-csl-sentence-case-bibtex-titles'

[András Simonyi]

[-- Attachment #1: Type: text/plain, Size: 3051 bytes --]

Dear Ihor and Max,

thanks for the feedback and apologies for responding that late.

On Wed, 15 May 2024 at 13:48, Max Nikulin <manikulin@gmail.com> wrote:

> Is it possible to keep title formatting from .bib files till it becomes
> known that specific style requires sentence case for particular entry
> type? I had a hope that it might alleviate the issue and to make things
> working out of the box for more users.

The problem is that many (maybe most) CSL styles simply assume that
titles are in sentence-case in the input without
making this explicit. The APA citation style is a prominent example:
the APA guidelines require sentence-case titles but
AFAICS the standard CSL implementations do not encode this explicitly.
Of course it could be checked whether
a CSL style contains explicit instructions about how to format titles
and skip  sentence-case conversion of input for those but this
probably would not
solve the problems of users who do not want the conversion at all.

> > I'm a bit unsure about naming the option:
> > Perhaps `org-cite-csl-sentence-case-bibtex-titles-without-langid'
>
> A variant: org-cite-csl-bibtex-title-to-sentence-case

thanks, in the attached patch I went with (a slightly modified version of) this.

> Does it means that you added one more argument to `citeproc-create' and
> that consistent Org and citeproc-el versions must be used?

no, at least in the sense that this argument has been there since 2021.

> It may be tricky to preserve
> backward-forward compatibility on this step, but it should make further
> changes easier. It may be reasonable to explicitly add version of
> "protocol" to the property list, so that citeproc-el may decide if error
> should be signaled in the case of serious version difference.

although I don't think backward compatibility could be a problem in
that particular case, in the future
a mechanism like this  might be useful, so I plan to implement it,
thanks for the suggestion.

> It is not clear for me why `org-cite-csl-sentence-case-bibtex-titles' is
> a part of Org, not of citeproc-el. The only thing that Org can do is to
> pass it to citeproc-el. It is not configurable per .org file and likely
> it should not be. From my point of view it might be more suitable per
> .bib file. Anyway it is almost unrelated to Org.

citeproc-el doesn't have customizable user options by design because
it was conceived as a relatively low level
rendering library. Also, it's used by several user-facing packages by
now in addition to Org (org-ref, citar etc.) so
I think it would be both confusing and difficult to try to keep this
type of customizations in the library.

> On Mon, 17 Jun 2024 at 13:37, Ihor Radchenko <yantar92@posteo.net> wrote:

> > > Also, it would be nice to point out the CSL and Bibtex have different> > > conventions for the title field.

I added a few words about this now both in the docstring and in the
ORG-NEWS entry -- I've attached
the updated patch.

best wishes,
András

[-- Attachment #2: 0001-oc-csl-New-custom-option-org-cite-csl-bibtex-titles-.patch --]
[-- Type: text/x-patch, Size: 3251 bytes --]

From e89c7872d8c287fb3affd9c815a1423b366c2bb9 Mon Sep 17 00:00:00 2001
From: Andras Simonyi <andras.simonyi@gmail.com>
Date: Wed, 19 Jun 2024 08:26:19 +0200
Subject: [PATCH] oc-csl: New custom option
 `org-cite-csl-bibtex-titles-to-sentence-case'

* lisp/oc-csl.el (org-cite-csl-bibtex-titles-to-sentence-case): New
variable.
(org-cite-csl--processor): Create the itemgetter using the new option.
* etc/ORG-NEWS (New option
~org-cite-csl-bibtex-titles-to-sentence-case~): Announce the change.
---
 etc/ORG-NEWS   | 11 +++++++++++
 lisp/oc-csl.el | 21 ++++++++++++++++++++-
 2 files changed, 31 insertions(+), 1 deletion(-)

diff --git a/etc/ORG-NEWS b/etc/ORG-NEWS
index b81dabb15..78a60382d 100644
--- a/etc/ORG-NEWS
+++ b/etc/ORG-NEWS
@@ -61,6 +61,17 @@ The default value used to be pulled from =dvipng= process type from
 =latexmk= (when available), or running =latex= multiple times, so that
 all the references are resolved in the generated png.
 
+*** New option ~org-cite-csl-bibtex-titles-to-sentence-case~
+
+When this option is non-nil then title fields in bibtex bibliography
+entries are converted to sentence-case before being formatted
+according to a CSL style, except for entries with a =langid= field
+specifying a non-English language.  When nil, this conversion is
+limited to entries having a =langid= field specifying a variant of
+English.  The default value is ~t~ as the CSL standard assumes that
+English titles are specified in sentence-case but the bibtex
+bibliography format requires them to be written in title-case.
+
 ** New functions and changes in function arguments
 
 # This also includes changes in function behavior from Elisp perspective.
diff --git a/lisp/oc-csl.el b/lisp/oc-csl.el
index b8691bf6f..338f72e9d 100644
--- a/lisp/oc-csl.el
+++ b/lisp/oc-csl.el
@@ -321,6 +321,24 @@ in the bibliography measured in characters."
   :type 'string
   :package-version '(Org . "9.7"))
 
+(defcustom org-cite-csl-bibtex-titles-to-sentence-case t
+  "Convert bibtex title fields to sentence-case by default.
+
+When non-nil, title fields in bibtex bibliography entries are
+converted to sentence-case before being formatted according to a
+CSL style, except for entries with a `langid' field specifying a
+non-English language.  When nil, title conversion is limited to
+entries having a `langid' field specifying a variant of English.
+
+Conversion of titles to sentence-case by default is in most cases
+useful because the CSL standard assumes that English titles are
+specified in sentence-case but the bibtex bibliography format
+requires them to be written in title-case."
+  :group 'org-cite
+  :package-version '(Org . "9.8")
+  :type 'boolean
+  :safe #'booleanp)
+
 \f
 ;;; Internal variables
 (defconst org-cite-csl--etc-dir
@@ -579,7 +597,8 @@ property in INFO."
              (processor
               (citeproc-create
                (org-cite-csl--style-file info)
-               (citeproc-hash-itemgetter-from-any bibliography)
+               (citeproc-hash-itemgetter-from-any
+                bibliography (not org-cite-csl-bibtex-titles-to-sentence-case))
                (org-cite-csl--locale-getter)
                locale)))
         (plist-put info :cite-citeproc-processor processor)
-- 
2.34.1

Re: [PATCH] oc-csl: New custom option `org-cite-csl-sentence-case-bibtex-titles'

[Ihor Radchenko]

András Simonyi <andras.simonyi@gmail.com> writes:

> I added a few words about this now both in the docstring and in the
> ORG-NEWS entry -- I've attached
> the updated patch.

Thanks! I have no further comments.
Feel free to install the patch.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: [PATCH] oc-csl: New custom option `org-cite-csl-sentence-case-bibtex-titles'

[András Simonyi]

Dear All,

On Wed, 19 Jun 2024 at 16:35, Ihor Radchenko <yantar92@posteo.net> wrote:

> Thanks! I have no further comments.
> Feel free to install the patch.

Thanks, I'have applied the patch to main.
https://git.savannah.gnu.org/cgit/emacs/org-mode.git/commit/?id=782f6c33beed8d7baac9549b94a6a08040eba838

best wishes,
András

Re: [PATCH] oc-csl: New custom option `org-cite-csl-sentence-case-bibtex-titles'

[Max Nikulin]

On 19/06/2024 16:27, András Simonyi wrote:
> On Wed, 15 May 2024 at 13:48, Max Nikulin wrote:
> 
>> I had a hope that it might alleviate the issue and to make things
>> working out of the box for more users.
[...]
> Of course it could be checked whether
> a CSL style contains explicit instructions about how to format titles
> and skip  sentence-case conversion of input for those but this
> probably would not
> solve the problems of users who do not want the conversion at all.

Do you mean using title (original) case despite some style requites 
sentence case for English entries? I believed that fields should be 
surrounded with double braces {{...}} this case and the only issue is 
difference of original capitalization and result of conversion to 
sentence case and then back to title case.

>> It is not clear for me why `org-cite-csl-sentence-case-bibtex-titles' is
>> a part of Org, not of citeproc-el. The only thing that Org can do is to
>> pass it to citeproc-el. It is not configurable per .org file and likely
>> it should not be. From my point of view it might be more suitable per
>> .bib file. Anyway it is almost unrelated to Org.
> 
> citeproc-el doesn't have customizable user options by design because
> it was conceived as a relatively low level
> rendering library. Also, it's used by several user-facing packages by
> now in addition to Org (org-ref, citar etc.) so
> I think it would be both confusing and difficult to try to keep this
> type of customizations in the library.

My expectation is that to get consistent results across applications, 
they should share the same preference. However since your choice is 
conscious, I do not insist.

Re: [PATCH] oc-csl: New custom option `org-cite-csl-sentence-case-bibtex-titles'

[András Simonyi]

Dear All,

(sorry for the late reply again)
On Fri, 21 Jun 2024 at 12:27, Max Nikulin <manikulin@gmail.com> wrote:
>
> On 19/06/2024 16:27, András Simonyi wrote:
> > Of course it could be checked whether
> > a CSL style contains explicit instructions about how to format titles
> > and skip  sentence-case conversion of input for those but this
> > probably would not
> > solve the problems of users who do not want the conversion at all.
>
> Do you mean using title (original) case despite some style requites
> sentence case for English entries? I believed that fields should be
> surrounded with double braces {{...}} this case and the only issue is
> difference of original capitalization and result of conversion to
> sentence case and then back to title case.

Well, what they typically want is using the original title case
despite a style not containing any explicit instruction that the title
field should be in title-case, which, according to the latest. 1.02
CSL standard (https://docs.citationstyles.org/en/stable/specification.html),
implies that the field in question  should be formatted in
sentence-case:

"it is generally preferable to store strings such as titles in
sentence case, and only use text-case if a style desires another case.
Sentence case conversion is deprecated and will be removed in a future version."

> My expectation is that to get consistent results across applications,
> they should share the same preference. However since your choice is
> conscious, I do not insist.

thanks, I agree in general, but in this particular case it seems to me
that, in practice, it'd be hard to introduce common settings across
applications, especially for org-ref and org-cite, both of which have
a number of CSL-related user options.

best wishes,
András

Re: [PATCH] oc-csl: New custom option `org-cite-csl-sentence-case-bibtex-titles'

[Max Nikulin]

On 06/07/2024 16:59, András Simonyi wrote:
> 
> Well, what they typically want is using the original title case
> despite a style not containing any explicit instruction that the title
> field should be in title-case, which, according to the latest. 1.02
> CSL standard (https://docs.citationstyles.org/en/stable/specification.html),
> implies that the field in question  should be formatted in
> sentence-case:

If a style does not specify conversion or use "title" then result should 
be in title case. Title case is recommended for BibTeX databases. If you 
pass string in title case then conversion to title case should not 
change it. You do not need any conversion.

Original BibTeX title should be converted to sentence case only if 
specific style explicitly demand it.

I still have a hope that it should work without user settings in most 
cases. Maybe it is better to resort to particular examples with some 
style and a bib entry.

Re: [PATCH] oc-csl: New custom option `org-cite-csl-sentence-case-bibtex-titles'

[András Simonyi]

On Mon, 8 Jul 2024 at 12:57, Max Nikulin <manikulin@gmail.com> wrote:
> On 06/07/2024 16:59, András Simonyi wrote:
> If a style does not specify conversion or use "title" then result should
> be in title case. Title case is recommended for BibTeX databases. If you
> pass string in title case then conversion to title case should not
> change it. You do not need any conversion.
>
> Original BibTeX title should be converted to sentence case only if
> specific style explicitly demand it.

What I've been arguing for in this thread and what motivates the
current default behavior is that it is a general, style-independent
CSL assumption that the processor's input for title fields is in
sentence-case, and this requires converting the BibLaTeX title fields
in title-case to sentence-case before feeding into the CSL processor
independently of the style  -- please refer to my email dated 14 May
for further details and the references therein.

best wishes,
András

Re: [PATCH] oc-csl: New custom option `org-cite-csl-sentence-case-bibtex-titles'

[András Simonyi]

A quick correction: instead of "BibLaTex title fields" I should have written
"BibTeX or biblatex title fields".

best wishes,
András

On Mon, 8 Jul 2024 at 14:56, András Simonyi <andras.simonyi@gmail.com> wrote:
>
> On Mon, 8 Jul 2024 at 12:57, Max Nikulin <manikulin@gmail.com> wrote:
> > On 06/07/2024 16:59, András Simonyi wrote:
> > If a style does not specify conversion or use "title" then result should
> > be in title case. Title case is recommended for BibTeX databases. If you
> > pass string in title case then conversion to title case should not
> > change it. You do not need any conversion.
> >
> > Original BibTeX title should be converted to sentence case only if
> > specific style explicitly demand it.
>
> What I've been arguing for in this thread and what motivates the
> current default behavior is that it is a general, style-independent
> CSL assumption that the processor's input for title fields is in
> sentence-case, and this requires converting the BibLaTeX title fields
> in title-case to sentence-case before feeding into the CSL processor
> independently of the style  -- please refer to my email dated 14 May
> for further details and the references therein.
>
> best wishes,
> András

Re: [PATCH] oc-csl: New custom option `org-cite-csl-sentence-case-bibtex-titles'

[Max Nikulin]

On 08/07/2024 19:56, András Simonyi wrote:
> What I've been arguing for in this thread and what motivates the
> current default behavior is that it is a general, style-independent
> CSL assumption that the processor's input for title fields is in
> sentence-case, and this requires converting the BibLaTeX title fields
> in title-case to sentence-case before feeding into the CSL processor
> independently of the style  -- please refer to my email dated 14 May
> for further details and the references therein.

András, in 
https://list.orgmode.org/d8b1b73f-6b4e-4946-8873-748f78e19468@gmail.com 
I tried to ask if it is possible to determine if a specific CSL style 
uses title or sentence case. I believe that without "independently of 
the style" requirement behavior may be more convenient for users.

Consider a user who have .bib files following BibTeX recommendations: 
title case with hints where capital letters should be preserved during 
conversion to sentence case. Their is preparing a couple of papers. One 
journal requires StyleT with title case, another one StyleS with 
sentence case. If citeproc may determine what kind of capitalization is 
used by each style then it performs transformation to sentence case for 
StyleS, but passes strings as is for StyleT. Conversion to title case 
for StyleT is applied, but has no effect.

I can not figure out what user case is not covered and requires explicit 
setting.

Feel free to ignore this message.

Re: [PATCH] oc-csl: New custom option `org-cite-csl-sentence-case-bibtex-titles'

[Max Nikulin]

On 11/05/2024 22:33, András Simonyi wrote:
> since bibtex and biblatex requires title fields to be in title case
> but CSL assumes that they are in sentence-case, citeproc-el converts
> title fields in bib(la)tex bibliography databases into sentence-case
> before processing them except for entries with an explicit non-English
> langid value.

I am not a user of citeproc-el, so feel free to disregard my comments.

In the past I had to adjust BibTeX styles, but yesterday I was surprised 
that there are options for upper case, lower case, and sentence-style 
capitalization, but not for title-style capitalization. It seems that 
both approaches with title case and with sentence case have some 
shortcomings. Likely title case like in BibTeX requires more explicit 
hints and perhaps there are cases when available hints are not enough to 
get specific formatting. I still expect that CSL needs hints as well to 
avoid improper formatting.

Is it possible to keep title formatting from .bib files till it becomes 
known that specific style requires sentence case for particular entry 
type? I had a hope that it might alleviate the issue and to make things 
working out of the box for more users.

> I'm a bit unsure about naming the option:
> Perhaps `org-cite-csl-sentence-case-bibtex-titles-without-langid'

A variant: org-cite-csl-bibtex-title-to-sentence-case

> @@ -584,7 +599,8 @@ property in INFO."
>               (processor
>                (citeproc-create
>                 (org-cite-csl--style-file info)
> -               (citeproc-hash-itemgetter-from-any bibliography)
> +               (citeproc-hash-itemgetter-from-any
> +                bibliography (not org-cite-csl-sentence-case-bibtex-titles))
>                 (org-cite-csl--locale-getter)
>                 locale)))
>          (plist-put info :cite-citeproc-processor processor)

I am not in the context, so I may be completely wrong.

Does it means that you added one more argument to `citeproc-create' and 
that consistent Org and citeproc-el versions must be used? If so, 
wouldn't it better to pass a property list to allow newer Org to work 
with older citeproc-el or vice versa? It may be tricky to preserve 
backward-forward compatibility on this step, but it should make further 
changes easier. It may be reasonable to explicitly add version of 
"protocol" to the property list, so that citeproc-el may decide if error 
should be signaled in the case of serious version difference.

It is not clear for me why `org-cite-csl-sentence-case-bibtex-titles' is 
a part of Org, not of citeproc-el. The only thing that Org can do is to 
pass it to citeproc-el. It is not configurable per .org file and likely 
it should not be. From my point of view it might be more suitable per 
.bib file. Anyway it is almost unrelated to Org.