bug#44976: 28.0.50; bibtex.el does not consider biblatex field name aliases

bug#44976: 28.0.50; bibtex.el does not consider biblatex field name aliases

[leonard]

bibtex.el does not consider biblatex aliases correctly. The biblatex package
documentation specifies in Section 2.2.5 beginning on page 32 a series of
Field Aliases.

These include for example the alias "journal" for "journaltitle" which helps
biblatex to preserve backwards compatibility with bibtex. This causes bibtex.el
to throw error messages of the style "Mandatory field `%s' is missing" despite
the alias field being present.

To reproduce run `emacs -Q`. Then switch to `bibtex-mode` and set
`(bibtex-set-dialect 'biblatex t)`. Finally paste

  @article{doe2020lorem,
    author = {John Doe},
    title = {Lorem Ipsum},
    date = {2020},
    journal = {Communications of the ACM}
  }

and navigate the cursor into the @article entry. Then execute `M-x
bibtex-clean-entry` and observe "Mandatory field ‘journaltitle’ is missing"
error message.

In GNU Emacs 28.0.50 (build 1, x86_64-pc-linux-gnu, GTK+ Version 3.24.20, cairo version 1.16.0)
 of 2020-11-30 built locally
Repository revision: 21104e6808a4496afb8163d92c6fb4d59e3010b7
Repository branch: feature/native-comp
Windowing system distributor 'The X.Org Foundation', version 11.0.12008000
System Description: Ubuntu 20.04.1 LTS

Configured using:
 'configure --with-cairo --with-modules --with-nativecomp'

Configured features:
XPM JPEG TIFF GIF PNG RSVG CAIRO SOUND GPM DBUS GSETTINGS GLIB NOTIFY
INOTIFY ACL LIBSELINUX GNUTLS LIBXML2 FREETYPE HARFBUZZ M17N_FLT LIBOTF
ZLIB TOOLKIT_SCROLL_BARS GTK3 X11 XDBE XIM MODULES NATIVE_COMP THREADS
LIBSYSTEMD PDUMPER LCMS2

Important settings:
  value of $LANG: en_US.UTF-8
  value of $XMODIFIERS: @im=ibus
  locale-coding-system: utf-8-unix

Major mode: BibTeX

Minor modes in effect:
  tooltip-mode: t
  global-eldoc-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  line-number-mode: t
  transient-mark-mode: t

Load-path shadows:
None found.

Features:
(shadow sort mail-extr emacsbug message rmc puny dired dired-loaddefs
rfc822 mml mml-sec epa derived epg epg-config gnus-util rmail
rmail-loaddefs auth-source eieio eieio-core eieio-loaddefs
password-cache json map text-property-search mm-decode mm-bodies
mm-encode mail-parse rfc2231 mailabbrev gmm-utils mailheader sendmail
comp comp-cstr warnings rx cl-seq cl-macs cl-extra seq byte-opt gv
bytecomp byte-compile cconv rfc2047 rfc2045 ietf-drums mm-util
mail-prsvr mail-utils help-fns radix-tree cl-print debug backtrace
help-mode find-func bibtex easymenu time-date subr-x cl-loaddefs cl-lib
tooltip eldoc electric uniquify ediff-hook vc-hooks lisp-float-type
mwheel term/x-win x-win term/common-win x-dnd tool-bar dnd fontset image
regexp-opt fringe tabulated-list replace newcomment text-mode elisp-mode
lisp-mode prog-mode register page tab-bar menu-bar rfn-eshadow isearch
timer select scroll-bar mouse jit-lock font-lock syntax facemenu
font-core term/tty-colors frame minibuffer cl-generic cham georgian
utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao korean
japanese eucjp-ms cp51932 hebrew greek romanian slovak czech european
ethiopic indian cyrillic chinese composite charscript charprop
case-table epa-hook jka-cmpr-hook help simple abbrev obarray
cl-preloaded nadvice button loaddefs faces cus-face pcase macroexp files
window text-properties overlay sha1 md5 base64 format env code-pages
mule custom widget hashtable-print-readable backquote threads dbusbind
inotify lcms2 dynamic-setting system-font-setting font-render-setting
cairo move-toolbar gtk x-toolkit x multi-tty make-network-process
nativecomp emacs)

Memory information:
((conses 16 123914 7697)
 (symbols 48 8538 1)
 (strings 32 27224 3373)
 (string-bytes 1 883468)
 (vectors 16 16711)
 (vector-slots 8 323004 11886)
 (floats 8 29 34)
 (intervals 56 11155 0)
 (buffers 984 14))

bug#44976: 28.0.50; bibtex.el does not consider biblatex field name aliases

[Basil L. Contovounesios]

severity 44976 wishlist
quit

leonard@lausen.nl writes:

> bibtex.el does not consider biblatex aliases correctly. The biblatex package
> documentation specifies in Section 2.2.5 beginning on page 32 a series of
> Field Aliases.
>
> These include for example the alias "journal" for "journaltitle" which helps
> biblatex to preserve backwards compatibility with bibtex. This causes bibtex.el
> to throw error messages of the style "Mandatory field `%s' is missing" despite
> the alias field being present.

I, for one, would be in favour of supporting these aliases.

Roland, are there any reasons not to?

Thanks,

-- 
Basil

bug#44976: 28.0.50; bibtex.el does not consider biblatex field name aliases

[Roland Winkler]

On Tue Dec 1 2020 Basil L. Contovounesios wrote:
> > bibtex.el does not consider biblatex aliases correctly. The biblatex package
> > documentation specifies in Section 2.2.5 beginning on page 32 a series of
> > Field Aliases.
> >
> > These include for example the alias "journal" for "journaltitle" which helps
> > biblatex to preserve backwards compatibility with bibtex. This causes bibtex.el
> > to throw error messages of the style "Mandatory field `%s' is missing" despite
> > the alias field being present.
> 
> I, for one, would be in favour of supporting these aliases.
> 
> Roland, are there any reasons not to?

Years ago, I added biblatex support to bibtex.el.  However, I myself
only use old-fashioned BibTeX, so I am not an expert of how biblatex
is supposed to work, and it can well be that the current code can be
improved in one or the other way.

Do I understand correctly that, from biblatex's perspective,
"journal" and "journaltitle" are alternatives similar to "year" and
"date"?  The code in bibtex.el already includes the machinery to
support such alternatives.  So all that's needed for this is to
replace in bibtex-biblatex-entry-alist the elements

  ("journaltitle")

with

  ("journaltitle" nil nil 17) ("journal" nil nil 17)

with 17 replaced by the lowest unused number for the alternatives of
this entry type, see "ALTERNATIVES" in the docstring of
bibtex-BibTeX-entry-alist.

Or am I missing something?

When bibtex-entry inserts a template for a new entry, this includes
the different alternatives with prefix "ALT".  It could be useful to
distingiush between alternatives that are listed in bibtex-entry's
templates and "hidden" alternatives that are only silently accepted
in the background because of biblatex's backward compatibility with
the BibTeX format.  For example, the ALTERNATIVE number could be
positive or negative.  Then, only if this number is positive, all
alternatives are all included in the entry template with the ALT prefix.
Otherwise, only the first alternative in bibtex-biblatex-entry-alist
is printed without the prefix.  Such a feature could also make sense
for "year" and "date".

bug#44976: 28.0.50; bibtex.el does not consider biblatex field name aliases

[leonard]

> Do I understand correctly that, from biblatex's perspective,
> "journal" and "journaltitle" are alternatives similar to "year" and
> "date"? The code in bibtex.el already includes the machinery to
> support such alternatives.

I think you understand correctly. The biblatex doc specifies:

"The aliases listed in this section are provided for backwards compatibility with traditional
BibTeX and other applications based on traditional BibTeX styles. Note that these aliases are
immediately resolved as the bibfile is processed. All bibliography and citation styles must use the
names of the fields they point to, not the alias. In bibfiles, you may use either the alias or the
field name but not both at the same time."
https://mirrors.concertpass.com/tex-archive/macros/latex/contrib/biblatex/doc/biblatex.pdf#subsubsection.2.2.5

> So all that's needed for this is to replace in bibtex-biblatex-entry-alist the elements
>
> ("journaltitle")
> 
> with
> 
> ("journaltitle" nil nil 17) ("journal" nil nil 17)
> 
> with 17 replaced by the lowest unused number for the alternatives of
> this entry type, see "ALTERNATIVES" in the docstring of
> bibtex-BibTeX-entry-alist.

Great!

> Or am I missing something?
> 
> When bibtex-entry inserts a template for a new entry, this includes
> the different alternatives with prefix "ALT". It could be useful to
> distingiush between alternatives that are listed in bibtex-entry's
> templates and "hidden" alternatives that are only silently accepted
> in the background because of biblatex's backward compatibility with
> the BibTeX format. For example, the ALTERNATIVE number could be
> positive or negative. Then, only if this number is positive, all
> alternatives are all included in the entry template with the ALT prefix.
> Otherwise, only the first alternative in bibtex-biblatex-entry-alist
> is printed without the prefix. Such a feature could also make sense
> for "year" and "date".

That would be a nice feature as well.

Do you have time to commit a fix for the aliases and possibly add the negative/positive distinction of ALTERNATIVEs?

bug#44976: 28.0.50; bibtex.el does not consider biblatex field name aliases

[Roland Winkler]

On Thu Dec 3 2020 leonard@lausen.nl wrote:
> Do you have time to commit a fix for the aliases and possibly add
> the negative/positive distinction of ALTERNATIVEs?

Thanks for the clarifications.  I should have a draft of this soon.

bug#44976: 28.0.50; bibtex.el does not consider biblatex field name aliases

[Roland Winkler]

On Sun Dec 13 2020 Roland Winkler wrote:
> On Thu Dec 3 2020 leonard@lausen.nl wrote:
> > Do you have time to commit a fix for the aliases and possibly add
> > the negative/positive distinction of ALTERNATIVEs?
> 
> Thanks for the clarifications.  I should have a draft of this soon.

I have installed a patch for this
(commit aac17c9dca21462df57367123301b7c940f9243a).

In the end, the patch was significantly more substantial than what I
had expected to make it work with the code for validating entries.
Please, test it and let me know how it works for you.

Please, see also the "FIXME" in the new function
bibtex--skip-field-aliases.  I am not yet entirely happy with the
new code.  But I do not know either how it works in real life.
Maybe my concerns are not justified.  (As I said, I only use
old-fashioned BibTeX.  Therefore, I am not familar with what
biblatex users might like to do.)

Also, I have not yet decided how the new behavior should possibly be
advertised in etc/NEWS.  As I said in the FIXME, most of the changes
are under the hood.  In that sense, I am not sure how much this
should possibly be advertised.

bug#44976: 28.0.50; bibtex.el does not consider biblatex field name aliases

[Glenn Morris]

"Roland Winkler" wrote:

> I have installed a patch for this
> (commit aac17c9dca21462df57367123301b7c940f9243a).

After this change, bibtex-biblatex-entry-alist fails to match its custom
type. See test failure at
https://hydra.nixos.org/build/134061256

bug#44976: 28.0.50; bibtex.el does not consider biblatex field name aliases

[Roland Winkler]

On Sat Jan 2 2021 Glenn Morris wrote:
> "Roland Winkler" wrote:
> 
> > I have installed a patch for this
> > (commit aac17c9dca21462df57367123301b7c940f9243a).
> 
> After this change, bibtex-biblatex-entry-alist fails to match its custom
> type. See test failure at
> https://hydra.nixos.org/build/134061256

Thanks, should be fixed now (commit dde3269633550debb8b13cdc77136fe638c8e1fc).

Kind of related:

With emacs 25.1, it takes me about 2s to get the customization
buffer for bibtex-biblatex-entry-alist.  But emacs 27.1 needs about
20s.  Is this a known problem?

bug#44976: 28.0.50; bibtex.el does not consider biblatex field name aliases

[Lars Ingebrigtsen]

"Roland Winkler" <winkler@gnu.org> writes:

> Thanks, should be fixed now (commit dde3269633550debb8b13cdc77136fe638c8e1fc).

If I understand correctly, the reported issues here were fixed, so I'm
closing this bug report.

> Kind of related:
>
> With emacs 25.1, it takes me about 2s to get the customization
> buffer for bibtex-biblatex-entry-alist.  But emacs 27.1 needs about
> 20s.  Is this a known problem?

Nope.  I'll open a new bug report for this.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no