bug#44597: 26.3; bibtex should allow reverse sorting

["Roland Winkler" <winkler@gnu.org>]

On Sun Dec 13 2020 Francesco Potortì wrote:
> The bibtex-get-date function in my patch does not assume that the "day"
> field is present.  When it is missing, it uses the first day of the
> month, or day 30 if the LATEST argument is t.

My point is that only rarely users will have "day" fields in their
database.  Then it can happen easily that multiple entries have the
same "year" and "month" field, but no other criterion to sort these
entries.

To resolve this ambiguity, if *I* was using such a scheme, I would
want to use other fields like "author" or "title".  That's when
individual preferences enter the stage.  As we are talking about a
scheme that (I am pretty confident) will attract only a few users in
the first place, I believe it is better to let users define their
own functions for this.

> As far as I can tell, "date" and "day" are sometimes used even if not
> standard.

The "date" field is used by biblatex, which is a largely
backward-compatible successor of BibTeX.  Its "date" field is
supposed to follow iso-8601, and that's also assumed by the code
example below.  (In BibTeX mode, you can switch from old-fashioned
BibTeX to biblatex, see `bibtex-dialect'.)

I have never seen a "day" field nor any code that supports it
(nor have I seen iso-8601 "date" fields that actually included a day).

> I personally use "day" to keep track of the dates of congresses
> (for @inproceedings).

Sure, that makes a date-based sorting more straightforward and
meaningful.

> >How about instead a new customizable sorting scheme, where the
> >value of bibtex-maintain-sorted-entries is a cons pair
> >
> >  (INDEX-FUN . PREDICATE)
> >
> That would be certainly more flexible.  However I think it is at
> least equally important to provide some precooked solutions.  I
> myself would have spared some work some months ago if I had the
> 'reverse and 'reversedate methods available.

The patch below implements such a scheme (not based on a cons pair
but a list that may include a third element INIT-FUN).

To test the code, I also hacked a function `my-bibtex-index-date'
that provides an example for INDEX-FUN, see below.  It may well be
that it does not exactly match your taste, but it should be
straightforward to adapt the code.  The PREDICATE is then standard
`string-lessp' (or `string-greaterp' for reversed sorting).

Can you please test this?  Thanks.


--8<---------------cut here---------------start------------->8---
(defvar my-bibtex-month-alist
  "Alist associating month names with month numbers.  Case is ignored."
  ;; Add more (abbreviated) month names in your favorite languages.
  (let ((i 0))
    (mapcar (lambda (elt)
              (setq i (1+ i))
              (cons (cdr elt) (format "%02d" i)))
            bibtex-predefined-month-strings)))

(defun my-bibtex-index-date ()
  "Return the date of current entry as a string in yyyymmdd format."
  ;; To follow crossrefs, use instead `bibtex-autokey-get-field' with
  ;; `bibtex-autokey-use-crossref' bound non-nil.  But remember that
  ;; BibTeX expects that a cross-referenced entry occurs later in the
  ;; database files than entries cross-referencing it.  That's why
  ;; `bibtex-maintain-sorted-entries' may take the value `crossref'.
  (let* ((str (bibtex-text-in-field '("date" "year")))
         (iso8601 (and str (iso8601-valid-p str)
                       (iso8601-parse str)))
         (year (or (and iso8601
                        (let ((year (decoded-time-year iso8601)))
                          (and year (format "%04d" year))))
                   ;; BibTeX permits a year field "(about 1984)", where only
                   ;; the last four nonpunctuation characters must be numerals.
                   (and str
                        (string-match "\\([0-9][0-9][0-9][0-9]\\)[^[:alnum:]]*\\'" str)
                        (match-string 1 str))
                   "0000")) ; Change according to taste.
         (month (or (and iso8601
                         (let ((month (decoded-time-month iso8601)))
                           (and month (format "%02d" month))))
                    (let ((month (bibtex-text-in-field "month")))
                      (if month
                          (cond ((cdr (assoc-string month
                                                    my-bibtex-month-alist t)))
                                ((string-match "\\`\\([01]?\\)[0-9]\\'" month)
                                 (if (match-string 1 month)
                                     (match-string 0 month)
                                   (concat "0" (match-string 0 month)))))))
                    "00")) ; Change according to taste.
         (day (or (and iso8601
                       (let ((day (decoded-time-day iso8601)))
                         (and day (format "%02d" day))))
                  (let ((day (bibtex-text-in-field "day")))
                    (if (and day (string-match "\\`\\([03]?\\)[0-9]" day))
                        (if (match-string 1 day)
                            day
                          (concat "0" day))))
                  "00"))) ; Change according to taste.  A `day' field is not
                          ; standard BibTeX.  We could also use other fields
                          ; if `day' is not defined for an entry.
    (concat year month day)))

(setq bibtex-maintain-sorted-entries
      '(my-bibtex-index-date string-lessp))
--8<---------------cut here---------------end--------------->8---


diff --git a/etc/NEWS b/etc/NEWS
index 8746337..4a8e70e 100644
--- a/etc/NEWS
+++ b/etc/NEWS
@@ -1481,6 +1481,10 @@ completions with more information in completion prefix and suffix.
 This new option allows the user to customize how case is converted
 when unifying entries.
 
+---
+*** The user option `bibtex-maintain-sorted-entries' now permits
+user-defined sorting schemes.
+
 +++
 *** 'format-seconds' can now be used for sub-second times.
 The new optional "," parameter has been added, and
diff --git a/lisp/textmodes/bibtex.el b/lisp/textmodes/bibtex.el
index a78219e..2c7633c 100644
--- a/lisp/textmodes/bibtex.el
+++ b/lisp/textmodes/bibtex.el
@@ -204,20 +204,33 @@ bibtex-clean-entry-hook
 (defcustom bibtex-maintain-sorted-entries nil
   "If non-nil, BibTeX mode maintains all entries in sorted order.
 Allowed non-nil values are:
-plain or t   All entries are sorted alphabetically.
-crossref     All entries are sorted alphabetically unless an entry has a
+plain or t   Sort entries alphabetically by keys.
+crossref     Sort entries alphabetically by keys unless an entry has a
              crossref field.  These crossrefed entries are placed in
              alphabetical order immediately preceding the main entry.
 entry-class  The entries are divided into classes according to their
              entry type, see `bibtex-sort-entry-class'.  Within each class
-             the entries are sorted alphabetically.
+             sort entries alphabetically by keys.
+(INDEX-FUN PREDICATE)
+(INDEX-FUN PREDICATE INIT-FUN)  Sort entries using INDEX-FUN and PREDICATE.
+             Function INDEX-FUN is called for each entry with point at the
+             end of the head of the entry.  Its return values are used to
+             sort the entries using PREDICATE.  Function PREDICATE takes two
+             arguments INDEX1 and INDEX2 as returned by INDEX-FUN.
+             It should return non-nil if INDEX1 should sort before INDEX2.
+             If INIT-FUN is non-nil, it should be a function that is called
+             with no arguments to initialize the sorting.
 See also `bibtex-sort-ignore-string-entries'."
   :group 'bibtex
   :type '(choice (const nil)
+                 (const t)
                  (const plain)
                  (const crossref)
                  (const entry-class)
-                 (const t))
+                 (group :tag "Custom scheme"
+                        (function :tag "Index-Fun")
+                        (function :tag "Predicate")
+                        (option (function :tag "Init-Fun"))))
   :safe (lambda (a) (memq a '(nil t plain crossref entry-class))))
 
 (defcustom bibtex-sort-entry-class
@@ -3998,28 +4011,15 @@ bibtex-narrow-to-entry
     (narrow-to-region (bibtex-beginning-of-entry)
                       (bibtex-end-of-entry))))
 
-(defun bibtex-entry-index ()
-  "Return index of BibTeX entry head at or past position of point.
-The index is a list (KEY CROSSREF-KEY ENTRY-TYPE) that is used for sorting
-the entries of the BibTeX buffer.  CROSSREF-KEY is nil unless the value
-of `bibtex-maintain-sorted-entries' is `crossref'.  Move point to the end
-of the head of the entry found.  Return nil if no entry found."
-  (let ((case-fold-search t))
-    (if (re-search-forward bibtex-entry-maybe-empty-head nil t)
-        (let ((key (bibtex-key-in-head))
-              ;; all entry types should be downcase (for ease of comparison)
-              (entry-type (downcase (bibtex-type-in-head))))
-          ;; Don't search CROSSREF-KEY if we don't need it.
-          (if (eq bibtex-maintain-sorted-entries 'crossref)
-              (let ((bounds (bibtex-search-forward-field
-                             "\\(OPT\\)?crossref" t)))
-                (list key
-                      (if bounds (bibtex-text-in-field-bounds bounds t))
-                      entry-type))
-            (list key nil entry-type))))))
-
-(defun bibtex-init-sort-entry-class-alist ()
-  "Initialize `bibtex-sort-entry-class-alist' (buffer-local)."
+(define-obsolete-function-alias 'bibtex-init-sort-entry-class-alist
+  #'bibtex-init-sort "28.1")
+(defun bibtex-init-sort (&optional parse)
+  "Initialize sorting of BibTeX entries.
+If PARSE is non-nil, also parse BibTeX keys."
+  (if (or parse
+          (and (eq bibtex-maintain-sorted-entries 'crossref)
+               (functionp bibtex-reference-keys)))
+      (bibtex-parse-keys))
   (unless (local-variable-p 'bibtex-sort-entry-class-alist)
     (setq-local bibtex-sort-entry-class-alist
                 (let ((i -1) alist)
@@ -4029,7 +4029,36 @@ bibtex-init-sort-entry-class-alist
                       ;; All entry types should be downcase (for ease of comparison).
                       (push (cons (if (stringp entry) (downcase entry) entry) i)
                             alist)))
-                  alist))))
+                  alist)))
+  ;; Custom sorting scheme
+  (if (and (consp bibtex-maintain-sorted-entries)
+           (nth 2 bibtex-maintain-sorted-entries))
+      (funcall (nth 2 bibtex-maintain-sorted-entries))))
+
+(defun bibtex-entry-index ()
+  "Return index of BibTeX entry head at or past position of point.
+The index is a list (KEY CROSSREF-KEY ENTRY-TYPE) that is used for sorting
+the entries of the BibTeX buffer.  CROSSREF-KEY is nil unless the value of
+`bibtex-maintain-sorted-entries' is `crossref'.
+If `bibtex-maintain-sorted-entries' is (INDEX-FUN ...), the index is the return
+value of INDEX-FUN.  Return nil if no entry found.
+Move point to the end of the head of the entry found."
+  (let ((case-fold-search t))
+    (if (re-search-forward bibtex-entry-maybe-empty-head nil t)
+        (if (consp bibtex-maintain-sorted-entries)
+            ;; Custom sorting scheme
+            (funcall (car bibtex-maintain-sorted-entries))
+          (let ((key (bibtex-key-in-head))
+                ;; ENTRY-TYPE should be downcase (for ease of comparison)
+                (entry-type (downcase (bibtex-type-in-head)))
+                bounds)
+            (list key
+                  ;; Don't search CROSSREF-KEY if we don't need it.
+                  (and (eq bibtex-maintain-sorted-entries 'crossref)
+                       (setq bounds (bibtex-search-forward-field
+                                      "\\(OPT\\)?crossref" t))
+                       (bibtex-text-in-field-bounds bounds t))
+                  entry-type))))))
 
 (defun bibtex-lessp (index1 index2)
   "Predicate for sorting BibTeX entries with indices INDEX1 and INDEX2.
@@ -4038,6 +4067,8 @@ bibtex-lessp
 If its value is nil use plain sorting."
   (cond ((not index1) (not index2)) ; indices can be nil
         ((not index2) nil)
+        ((consp bibtex-maintain-sorted-entries)
+         (funcall (cadr bibtex-maintain-sorted-entries) index1 index2))
         ((eq bibtex-maintain-sorted-entries 'crossref)
          ;; CROSSREF-KEY may be nil or it can point to an entry
          ;; in another BibTeX file.  In both cases we ignore CROSSREF-KEY.
@@ -4074,10 +4105,7 @@ bibtex-sort-buffer
 are ignored."
   (interactive)
   (bibtex-beginning-of-first-entry)     ; Needed by `sort-subr'
-  (bibtex-init-sort-entry-class-alist)  ; Needed by `bibtex-lessp'.
-  (if (and (eq bibtex-maintain-sorted-entries 'crossref)
-           (functionp bibtex-reference-keys))
-      (bibtex-parse-keys))              ; Needed by `bibtex-lessp'.
+  (bibtex-init-sort)                    ; Needed by `bibtex-lessp'.
   (sort-subr nil
              'bibtex-skip-to-valid-entry   ; NEXTREC function
              'bibtex-end-of-entry          ; ENDREC function
@@ -4228,10 +4256,7 @@ bibtex-prepare-new-entry
 search to look for place for KEY.  This requires that buffer is sorted,
 see `bibtex-validate'.
 Return t if preparation was successful or nil if entry KEY already exists."
-  (bibtex-init-sort-entry-class-alist)  ; Needed by `bibtex-lessp'.
-  (if (and (eq bibtex-maintain-sorted-entries 'crossref)
-           (functionp bibtex-reference-keys))
-      (bibtex-parse-keys))              ; Needed by `bibtex-lessp'.
+  (bibtex-init-sort)  ; Needed by `bibtex-lessp'.
   (let ((key (nth 0 index))
         key-exist)
     (cond ((or (null key)
@@ -4322,9 +4347,7 @@ bibtex-validate
             (setq syntax-error t)
 
           ;; Check for duplicate keys and correct sort order
-          (bibtex-init-sort-entry-class-alist)  ; Needed by `bibtex-lessp'.
-          (bibtex-parse-keys) ; Possibly needed by `bibtex-lessp'.
-                              ; Always needed by subsequent global key check.
+          (bibtex-init-sort t) ; Needed by `bibtex-lessp' and global key check.
           (let (previous current key-list)
             (bibtex-progress-message "Checking for duplicate keys")
             (bibtex-map-entries