bug#44597: 26.3; bibtex should allow reverse sorting

bug#44597: 26.3; bibtex should allow reverse sorting

[Francesco Potortì]

I wish bibtex mode would allow for reverse sorting of entries.

Some years ago I used to obtain that behaviour by advising the sort-subr
functions when called inside a bibtex-mode buffer.  No more.

The modern bibtex.el sorts an entry when bibtex-clean-entry is called
when bibtex-maintain-sorted-entries is t without calling sort-subr, but
relying only on bibtex-lessp, which is not easy to advice.

I have numbered labels, and I want them to stay in reverse order as they
are, but now I can get that behaviour only by setting
bibtex-maintain-sorted-entries to nil

Please add some way to reverse the order of entries

--
fp

bug#44597: 26.3; bibtex should allow reverse sorting

[Lars Ingebrigtsen]

Francesco Potortì <pot@gnu.org> writes:

> I wish bibtex mode would allow for reverse sorting of entries.
>
> Some years ago I used to obtain that behaviour by advising the sort-subr
> functions when called inside a bibtex-mode buffer.  No more.
>
> The modern bibtex.el sorts an entry when bibtex-clean-entry is called
> when bibtex-maintain-sorted-entries is t without calling sort-subr, but
> relying only on bibtex-lessp, which is not easy to advice.

I haven't looked at the code closely, but wouldn't advising bibtex-lessp
to just swap the parameters work?

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#44597: 26.3; bibtex should allow reverse sorting

[Francesco Potortì]

>> I wish bibtex mode would allow for reverse sorting of entries.
>>
>> Some years ago I used to obtain that behaviour by advising the sort-subr
>> functions when called inside a bibtex-mode buffer.  No more.
>>
>> The modern bibtex.el sorts an entry when bibtex-clean-entry is called
>> when bibtex-maintain-sorted-entries is t without calling sort-subr, but
>> relying only on bibtex-lessp, which is not easy to advice.
>
>I haven't looked at the code closely, but wouldn't advising bibtex-lessp
>to just swap the parameters work?

Yes.  In fact, after writing my bug report I looked at the code again
and I did just that :)

However, I think that reversing the sorting order is just the most
common of various criteria that one would like to use for sorting.
Recently I had a reason for getting one hundred bibtex entries sorted by
date in reverse order, a task that would take a long time by hand,
especially in front of frequent modifications of the list of entries to
be sorted.

Fortunately I was able to use bibtex2html to do the job, but the most
natural thing would have been to use bibtex.el with a proper sorting
function.  Ideally, bibtex.el should provide a hook where one can
install any function for sorting.  The hook should get some arguments
(the type of entry, the name of entry, the date, a bibtex custom field
and maybe others).  That would make bibtex.el more generally flexible.

bug#44597: 26.3; bibtex should allow reverse sorting

[Lars Ingebrigtsen]

Francesco Potortì <pot@gnu.org> writes:

> Ideally, bibtex.el should provide a hook where one can install any
> function for sorting.  The hook should get some arguments (the type of
> entry, the name of entry, the date, a bibtex custom field and maybe
> others).  That would make bibtex.el more generally flexible.

Yes, I think introducing a variable to control the sorting would be a
good idea.  It'd default to `bibtex-lessp' (or rather, something that
does the same), but (as you say) have an input that makes implementing
other sorting functions easier.

Patches welcome.  :-)

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#44597: 26.3; bibtex should allow reverse sorting

[Francesco Potortì]

>> Ideally, bibtex.el should provide a hook where one can install any
>> function for sorting.  The hook should get some arguments (the type of
>> entry, the name of entry, the date, a bibtex custom field and maybe
>> others).  That would make bibtex.el more generally flexible.
>
>Yes, I think introducing a variable to control the sorting would be a
>good idea.  It'd default to `bibtex-lessp' (or rather, something that
>does the same), but (as you say) have an input that makes implementing
>other sorting functions easier.
>
>Patches welcome.  :-)

This is a patch that improves things, in my opinion.  This is not the
ideal situation I mentioned above, but "il meglio è il nemico del bene"
(best is enemy of good).  So here you are, this one satisfies my needs
and is hoefully a step towards a more general solution.


diff -pub /home/pot/elisp/bibtex-trunk.el /home/pot/elisp/bibtex.el
--- /home/pot/elisp/bibtex-trunk.el	2020-12-03 18:52:11.000000000 +0100
+++ /home/pot/elisp/bibtex.el	2020-12-06 17:02:06.000000000 +0100
@@ -202,7 +202,10 @@ narrowed to just the entry."
 (defcustom bibtex-maintain-sorted-entries nil
   "If non-nil, BibTeX mode maintains all entries in sorted order.
 Allowed non-nil values are:
-plain or t   All entries are sorted alphabetically.
+plain or t   Entries are sorted alphabetically.
+reverse      Entries are sorted in reverse alphabetic order.
+date         Entries are sorted by date.
+reversedate  Entries are reverse sorted by date, starting with newest.
 crossref     All entries are sorted alphabetically unless an entry has a
              crossref field.  These crossrefed entries are placed in
              alphabetical order immediately preceding the main entry.
@@ -213,10 +216,13 @@ See also `bibtex-sort-ignore-string-entr
   :group 'bibtex
   :type '(choice (const nil)
                  (const plain)
+		 (const reverse)
+		 (const date)
+		 (const reversedate)
                  (const crossref)
                  (const entry-class)
                  (const t))
-  :safe (lambda (a) (memq a '(nil t plain crossref entry-class))))
+  :safe (lambda (a) (memq a '(nil t plain reverse date reversedate crossref entry-class))))
 
 (defcustom bibtex-sort-entry-class
   '(("String")
@@ -3993,23 +3999,25 @@ If mark is active count entries in regio
 
 (defun bibtex-entry-index ()
   "Return index of BibTeX entry head at or past position of point.
-The index is a list (KEY CROSSREF-KEY ENTRY-TYPE) that is used for sorting
-the entries of the BibTeX buffer.  CROSSREF-KEY is nil unless the value
-of `bibtex-maintain-sorted-entries' is `crossref'.  Move point to the end
-of the head of the entry found.  Return nil if no entry found."
+The index is a list (KEY CROSSREF-KEY ENTRY-TYPE DATE) that is
+used for sorting the entries of the BibTeX buffer.  CROSSREF-KEY
+is nil unless the value of `bibtex-maintain-sorted-entries' is
+`crossref'.  Move point to the end of the head of the entry
+found.  Return nil if no entry found."
   (let ((case-fold-search t))
     (if (re-search-forward bibtex-entry-maybe-empty-head nil t)
         (let ((key (bibtex-key-in-head))
               ;; all entry types should be downcase (for ease of comparison)
-              (entry-type (downcase (bibtex-type-in-head))))
+	      (crossref-key nil)
+              (entry-type (downcase (bibtex-type-in-head)))
+	      (date (bibtex-get-date)))
           ;; Don't search CROSSREF-KEY if we don't need it.
-          (if (eq bibtex-maintain-sorted-entries 'crossref)
+          (when (eq bibtex-maintain-sorted-entries 'crossref)
               (let ((bounds (bibtex-search-forward-field
                              "\\(OPT\\)?crossref" t)))
-                (list key
-                      (if bounds (bibtex-text-in-field-bounds bounds t))
-                      entry-type))
-            (list key nil entry-type))))))
+              (when bounds
+		(setq crossref-key (bibtex-text-in-field-bounds bounds t)))))
+          (list key crossref-key entry-type date)))))
 
 (defun bibtex-init-sort-entry-class-alist ()
   "Initialize `bibtex-sort-entry-class-alist' (buffer-local)."
@@ -4024,6 +4032,28 @@ of the head of the entry found.  Return
                      alist)))
            alist))))
 
+(defun bibtex-get-date (&optional latest)
+  "Return date of current entry as a string in yyyy-mm-dd format.
+If any part of date is missing, use the earliest possible missing
+value, unless LATEST is not nil, in which case use the latest possible
+missing value."
+  (let ((date (bibtex-autokey-get-field "date")))
+    (if (string> date "")
+	date
+      (let ((year (bibtex-autokey-get-field "year"))
+	    (month (bibtex-autokey-get-field
+		    "month" '(("^jan.*". "01") ("^feb.*". "02") ("^mar.*". "03")
+			      ("^apr.*". "04") ("^may.*". "05") ("^jun.*". "06")
+			      ("^jul.*". "07") ("^aug.*". "08") ("^sep.*". "09")
+			      ("^oct.*". "10") ("^nov.*". "11") ("^dec.*". "12"))))
+	    (day (bibtex-autokey-get-field "days" '(("-.*" . "")))))
+	(when (string= year "") (setq year (if latest "9999" "1900")))
+	(when (string= month "") (setq month (if latest "12" "01")))
+	(when (string= day "") (setq day (if latest "31" "01")))
+	(concat year "-" month "-" (if (= (string-width day) 1)
+				       (concat "0" day)
+				     (substring day 0 2)))))))
+
 (defun bibtex-lessp (index1 index2)
   "Predicate for sorting BibTeX entries with indices INDEX1 and INDEX2.
 Each index is a list (KEY CROSSREF-KEY ENTRY-TYPE).
@@ -4056,6 +4086,12 @@ If its value is nil use plain sorting."
            (or (< n1 n2)
                (and (= n1 n2)
                     (string-lessp (car index1) (car index2))))))
+	((eq bibtex-maintain-sorted-entries 'reversedate)
+	 (string-lessp (nth 3 index2) (nth 3 index1)))
+	((eq bibtex-maintain-sorted-entries 'date)
+	 (string-lessp (nth 3 index1) (nth 3 index2)))
+	((eq bibtex-maintain-sorted-entries 'reverse)
+	 (string-lessp (car index2) (car index1)))
         (t ; (eq bibtex-maintain-sorted-entries 'plain)
          (string-lessp (car index1) (car index2)))))

bug#44597: 26.3; bibtex should allow reverse sorting

[Lars Ingebrigtsen]

Francesco Potortì <pot@gnu.org> writes:

>>Yes, I think introducing a variable to control the sorting would be a
>>good idea.  It'd default to `bibtex-lessp' (or rather, something that
>>does the same), but (as you say) have an input that makes implementing
>>other sorting functions easier.
>>
>>Patches welcome.  :-)
>
> This is a patch that improves things, in my opinion.  This is not the
> ideal situation I mentioned above, but "il meglio è il nemico del bene"
> (best is enemy of good).  So here you are, this one satisfies my needs
> and is hoefully a step towards a more general solution.

[...]

>  Allowed non-nil values are:
> -plain or t   All entries are sorted alphabetically.
> +plain or t   Entries are sorted alphabetically.
> +reverse      Entries are sorted in reverse alphabetic order.
> +date         Entries are sorted by date.
> +reversedate  Entries are reverse sorted by date, starting with newest.

Thanks; this looks like a good feature to me.  I've added Roland to the
Cc's; I'm guessing he'll have some comments.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#44597: 26.3; bibtex should allow reverse sorting

[Francesco Potortì]

Ah, I forgot.

I did not include it in the patch, but I think that
bibtex-autokey-get-field should be renamed to bibtex-get-field

--
fp

bug#44597: 26.3; bibtex should allow reverse sorting

[Roland Winkler]

On Mon Dec 7 2020 Francesco Potortì wrote:
> Ah, I forgot.
> 
> I did not include it in the patch, but I think that
> bibtex-autokey-get-field should be renamed to bibtex-get-field

I'll look into this patch.

(I believe bibtex-autokey-get-field need not be renamed.
But it will be better to call instead the low-level function
bibtex-text-in-field.)

bug#44597: 26.3; bibtex should allow reverse sorting

[Francesco Potortì]

>>  Allowed non-nil values are:
>> -plain or t   All entries are sorted alphabetically.
>> +plain or t   Entries are sorted alphabetically.
>> +reverse      Entries are sorted in reverse alphabetic order.
>> +date         Entries are sorted by date.
>> +reversedate  Entries are reverse sorted by date, starting with newest.

Thinking again, maybe the reverse flag should be orthogonal to the
rest, to be applied separately?

bug#44597: 26.3; bibtex should allow reverse sorting

[Lars Ingebrigtsen]

Francesco Potortì <pot@gnu.org> writes:

>>>  Allowed non-nil values are:
>>> -plain or t   All entries are sorted alphabetically.
>>> +plain or t   Entries are sorted alphabetically.
>>> +reverse      Entries are sorted in reverse alphabetic order.
>>> +date         Entries are sorted by date.
>>> +reversedate  Entries are reverse sorted by date, starting with newest.
>
> Thinking again, maybe the reverse flag should be orthogonal to the
> rest, to be applied separately?

I think a single setting should be sufficient here.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#44597: 26.3; bibtex should allow reverse sorting

[Roland Winkler]

On Mon Dec 7 2020 Lars Ingebrigtsen wrote:
> I've added Roland to the Cc's; I'm guessing he'll have some
> comments.

I am not sure the proposed patch is the best way to go.  Suppose a
BibTeX file contains just several dozen entries per year of
publication.  Then sorting by date becomes meaningful only if the
date for each entry includes not only year and month, but also the
day.  I assume that's why the proposed patch also uses a `day'
field.  But such a field is not standard for BibTeX entries, and I
am not aware of any "default" mechanisms to support it.  I guess you
maintain such a field by hand in your BibTeX database.  I am not
sure it makes sense for BibTeX mode to go in such a direction with
hard-coded hand-crafted solutions, where likely the next user wants
to do things just a little different.

How about instead a new customizable sorting scheme, where the value
of bibtex-maintain-sorted-entries is a cons pair

  (INDEX-FUN . PREDICATE)

where INDEX-FUN is a user-defined function that replaces
bibtex-entry-index, and PREDICATE is a user-defined function that
replaces the default predicate bibtex-lessp.  Then users can define
custom sorting schemes to their liking, where PREDICATE sorts the
entries based on the return values of INDEX-FUN.

bug#44597: 26.3; bibtex should allow reverse sorting

[Francesco Potortì]

>I am not sure the proposed patch is the best way to go.  Suppose a
>BibTeX file contains just several dozen entries per year of
>publication.  Then sorting by date becomes meaningful only if the
>date for each entry includes not only year and month, but also the
>day.

The bibtex-get-date function in my patch does not assume that the "day"
field is present.  When it is missing, it uses the first day of the
month, or day 30 if the LATEST argument is t.

By the way, there is an error in my patch: it shuld be "day", not
"days".

>I assume that's why the proposed patch also uses a `day'
>field.  But such a field is not standard for BibTeX entries, and I
>am not aware of any "default" mechanisms to support it.

As far as I can tell, "date" and "day" are sometimes used even if not
standard.  Anyway, if any of "date" or "day" are present, the patch will
use them, else it will use what's available.

I personally use "day" to keep track of the dates of congresses (for
@inproceedings).

>How about instead a new customizable sorting scheme, where the value
>of bibtex-maintain-sorted-entries is a cons pair
>
>  (INDEX-FUN . PREDICATE)
>
>where INDEX-FUN is a user-defined function that replaces
>bibtex-entry-index, and PREDICATE is a user-defined function that
>replaces the default predicate bibtex-lessp.  Then users can define
>custom sorting schemes to their liking, where PREDICATE sorts the
>entries based on the return values of INDEX-FUN.

That would be certainly more flexible.  However I think it is at least
equally important to provide some precooked solutions.  I myself would
have spared some work some months ago if I had the 'reverse and
'reversedate methods available.

bug#44597: 26.3; bibtex should allow reverse sorting

[Roland Winkler]

On Sun Dec 13 2020 Francesco Potortì wrote:
> The bibtex-get-date function in my patch does not assume that the "day"
> field is present.  When it is missing, it uses the first day of the
> month, or day 30 if the LATEST argument is t.

My point is that only rarely users will have "day" fields in their
database.  Then it can happen easily that multiple entries have the
same "year" and "month" field, but no other criterion to sort these
entries.

To resolve this ambiguity, if *I* was using such a scheme, I would
want to use other fields like "author" or "title".  That's when
individual preferences enter the stage.  As we are talking about a
scheme that (I am pretty confident) will attract only a few users in
the first place, I believe it is better to let users define their
own functions for this.

> As far as I can tell, "date" and "day" are sometimes used even if not
> standard.

The "date" field is used by biblatex, which is a largely
backward-compatible successor of BibTeX.  Its "date" field is
supposed to follow iso-8601, and that's also assumed by the code
example below.  (In BibTeX mode, you can switch from old-fashioned
BibTeX to biblatex, see `bibtex-dialect'.)

I have never seen a "day" field nor any code that supports it
(nor have I seen iso-8601 "date" fields that actually included a day).

> I personally use "day" to keep track of the dates of congresses
> (for @inproceedings).

Sure, that makes a date-based sorting more straightforward and
meaningful.

> >How about instead a new customizable sorting scheme, where the
> >value of bibtex-maintain-sorted-entries is a cons pair
> >
> >  (INDEX-FUN . PREDICATE)
> >
> That would be certainly more flexible.  However I think it is at
> least equally important to provide some precooked solutions.  I
> myself would have spared some work some months ago if I had the
> 'reverse and 'reversedate methods available.

The patch below implements such a scheme (not based on a cons pair
but a list that may include a third element INIT-FUN).

To test the code, I also hacked a function `my-bibtex-index-date'
that provides an example for INDEX-FUN, see below.  It may well be
that it does not exactly match your taste, but it should be
straightforward to adapt the code.  The PREDICATE is then standard
`string-lessp' (or `string-greaterp' for reversed sorting).

Can you please test this?  Thanks.


--8<---------------cut here---------------start------------->8---
(defvar my-bibtex-month-alist
  "Alist associating month names with month numbers.  Case is ignored."
  ;; Add more (abbreviated) month names in your favorite languages.
  (let ((i 0))
    (mapcar (lambda (elt)
              (setq i (1+ i))
              (cons (cdr elt) (format "%02d" i)))
            bibtex-predefined-month-strings)))

(defun my-bibtex-index-date ()
  "Return the date of current entry as a string in yyyymmdd format."
  ;; To follow crossrefs, use instead `bibtex-autokey-get-field' with
  ;; `bibtex-autokey-use-crossref' bound non-nil.  But remember that
  ;; BibTeX expects that a cross-referenced entry occurs later in the
  ;; database files than entries cross-referencing it.  That's why
  ;; `bibtex-maintain-sorted-entries' may take the value `crossref'.
  (let* ((str (bibtex-text-in-field '("date" "year")))
         (iso8601 (and str (iso8601-valid-p str)
                       (iso8601-parse str)))
         (year (or (and iso8601
                        (let ((year (decoded-time-year iso8601)))
                          (and year (format "%04d" year))))
                   ;; BibTeX permits a year field "(about 1984)", where only
                   ;; the last four nonpunctuation characters must be numerals.
                   (and str
                        (string-match "\\([0-9][0-9][0-9][0-9]\\)[^[:alnum:]]*\\'" str)
                        (match-string 1 str))
                   "0000")) ; Change according to taste.
         (month (or (and iso8601
                         (let ((month (decoded-time-month iso8601)))
                           (and month (format "%02d" month))))
                    (let ((month (bibtex-text-in-field "month")))
                      (if month
                          (cond ((cdr (assoc-string month
                                                    my-bibtex-month-alist t)))
                                ((string-match "\\`\\([01]?\\)[0-9]\\'" month)
                                 (if (match-string 1 month)
                                     (match-string 0 month)
                                   (concat "0" (match-string 0 month)))))))
                    "00")) ; Change according to taste.
         (day (or (and iso8601
                       (let ((day (decoded-time-day iso8601)))
                         (and day (format "%02d" day))))
                  (let ((day (bibtex-text-in-field "day")))
                    (if (and day (string-match "\\`\\([03]?\\)[0-9]" day))
                        (if (match-string 1 day)
                            day
                          (concat "0" day))))
                  "00"))) ; Change according to taste.  A `day' field is not
                          ; standard BibTeX.  We could also use other fields
                          ; if `day' is not defined for an entry.
    (concat year month day)))

(setq bibtex-maintain-sorted-entries
      '(my-bibtex-index-date string-lessp))
--8<---------------cut here---------------end--------------->8---


diff --git a/etc/NEWS b/etc/NEWS
index 8746337..4a8e70e 100644
--- a/etc/NEWS
+++ b/etc/NEWS
@@ -1481,6 +1481,10 @@ completions with more information in completion prefix and suffix.
 This new option allows the user to customize how case is converted
 when unifying entries.
 
+---
+*** The user option `bibtex-maintain-sorted-entries' now permits
+user-defined sorting schemes.
+
 +++
 *** 'format-seconds' can now be used for sub-second times.
 The new optional "," parameter has been added, and
diff --git a/lisp/textmodes/bibtex.el b/lisp/textmodes/bibtex.el
index a78219e..2c7633c 100644
--- a/lisp/textmodes/bibtex.el
+++ b/lisp/textmodes/bibtex.el
@@ -204,20 +204,33 @@ bibtex-clean-entry-hook
 (defcustom bibtex-maintain-sorted-entries nil
   "If non-nil, BibTeX mode maintains all entries in sorted order.
 Allowed non-nil values are:
-plain or t   All entries are sorted alphabetically.
-crossref     All entries are sorted alphabetically unless an entry has a
+plain or t   Sort entries alphabetically by keys.
+crossref     Sort entries alphabetically by keys unless an entry has a
              crossref field.  These crossrefed entries are placed in
              alphabetical order immediately preceding the main entry.
 entry-class  The entries are divided into classes according to their
              entry type, see `bibtex-sort-entry-class'.  Within each class
-             the entries are sorted alphabetically.
+             sort entries alphabetically by keys.
+(INDEX-FUN PREDICATE)
+(INDEX-FUN PREDICATE INIT-FUN)  Sort entries using INDEX-FUN and PREDICATE.
+             Function INDEX-FUN is called for each entry with point at the
+             end of the head of the entry.  Its return values are used to
+             sort the entries using PREDICATE.  Function PREDICATE takes two
+             arguments INDEX1 and INDEX2 as returned by INDEX-FUN.
+             It should return non-nil if INDEX1 should sort before INDEX2.
+             If INIT-FUN is non-nil, it should be a function that is called
+             with no arguments to initialize the sorting.
 See also `bibtex-sort-ignore-string-entries'."
   :group 'bibtex
   :type '(choice (const nil)
+                 (const t)
                  (const plain)
                  (const crossref)
                  (const entry-class)
-                 (const t))
+                 (group :tag "Custom scheme"
+                        (function :tag "Index-Fun")
+                        (function :tag "Predicate")
+                        (option (function :tag "Init-Fun"))))
   :safe (lambda (a) (memq a '(nil t plain crossref entry-class))))
 
 (defcustom bibtex-sort-entry-class
@@ -3998,28 +4011,15 @@ bibtex-narrow-to-entry
     (narrow-to-region (bibtex-beginning-of-entry)
                       (bibtex-end-of-entry))))
 
-(defun bibtex-entry-index ()
-  "Return index of BibTeX entry head at or past position of point.
-The index is a list (KEY CROSSREF-KEY ENTRY-TYPE) that is used for sorting
-the entries of the BibTeX buffer.  CROSSREF-KEY is nil unless the value
-of `bibtex-maintain-sorted-entries' is `crossref'.  Move point to the end
-of the head of the entry found.  Return nil if no entry found."
-  (let ((case-fold-search t))
-    (if (re-search-forward bibtex-entry-maybe-empty-head nil t)
-        (let ((key (bibtex-key-in-head))
-              ;; all entry types should be downcase (for ease of comparison)
-              (entry-type (downcase (bibtex-type-in-head))))
-          ;; Don't search CROSSREF-KEY if we don't need it.
-          (if (eq bibtex-maintain-sorted-entries 'crossref)
-              (let ((bounds (bibtex-search-forward-field
-                             "\\(OPT\\)?crossref" t)))
-                (list key
-                      (if bounds (bibtex-text-in-field-bounds bounds t))
-                      entry-type))
-            (list key nil entry-type))))))
-
-(defun bibtex-init-sort-entry-class-alist ()
-  "Initialize `bibtex-sort-entry-class-alist' (buffer-local)."
+(define-obsolete-function-alias 'bibtex-init-sort-entry-class-alist
+  #'bibtex-init-sort "28.1")
+(defun bibtex-init-sort (&optional parse)
+  "Initialize sorting of BibTeX entries.
+If PARSE is non-nil, also parse BibTeX keys."
+  (if (or parse
+          (and (eq bibtex-maintain-sorted-entries 'crossref)
+               (functionp bibtex-reference-keys)))
+      (bibtex-parse-keys))
   (unless (local-variable-p 'bibtex-sort-entry-class-alist)
     (setq-local bibtex-sort-entry-class-alist
                 (let ((i -1) alist)
@@ -4029,7 +4029,36 @@ bibtex-init-sort-entry-class-alist
                       ;; All entry types should be downcase (for ease of comparison).
                       (push (cons (if (stringp entry) (downcase entry) entry) i)
                             alist)))
-                  alist))))
+                  alist)))
+  ;; Custom sorting scheme
+  (if (and (consp bibtex-maintain-sorted-entries)
+           (nth 2 bibtex-maintain-sorted-entries))
+      (funcall (nth 2 bibtex-maintain-sorted-entries))))
+
+(defun bibtex-entry-index ()
+  "Return index of BibTeX entry head at or past position of point.
+The index is a list (KEY CROSSREF-KEY ENTRY-TYPE) that is used for sorting
+the entries of the BibTeX buffer.  CROSSREF-KEY is nil unless the value of
+`bibtex-maintain-sorted-entries' is `crossref'.
+If `bibtex-maintain-sorted-entries' is (INDEX-FUN ...), the index is the return
+value of INDEX-FUN.  Return nil if no entry found.
+Move point to the end of the head of the entry found."
+  (let ((case-fold-search t))
+    (if (re-search-forward bibtex-entry-maybe-empty-head nil t)
+        (if (consp bibtex-maintain-sorted-entries)
+            ;; Custom sorting scheme
+            (funcall (car bibtex-maintain-sorted-entries))
+          (let ((key (bibtex-key-in-head))
+                ;; ENTRY-TYPE should be downcase (for ease of comparison)
+                (entry-type (downcase (bibtex-type-in-head)))
+                bounds)
+            (list key
+                  ;; Don't search CROSSREF-KEY if we don't need it.
+                  (and (eq bibtex-maintain-sorted-entries 'crossref)
+                       (setq bounds (bibtex-search-forward-field
+                                      "\\(OPT\\)?crossref" t))
+                       (bibtex-text-in-field-bounds bounds t))
+                  entry-type))))))
 
 (defun bibtex-lessp (index1 index2)
   "Predicate for sorting BibTeX entries with indices INDEX1 and INDEX2.
@@ -4038,6 +4067,8 @@ bibtex-lessp
 If its value is nil use plain sorting."
   (cond ((not index1) (not index2)) ; indices can be nil
         ((not index2) nil)
+        ((consp bibtex-maintain-sorted-entries)
+         (funcall (cadr bibtex-maintain-sorted-entries) index1 index2))
         ((eq bibtex-maintain-sorted-entries 'crossref)
          ;; CROSSREF-KEY may be nil or it can point to an entry
          ;; in another BibTeX file.  In both cases we ignore CROSSREF-KEY.
@@ -4074,10 +4105,7 @@ bibtex-sort-buffer
 are ignored."
   (interactive)
   (bibtex-beginning-of-first-entry)     ; Needed by `sort-subr'
-  (bibtex-init-sort-entry-class-alist)  ; Needed by `bibtex-lessp'.
-  (if (and (eq bibtex-maintain-sorted-entries 'crossref)
-           (functionp bibtex-reference-keys))
-      (bibtex-parse-keys))              ; Needed by `bibtex-lessp'.
+  (bibtex-init-sort)                    ; Needed by `bibtex-lessp'.
   (sort-subr nil
              'bibtex-skip-to-valid-entry   ; NEXTREC function
              'bibtex-end-of-entry          ; ENDREC function
@@ -4228,10 +4256,7 @@ bibtex-prepare-new-entry
 search to look for place for KEY.  This requires that buffer is sorted,
 see `bibtex-validate'.
 Return t if preparation was successful or nil if entry KEY already exists."
-  (bibtex-init-sort-entry-class-alist)  ; Needed by `bibtex-lessp'.
-  (if (and (eq bibtex-maintain-sorted-entries 'crossref)
-           (functionp bibtex-reference-keys))
-      (bibtex-parse-keys))              ; Needed by `bibtex-lessp'.
+  (bibtex-init-sort)  ; Needed by `bibtex-lessp'.
   (let ((key (nth 0 index))
         key-exist)
     (cond ((or (null key)
@@ -4322,9 +4347,7 @@ bibtex-validate
             (setq syntax-error t)
 
           ;; Check for duplicate keys and correct sort order
-          (bibtex-init-sort-entry-class-alist)  ; Needed by `bibtex-lessp'.
-          (bibtex-parse-keys) ; Possibly needed by `bibtex-lessp'.
-                              ; Always needed by subsequent global key check.
+          (bibtex-init-sort t) ; Needed by `bibtex-lessp' and global key check.
           (let (previous current key-list)
             (bibtex-progress-message "Checking for duplicate keys")
             (bibtex-map-entries

bug#44597: 26.3; bibtex should allow reverse sorting

[Francesco Potortì]

>> The bibtex-get-date function in my patch does not assume that the "day"
>> field is present.  When it is missing, it uses the first day of the
>> month, or day 30 if the LATEST argument is t.
>
>My point is that only rarely users will have "day" fields in their
>database.  Then it can happen easily that multiple entries have the
>same "year" and "month" field, but no other criterion to sort these
>entries.

In my use case, I needed to sort about 100 bibtex entries in reversed
date order, without any more criterion, and I had to resort to an
external tool.  The reason was that i had to participate to a career
selection where people was asked to provide a list of their own papers
listed in reverse chronological order.

I had no need for a secondary sort key, so if I could just do that with
bibtex.el it would have saved my day.

In fact, I had thought about writing a more generic patch, but with
limited time on my hands, I preferred improving the current situation
rather than trying to do the best.

>To resolve this ambiguity, if *I* was using such a scheme, I would
>want to use other fields like "author" or "title".

Certainly a more general scheme which allows arbitrary sort keys would
be best.

>						     That's when
>individual preferences enter the stage.  As we are talking about a
>scheme that (I am pretty confident) will attract only a few users in
>the first place, I believe it is better to let users define their
>own functions for this.

Sure.  But in any case, having a set of predefined sorting methods would
improve a lot over the current situation.  I assume that most users are
not able or willing to write their own function.

>> As far as I can tell, "date" and "day" are sometimes used even if not
>> standard.
>
>The "date" field is used by biblatex, which is a largely
>backward-compatible successor of BibTeX.  Its "date" field is
>supposed to follow iso-8601, and that's also assumed by the code
>example below.  (In BibTeX mode, you can switch from old-fashioned
>BibTeX to biblatex, see `bibtex-dialect'.)

Yes, that's what I have found looking around.  It is supported in my
patch, though I have never used biblatex.  I see now that biblatex has a
successor called biber.  Maybe one day I'll try it.

>I have never seen a "day" field nor any code that supports it
>(nor have I seen iso-8601 "date" fields that actually included a day).
>
>> I personally use "day" to keep track of the dates of congresses
>> (for @inproceedings).
>
>Sure, that makes a date-based sorting more straightforward and
>meaningful.

Again, that depends on the use case.  I am sure my patch can be useful
even without a secondary sorting key and without a day being specified.

>> >How about instead a new customizable sorting scheme, where the
>> >value of bibtex-maintain-sorted-entries is a cons pair
>> >
>> >  (INDEX-FUN . PREDICATE)
>> >
>> That would be certainly more flexible.  However I think it is at
>> least equally important to provide some precooked solutions.  I
>> myself would have spared some work some months ago if I had the
>> 'reverse and 'reversedate methods available.
>
>The patch below implements such a scheme (not based on a cons pair
>but a list that may include a third element INIT-FUN).
>
>To test the code, I also hacked a function `my-bibtex-index-date'
>that provides an example for INDEX-FUN, see below.  It may well be
>that it does not exactly match your taste, but it should be
>straightforward to adapt the code.  The PREDICATE is then standard
>`string-lessp' (or `string-greaterp' for reversed sorting).
>
>Can you please test this?  Thanks.

I will, but please be patient.  I do this in my spare time, which is
highly variable.

Thanks, will come back to you as soon as I can

bug#44597: 26.3; bibtex should allow reverse sorting

[Roland Winkler]

On Sat Dec 19 2020 Francesco Potortì wrote:
> In my use case, I needed to sort about 100 bibtex entries in
> reversed date order, without any more criterion, and I had to
> resort to an external tool.  The reason was that i had to
> participate to a career selection where people was asked to
> provide a list of their own papers listed in reverse chronological
> order.
> 
> I had no need for a secondary sort key, so if I could just do that
> with bibtex.el it would have saved my day.

I believe in the usual world of BibTeX, this is a rather unique
situation.  BibTeX mode has been around for many years, when nobody
ever requested this.

> In fact, I had thought about writing a more generic patch, but
> with limited time on my hands, I preferred improving the current
> situation rather than trying to do the best.

I doubt that it is possible to provide a more generic patch that
could foresee these rare cases.

Actually, I just thought of a pretty cheap very different solution
for your problem, which is to regenerate the keys for your entries
such that the existing standard "sort by keys" approach gives you
whatever you need.  The autokey algorithm for generating the keys is
pretty powerful and easy to customize beyond the default.

> Again, that depends on the use case.  I am sure my patch can be useful
> even without a secondary sorting key and without a day being
> specified.

I believe we try here to predict and comment on rare scenarios which
is rather difficult.

I just checked the BibTeX page of emacswiki

 https://www.emacswiki.org/emacs/BibTeX

because I thought people might like to use it to collect code
snippets beyond the default.  It was last updated in 2014.
So it seems to me this is not an urgent matter.

> >Can you please test this?  Thanks.
> 
> I will, but please be patient.  I do this in my spare time, which is
> highly variable.

Thanks.  To simplify things, I just installed my patch for
user-defined sorting schemes.  Even if it does not provide an
easy-to-use solution, it should cover even the most exotic
scenarios.

bug#44597: 26.3; bibtex should allow reverse sorting

[Francesco Potortì]

>> In my use case, I needed to sort about 100 bibtex entries in
>> reversed date order, without any more criterion, and I had to
>> resort to an external tool.  The reason was that i had to
>> participate to a career selection where people was asked to
>> provide a list of their own papers listed in reverse chronological
>> order.
>> 
>> I had no need for a secondary sort key, so if I could just do that
>> with bibtex.el it would have saved my day.
>
>I believe in the usual world of BibTeX, this is a rather unique
>situation.

In the end I managed to use bibtex2html, which only sorting options
other than the default are --sort-by-date and --reverse-sort.  I think
that sorting by date should not be thought of as unique, on the opposite
I'd see it as quite natural.

Whether I want to create an academic CV or the list of papers I read in
the past and found interesting, sorting by date would be one of the few
obvious generic choices, most of the rest being a matter of personal
requirements.

Even when you look up bibliographic databases, sorting by date is always
a possible option, usually the default one (in reverse mode).

>Actually, I just thought of a pretty cheap very different solution
>for your problem, which is to regenerate the keys for your entries
>such that the existing standard "sort by keys" approach gives you
>whatever you need.

That would not have worked for me.  I want to keep my key as they are,
as I reference them elsewhere in many documents.  I think this is the
normal approach.

>The autokey algorithm for generating the keys is
>pretty powerful and easy to customize beyond the default.

That's good if you often add lots of new documents.  My case is that of
long-term, slowly-growing bibliographies which occasionally may be need
to be sorted differently and whose keys need to be persistent.  I think
this is the usual case for a personal bibliography.

>> Again, that depends on the use case.  I am sure my patch can be useful
>> even without a secondary sorting key and without a day being
>> specified.
>
>I believe we try here to predict and comment on rare scenarios which
>is rather difficult.

I don't know why you think that sorting by date would be so unusual.
Unless you have a personal reason to generate and sort keys in some
specific way, date is the most obvious sorting key you can think of.

> https://www.emacswiki.org/emacs/BibTeX
>
>because I thought people might like to use it to collect code
>snippets beyond the default.  It was last updated in 2014.
>So it seems to me this is not an urgent matter.

Sure.  Also I think that bibtex is not much used, in fact.

bug#44597: 26.3; bibtex should allow reverse sorting

[Roland Winkler]

On Fri, Dec 18 2020, Roland Winkler wrote:
> Thanks.  To simplify things, I just installed my patch for
> user-defined sorting schemes.  Even if it does not provide an
> easy-to-use solution, it should cover even the most exotic
> scenarios.

Closing.