bug#36822: 26.2; Calc: calc-julian produces incorrect result

bug#36822: 26.2; Calc: calc-julian produces incorrect result

[Michel Schinz]

[-- Attachment #1: Type: text/plain, Size: 3677 bytes --]

To reproduce:
- start calc (M-x calc)
- enter algebraic mode (')
- enter today's date as <2019-07-27> then hit RET
- convert to Julian day count (t J)
- notice that the result is 2458690, which is wrong (the correct value
 is 2458692)

Oddly enough, Emacs' calendar produces the correct value. To see this,
start the calendar (M-x calendar), navigate to the date above, then
press "p a". This displays the correct value:

Astronomical (Julian) day number (at noon UTC): 2458692.0

To cross-check, several web sites are available to compute the Julian
day number, e.g. http://numerical.recipes/julian.html


In GNU Emacs 26.2 (build 1, x86_64-apple-darwin18.2.0, NS appkit-1671.20 Version 10.14.3 (Build 18D109))
of 2019-04-13 built on builder10-14.porkrind.org
Windowing system distributor 'Apple', version 10.3.1671
Recent messages:
For information about GNU Emacs and the GNU system, type C-h C-a.
user-error: The mark is not set now, so there is no region
Welcome to the GNU Emacs Calculator! Press ‘?’ or ‘h’ for help, ‘q’ to quit
<help> DEL is undefined

Configured using:
'configure --with-ns '--enable-locallisppath=/Library/Application
Support/Emacs/${version}/site-lisp:/Library/Application
Support/Emacs/site-lisp' --with-modules'

Configured features:
NOTIFY ACL GNUTLS LIBXML2 ZLIB TOOLKIT_SCROLL_BARS NS MODULES THREADS

Important settings:
 value of $LANG: fr_CH.UTF-8
 locale-coding-system: utf-8-unix

Major mode: Lisp Interaction

Minor modes in effect:
 tooltip-mode: t
 global-eldoc-mode: t
 eldoc-mode: t
 electric-indent-mode: t
 mouse-wheel-mode: t
 tool-bar-mode: t
 menu-bar-mode: t
 file-name-shadow-mode: t
 global-font-lock-mode: t
 font-lock-mode: t
 blink-cursor-mode: t
 auto-composition-mode: t
 auto-encryption-mode: t
 auto-compression-mode: t
 line-number-mode: t
 transient-mark-mode: t

Load-path shadows:
None found.

Features:
(shadow sort mail-extr emacsbug message rmc puny seq byte-opt gv
bytecomp byte-compile cconv cl-loaddefs cl-lib dired dired-loaddefs
format-spec rfc822 mml mml-sec password-cache epa derived epg epg-config
gnus-util rmail rmail-loaddefs mm-decode mm-bodies mm-encode mail-parse
rfc2231 mailabbrev gmm-utils mailheader sendmail rfc2047 rfc2045
ietf-drums mm-util mail-prsvr mail-utils calccomp calc-arith calc-alg
calc-misc calc-vec calc-forms calc-ext calc-aent calc-menu easymenu calc
calc-loaddefs calc-macs elec-pair time-date tooltip eldoc electric
uniquify ediff-hook vc-hooks lisp-float-type mwheel term/ns-win ns-win
ucs-normalize mule-util term/common-win tool-bar dnd fontset image
regexp-opt fringe tabulated-list replace newcomment text-mode elisp-mode
lisp-mode prog-mode register page menu-bar rfn-eshadow isearch timer
select scroll-bar mouse jit-lock font-lock syntax facemenu font-core
term/tty-colors frame cl-generic cham georgian utf-8-lang misc-lang
vietnamese tibetan thai tai-viet lao korean japanese eucjp-ms cp51932
hebrew greek romanian slovak czech european ethiopic indian cyrillic
chinese composite charscript charprop case-table epa-hook jka-cmpr-hook
help simple abbrev obarray minibuffer cl-preloaded nadvice loaddefs
button faces cus-face macroexp files text-properties overlay sha1 md5
base64 format env code-pages mule custom widget hashtable-print-readable
backquote threads kqueue cocoa ns multi-tty make-network-process emacs)

Memory information:
((conses 16 238377 12447)
(symbols 48 23518 1)
(miscs 40 46 241)
(strings 32 36281 1818)
(string-bytes 1 950248)
(vectors 16 37616)
(vector-slots 8 761908 14102)
(floats 8 51 104)
(intervals 56 230 0)
(buffers 992 13))

[-- Attachment #2: Type: text/html, Size: 5259 bytes --]

bug#36822: 26.2; Calc: calc-julian produces incorrect result

[Mattias Engdegård]

tags 36822 confirmed
quit

Thank you; Reproduced, and it does look wrong.
It looks like there are multiple flaws involved.

bug#36822: 26.2; Calc: calc-julian produces incorrect result

[Mattias Engdegård]

[-- Attachment #1: Type: text/plain, Size: 285 bytes --]

tags 36822 patch
quit

> It looks like there are multiple flaws involved.

This was indeed the case. Attached is a patch that attempts to fix those that I found (I didn't dig very deeply).

As I'm no calendar expert, I would greatly value an independent review of the patch.


[-- Attachment #2: 0001-Fix-various-Calc-date-conversions-bug-36822.patch --]
[-- Type: application/octet-stream, Size: 6835 bytes --]

From 0facfe8988f7d0614d8cce124a8a7cabaf7450ed Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Mattias=20Engdeg=C3=A5rd?= <mattiase@acm.org>
Date: Tue, 30 Jul 2019 17:33:19 +0200
Subject: [PATCH] Fix various Calc date conversions (bug#36822)

* lisp/calc/calc-forms.el (math-absolute-from-gregorian-dt):
Rewrite in a way that I understand, and that actually seems to work.
(math-absolute-from-julian-dt): Use Julian, not Gregorian, leap year
rules for counting days within a year.
(math-julian-date-beginning, math-julian-date-beginning-int):
Change constants to be consistent with their doc strings and the code:
use Rata Die epoch at Dec 31, 1 BC Gregorian proleptic, not Julian.
* doc/misc/calc.texi (Date Forms): Correct difference between Julian
Day and Rata Die.
* test/lisp/calc/calc-tests.el (calc-test-calendar): New test.
---
 doc/misc/calc.texi           |  4 ++--
 lisp/calc/calc-forms.el      | 38 ++++++++++++------------------------
 test/lisp/calc/calc-tests.el | 21 ++++++++++++++++++++
 3 files changed, 35 insertions(+), 28 deletions(-)

diff --git a/doc/misc/calc.texi b/doc/misc/calc.texi
index 75bbae58b2..c13ba8b940 100644
--- a/doc/misc/calc.texi
+++ b/doc/misc/calc.texi
@@ -11055,9 +11055,9 @@ Date Forms
 is the numbers of days since 12:00 noon (GMT) on November 24, 4714 BC
 in the Gregorian calendar (i.e., January 1, 4713 BC in the Julian
 calendar).  In Calc's scheme (in GMT) the Julian day origin is
-@mathit{-1721422.5}, because Calc starts at midnight instead of noon.
+@mathit{-1721424.5}, because Calc starts at midnight instead of noon.
 Thus to convert a Calc date code obtained by unpacking a
-date form into a Julian day number, simply add 1721422.5 after
+date form into a Julian day number, simply add 1721424.5 after
 compensating for the time zone difference.  The built-in @kbd{t J}
 command performs this conversion for you.
 
diff --git a/lisp/calc/calc-forms.el b/lisp/calc/calc-forms.el
index eb1a8248cd..71127ed779 100644
--- a/lisp/calc/calc-forms.el
+++ b/lisp/calc/calc-forms.el
@@ -585,29 +585,15 @@ math-absolute-from-gregorian-dt
   "Return the DATE of the day given by the Gregorian day YEAR MONTH DAY.
 Recall that DATE is the number of days since December 31, -1
 in the Gregorian calendar."
-  (if (eq year 0) (setq year -1))
-  (let ((yearm1 (math-sub year 1)))
-    (math-sub
-     ;; Add the number of days of the year and the numbers of days
-     ;; in the previous years (leap year days to be added separately)
-     (math-add (math-day-in-year year month day)
-               (math-add (math-mul 365 yearm1)
-                         ;; Add the number of Julian leap years
-                         (if (math-posp year)
-                             (math-quotient yearm1 4)
-                           (math-sub 365
-                                     (math-quotient (math-sub 3 year)
-                                                    4)))))
-     ;; Subtract the number of Julian leap years which are not
-     ;; Gregorian leap years.  In C=4N+r centuries, there will
-     ;; be 3N+r of these days.  The following will compute
-     ;; 3N+r.
-     (let* ((correction (math-mul (math-quotient yearm1 100) 3))
-            (res (math-idivmod correction 4)))
-       (math-add (if (= (cdr res) 0)
-                     0
-                   1)
-                 (car res))))))
+  (when (zerop year)                    ; Year -1 precedes year 1.
+    (setq year -1))
+  (let* ((y (if (> year 0) year (+ year 1)))  ; Astronomical year (with 0).
+         (y1 (- y 1)))                        ; Previous year.
+    (+ (* y1 365)                    ; Days up to the previous year...
+       (floor y1 4)                  ; ... including leap days.
+       (- (floor y1 100))
+       (floor y1 400)
+       (math-day-in-year year month day))))
 
 (defun math-absolute-from-julian-dt (year month day)
   "Return the DATE of the day given by the Julian day YEAR MONTH DAY.
@@ -618,7 +604,7 @@ math-absolute-from-julian-dt
     (math-sub
      ;; Add the number of days of the year and the numbers of days
      ;; in the previous years (leap year days to be added separately)
-     (math-add (math-day-in-year year month day)
+     (math-add (math-day-in-year year month day t)
                (math-add (math-mul 365 yearm1)
                          ;; Add the number of Julian leap years
                          (if (math-posp year)
@@ -712,11 +698,11 @@ math-format-date
 	       (setcdr math-fd-dt nil))
 	  fmt))))
 
-(defconst math-julian-date-beginning '(float 17214225 -1)
+(defconst math-julian-date-beginning '(float 17214245 -1)
   "The beginning of the Julian date calendar,
 as measured in the number of days before December 31, 1 BC (Gregorian).")
 
-(defconst math-julian-date-beginning-int 1721423
+(defconst math-julian-date-beginning-int 1721425
   "The beginning of the Julian date calendar,
 as measured in the integer number of days before December 31, 1 BC (Gregorian).")
 
diff --git a/test/lisp/calc/calc-tests.el b/test/lisp/calc/calc-tests.el
index 77d939eb40..e1ee20b5d2 100644
--- a/test/lisp/calc/calc-tests.el
+++ b/test/lisp/calc/calc-tests.el
@@ -193,6 +193,27 @@ calc-tests-simple
     (let ((calc-number-radix 36))
       (should (equal (math-format-number 12345678901) "36#5,O6A,QT1")))))
 
+(ert-deftest calc-test-calendar ()
+  "Test calendar conversions (bug#36822)."
+  (should (equal (calcFunc-julian (math-parse-date "2019-07-27")) 2458692))
+  (should (equal (math-parse-date "2019-07-27") '(date 737267)))
+  (should (equal (calcFunc-julian '(date 0)) 1721425))
+  (should (equal (math-date-to-gregorian-dt 1) '(1 1 1)))
+  (should (equal (math-date-to-gregorian-dt 0) '(-1 12 31)))
+  (should (equal (math-date-to-gregorian-dt -1721425) '(-4714 11 24)))
+  (should (equal (math-absolute-from-gregorian-dt 2019 7 27) 737267))
+  (should (equal (math-absolute-from-gregorian-dt 1 1 1) 1))
+  (should (equal (math-absolute-from-gregorian-dt -1 12 31) 0))
+  (should (equal (math-absolute-from-gregorian-dt -99 12 31) -35795))
+  (should (equal (math-absolute-from-gregorian-dt -4714 11 24) -1721425))
+  (should (equal (calcFunc-julian '(date -1721425)) 0))
+  (should (equal (math-date-to-julian-dt 1) '(1 1 3)))
+  (should (equal (math-date-to-julian-dt -1721425) '(-4713 1 1)))
+  (should (equal (math-absolute-from-julian-dt 2019 1 1) 737073))
+  (should (equal (math-absolute-from-julian-dt 1 1 3) 1))
+  (should (equal (math-absolute-from-julian-dt -101 1 1) -36892))
+  (should (equal (math-absolute-from-julian-dt -101 3 1) -36832))
+  (should (equal (math-absolute-from-julian-dt -4713 1 1) -1721425)))
 
 (provide 'calc-tests)
 ;;; calc-tests.el ends here
-- 
2.20.1 (Apple Git-117)

bug#36822: 26.2; Calc: calc-julian produces incorrect result

[Mattias Engdegård]

Patch pushed to master; it should at least make the code no worse than before.
Thanks for reporting the bug!