Arithmetic range error

Arithmetic range error

[Colin Baxter]

I originally posted an Arithmetic range error using org-publish to
gmane.emacs.orgmode. My original post is
[[gnus:gmane.emacs.orgmode#87ef8jwzs5.fsf@yandex.com][here]].

It was suggested that I also report it to emacs-dev. The error occurs
using org-mode from git and seems to effect emacs-26.1 but not
emacs-27.0.50. With a very inelegant ECM below, I can reproduce the
arithmetic range error. I have tested the ECM on machines running Debian
3.16.0-7-686-pae and Debian 4.9.0-8-686-pae, with the same results.

------ Begin ECM -------------

1. emacs -Q <RET>

2. We need a directory for testing. This ECM uses ~/temp.

3. In the scratch buffer, evaluate the following:

#+begin_src emacs-lisp
(add-to-list 'load-path (expand-file-name "~/path/to/git/org-mode/lisp"))
#+end_src

#+begin_src emacs-lisp
(setq org-publish-project-alist
      '(("org"
         :base-directory "~/temp/"
         :publishing-directory "~/temp"
         :publishing-function org-html-publish-to-html)))
#+end_src

4. Create file ~/temp/test.org.

5. Enter some text (e.g. This is a test) in test.org and save file.

6. M-x org-publish-current-file <RET>.

7. A satisfactory ~/temp/test.html is produced.

8. emacs-26.1 gives 'org-publish-cache-ctime-of-src: Arithmetic range
error: "floor", 1549541220.7500212'

9. emacs-27.0.50 gives no arithmetic range error.

10. Also no errors are produced if emacs-26.1 and emacs-27.0.50 use only
their generic org-modes, namely 'release_9.1.9-65-g5e4542'.

------ End ECM -------------

The issue is apparently triggered by a backport of Emacs's 662bee7d7,
specifically:

* lisp/ox-publish.el (org-publish-cache-ctime-of-src):
Prefer float-time to doing time arithmetic by hand.
[...]
@@ -1364,8 +1366,7 @@ (defun org-publish-cache-ctime-of-src (file)
 	       (expand-file-name (or (file-symlink-p file) file)
 				 (file-name-directory file)))))
     (if (not attr) (error "No such file: \"%s\"" file)
-      (+ (ash (car (nth 5 attr)) 16)
-	 (cadr (nth 5 attr))))))
+      (floor (float-time (file-attribute-modification-time attr))))))


It would appear that the code is sound, but if the above commit is
reversed then the error disappears.

-- 
Colin Baxter
m43cap@yandex.com

Re: Arithmetic range error

[Robert Pluim]

Colin Baxter <m43cap@yandex.com> writes:

> I originally posted an Arithmetic range error using org-publish to
> gmane.emacs.orgmode. My original post is
> [[gnus:gmane.emacs.orgmode#87ef8jwzs5.fsf@yandex.com][here]].
>
> It was suggested that I also report it to emacs-dev. The error occurs
> using org-mode from git and seems to effect emacs-26.1 but not
> emacs-27.0.50. With a very inelegant ECM below, I can reproduce the
> arithmetic range error. I have tested the ECM on machines running Debian
> 3.16.0-7-686-pae and Debian 4.9.0-8-686-pae, with the same results.
>

I canʼt reproduce this. Did you use the same emacs to compile org as
youʼre running here? Does the problem go away if you run org
uncompiled? Do you have any 'interesting' items in
~/.org-timestamps/org.cache?

According to my reading of the code, this can only happen if youʼre
overflowing the maximum integer value on your machine, but the value
in your backtrace is well below that even on 32 bit machines.

Could you let us know your values of most-positive-fixnum,
system-configuration, system-configuration-options and
system-configuration-features?

Robert

Re: Arithmetic range error

[Colin Baxter]

Dear Robert,
>>>>> Robert Pluim <rpluim@gmail.com> writes:

    > Colin Baxter <m43cap@yandex.com> writes:
    >> I originally posted an Arithmetic range error using org-publish
    >> to gmane.emacs.orgmode. My original post is
    >> [[gnus:gmane.emacs.orgmode#87ef8jwzs5.fsf@yandex.com][here]].
    >> 
    >> It was suggested that I also report it to emacs-dev. The error
    >> occurs using org-mode from git and seems to effect emacs-26.1 but
    >> not emacs-27.0.50. With a very inelegant ECM below, I can
    >> reproduce the arithmetic range error. I have tested the ECM on
    >> machines running Debian 3.16.0-7-686-pae and Debian
    >> 4.9.0-8-686-pae, with the same results.
    >> 

    > I canʼt reproduce this. Did you use the same emacs to compile org
    > as youʼre running here?

Yes.

    > Does the problem go away if you run org uncompiled?

No.

    > Do you have any 'interesting' items in ~/.org-timestamps/org.cache?

Don't think so. The error still occurs if ~/.org-timestamps is empty.

    > According to my reading of the code, this can only happen if
    > youʼre overflowing the maximum integer value on your machine, but
    > the value in your backtrace is well below that even on 32 bit
    > machines.

    > Could you let us know your values of most-positive-fixnum,
    > system-configuration, system-configuration-options and
    > system-configuration-features?

most-positive-fixnum:
536870911

system-configuration:
i686-pc-linux-gnu

system-configuration-options:
--prefix=/home/redknight/local/ --with-mailutils --without-sound

system-configuration-features:
"XPM JPEG TIFF GIF PNG RSVG IMAGEMAGICK GPM DBUS GSETTINGS NOTIFY ACL LIBSELINUX GNUTLS LIBXML2 FREETYPE M17N_FLT LIBOTF XFT ZLIB TOOLKIT_SCROLL_BARS GTK3 X11 THREADS LCMS2"

Best wishes,

Colin.

-- 
Colin Baxter
m43cap@yandex.com

Re: Arithmetic range error

[Robert Pluim]

Colin Baxter <m43cap@yandex.com> writes:

> Dear Robert,
>     > According to my reading of the code, this can only happen if
>     > youʼre overflowing the maximum integer value on your machine, but
>     > the value in your backtrace is well below that even on 32 bit
>     > machines.
>
>     > Could you let us know your values of most-positive-fixnum,
>     > system-configuration, system-configuration-options and
>     > system-configuration-features?
>
> most-positive-fixnum:
> 536870911
>

And that shows that Iʼd forgotten about the tag bits in emacs
integers. 'floor' is trying to convert 1549541220, which is greater
than your most-positive-fixnum.

You can either switch to a 64 bit platform, or try rebuilding emacs
with '--wide-int', which will attempt to use 62 bit integers (or
switch to the unreleased emacs-27, which has essentially unbounded
integers).

Robert

Re: Arithmetic range error

[Eli Zaretskii]

> From: Robert Pluim <rpluim@gmail.com>
> Date: Fri, 08 Feb 2019 14:15:15 +0100
> Cc: emacs-devel@gnu.org
> 
> > most-positive-fixnum:
> > 536870911
> >
> 
> And that shows that Iʼd forgotten about the tag bits in emacs
> integers. 'floor' is trying to convert 1549541220, which is greater
> than your most-positive-fixnum.
> 
> You can either switch to a 64 bit platform, or try rebuilding emacs
> with '--wide-int', which will attempt to use 62 bit integers (or
> switch to the unreleased emacs-27, which has essentially unbounded
> integers).

All true, but we still didn't drop support for 32-bit platforms
without wide-int (and don't plan to do so any time soon), so we should
fix this for those platforms' sake.

Re: Arithmetic range error

[Robert Pluim]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Robert Pluim <rpluim@gmail.com>
>> Date: Fri, 08 Feb 2019 14:15:15 +0100
>> Cc: emacs-devel@gnu.org
>> 
>> > most-positive-fixnum:
>> > 536870911
>> >
>> 
>> And that shows that Iʼd forgotten about the tag bits in emacs
>> integers. 'floor' is trying to convert 1549541220, which is greater
>> than your most-positive-fixnum.
>> 
>> You can either switch to a 64 bit platform, or try rebuilding emacs
>> with '--wide-int', which will attempt to use 62 bit integers (or
>> switch to the unreleased emacs-27, which has essentially unbounded
>> integers).
>
> All true, but we still didn't drop support for 32-bit platforms
> without wide-int (and don't plan to do so any time soon), so we should
> fix this for those platforms' sake.

The problem comes from org's desire to have the ctime as a single
integer. Reverting the commit that changed org to use 'floor' will
just result in truncation (but people might not care about that, since
weʼd be dropping the high bits of the timestamp). Ideally org would
handle the list form of timestamps, but thatʼs something for org
developers to decide.

Robert

Re: Arithmetic range error

[Eli Zaretskii]

> From: Robert Pluim <rpluim@gmail.com>
> Date: Fri, 08 Feb 2019 14:34:29 +0100
> Cc: emacs-devel@gnu.org
> 
> The problem comes from org's desire to have the ctime as a single
> integer. Reverting the commit that changed org to use 'floor' will
> just result in truncation (but people might not care about that, since
> weʼd be dropping the high bits of the timestamp). Ideally org would
> handle the list form of timestamps, but thatʼs something for org
> developers to decide.

Indeed, this is a matter for Org developers to handle.  CC'ing Bastien
for that reason.

Re: Arithmetic range error

[Robert Pluim]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Robert Pluim <rpluim@gmail.com>
>> Date: Fri, 08 Feb 2019 14:34:29 +0100
>> Cc: emacs-devel@gnu.org
>> 
>> The problem comes from org's desire to have the ctime as a single
>> integer. Reverting the commit that changed org to use 'floor' will
>> just result in truncation (but people might not care about that, since
>> weʼd be dropping the high bits of the timestamp). Ideally org would
>> handle the list form of timestamps, but thatʼs something for org
>> developers to decide.
>
> Indeed, this is a matter for Org developers to handle.  CC'ing Bastien
> for that reason.

Nicolas does a lot of org work as well, CC'd.

Thinking about it some more, the old method already truncated the
ctime anyway, and has done for a long time, so perhaps restoring it is
not unwarranted.

Robert

Re: Arithmetic range error

[Nicolas Goaziou]

Hello,

Robert Pluim <rpluim@gmail.com> writes:

> Eli Zaretskii <eliz@gnu.org> writes:
>
>>> From: Robert Pluim <rpluim@gmail.com>
>>> Date: Fri, 08 Feb 2019 14:34:29 +0100
>>> Cc: emacs-devel@gnu.org
>>> 
>>> The problem comes from org's desire to have the ctime as a single
>>> integer. Reverting the commit that changed org to use 'floor' will
>>> just result in truncation (but people might not care about that, since
>>> weʼd be dropping the high bits of the timestamp). Ideally org would
>>> handle the list form of timestamps, but thatʼs something for org
>>> developers to decide.
>>
>> Indeed, this is a matter for Org developers to handle.  CC'ing Bastien
>> for that reason.
>
> Nicolas does a lot of org work as well, CC'd.
>
> Thinking about it some more, the old method already truncated the
> ctime anyway, and has done for a long time, so perhaps restoring it is
> not unwarranted.

I modified Org (master branch) so that it uses the list representation
of time values in this situation. 

Thank you for the suggestion.

Regards,

-- 
Nicolas Goaziou

Re: Arithmetic range error

[Colin Baxter]

>>>>> "Nicolas" == Nicolas Goaziou <mail@nicolasgoaziou.fr> writes:

    Nicolas> Hello, Robert Pluim <rpluim@gmail.com> writes:

    >> Eli Zaretskii <eliz@gnu.org> writes:
    >> 
    >>>> From: Robert Pluim <rpluim@gmail.com> Date: Fri, 08 Feb 2019
    >>>> 14:34:29 +0100 Cc: emacs-devel@gnu.org
    >>>> 
    >>>> The problem comes from org's desire to have the ctime as a
    >>>> single integer. Reverting the commit that changed org to use
    >>>> 'floor' will just result in truncation (but people might not
    >>>> care about that, since weʼd be dropping the high bits of the
    >>>> timestamp). Ideally org would handle the list form of
    >>>> timestamps, but thatʼs something for org developers to decide.
    >>> 
    >>> Indeed, this is a matter for Org developers to handle.  CC'ing
    >>> Bastien for that reason.
    >> 
    >> Nicolas does a lot of org work as well, CC'd.
    >> 
    >> Thinking about it some more, the old method already truncated the
    >> ctime anyway, and has done for a long time, so perhaps restoring
    >> it is not unwarranted.

    Nicolas> I modified Org (master branch) so that it uses the list
    Nicolas> representation of time values in this situation.

This seems to have fixed the issue. I now see no arithmetic range error
using org-publish on my 32 bit machines. Thank you Nicolas, and to
everyone else for working on this.

Best wishes,

-- 
Colin Baxter
m43cap@yandex.com

Re: Arithmetic range error

[Paul Eggert]

Nicolas Goaziou wrote:
> I modified Org (master branch) so that it uses the list representation
> of time values in this situation.
I don't see why it's necessary to assume the list representation in 
ox-publish.el here:

   (let* ((key (org-publish-timestamp-filename filename pub-dir pub-func))
          (pstamp (pcase (org-publish-cache-get key)
                    ;; Old format, convert it back to a time value.
                    ((and stamp (pred wholenump)) (seconds-to-time stamp))
                    (stamp stamp)))

The call to seconds-to-time is not needed since the only use of pstamp is in 
(time-less-p pstamp ctime), which works just fine with integer timestamps. That 
is, you can simplify the above code to the following:

   (let* ((key (org-publish-timestamp-filename filename pub-dir pub-func))
          (pstamp (org-publish-cache-get key))

This simplification treats timestamps as reasonably-opaque objects, which is 
better since their format is subject to change.

Re: Arithmetic range error

[Nicolas Goaziou]

Hello,

Paul Eggert <eggert@cs.ucla.edu> writes:

> I don't see why it's necessary to assume the list representation in
> ox-publish.el here:
>
>   (let* ((key (org-publish-timestamp-filename filename pub-dir pub-func))
>          (pstamp (pcase (org-publish-cache-get key)
>                    ;; Old format, convert it back to a time value.
>                    ((and stamp (pred wholenump)) (seconds-to-time stamp))
>                    (stamp stamp)))

It's simply because I hadn't realized `time-less-p' also handled integers.

> The call to seconds-to-time is not needed since the only use of pstamp
> is in (time-less-p pstamp ctime), which works just fine with integer
> timestamps. That is, you can simplify the above code to the following:
>
>   (let* ((key (org-publish-timestamp-filename filename pub-dir pub-func))
>          (pstamp (org-publish-cache-get key))

Fixed. Thank you for the heads up.

Regards,

-- 
Nicolas Goaziou