bug#31118: 27.0.50; Can't load/compile websocket in 32bit master

bug#31118: 27.0.50; Can't load/compile websocket in 32bit master

[Stefan Monnier]

Package: Emacs
Version: 27.0.50


The websocket package available from GNU ELPA can't be used in a 32bit
build of master any more because of the #xffffffff constant in its code:

    (defun websocket-to-bytes (val nbytes)
      "Encode the integer VAL in NBYTES of data.
    NBYTES much be a power of 2, up to 8.
    
    This supports encoding values up to 536870911 bytes (2^29 - 1),
    approximately 537M long."
      (when (and (< nbytes 8)
                 (> val (expt 2 (* 8 nbytes))))
        ;; not a user-facing error, this must be caused from an error in
        ;; this library
        (error "websocket-to-bytes: Value %d could not be expressed in %d bytes"
               val nbytes))
      (if (= nbytes 8)
          (progn
            (let ((hi-32bits (lsh val -32))
                  ;; Test for systems that don't have > 32 bits, and
                  ;; for those systems just return the value.
                  (low-32bits (if (= 0 (expt 2 32))
                                  val
                                (logand #xffffffff val))))
              (when (or (> hi-32bits 0) (> (lsh low-32bits -29) 0))
                (signal 'websocket-frame-too-large val))
              (bindat-pack `((:val vec 2 u32))
                           `((:val . [,hi-32bits ,low-32bits])))))
        (bindat-pack
         `((:val ,(cond ((= nbytes 1) 'u8)
                        ((= nbytes 2) 'u16)
                        ((= nbytes 4) 'u32)
                        ;; Library error, not system error
                        (t (error "websocket-to-bytes: Unknown NBYTES: %S" nbytes)))))
         `((:val . ,val)))))

The code used to compile&run correctly before both on 32bit and 64bit
systems (but admittedly, it failed to run correctly on 64bit systems
after having been compiled on a 32bit system).


        Stefan

bug#31118: 27.0.50; Can't load/compile websocket in 32bit master

[Paul Eggert]

[-- Attachment #1: Type: text/plain, Size: 415 bytes --]

I fixed that by installing the attached into ELPA.

I can't build ELPA from scratch on master now, for other reasons. The build 
fails as follows:

In toplevel form:
packages/counsel-ebdb/counsel-ebdb.el:29:1:Error: Cannot open load file: No such 
file or directory, ebdb
make: *** [GNUmakefile:171: packages/counsel-ebdb/counsel-ebdb.elc] Error 1

Is anybody building ELPA regularly? It looks like not.

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-Port-websocket-to-bleeding-edge-32-bit-Emacs.patch --]
[-- Type: text/x-patch; name="0001-Port-websocket-to-bleeding-edge-32-bit-Emacs.patch", Size: 1436 bytes --]

From 76ee35ef60079bb6053e82e280274204a1b9fc70 Mon Sep 17 00:00:00 2001
From: Paul Eggert <eggert@cs.ucla.edu>
Date: Tue, 10 Apr 2018 15:34:09 -0700
Subject: [PATCH] Port websocket to bleeding-edge 32-bit Emacs

Problem reported by Stefan Monnier (Bug#31118).
* packages/websocket/websocket.el (websocket-to-bytes):
Do not assume that #xffffffff is a valid fixnum.
---
 packages/websocket/websocket.el | 9 +++------
 1 file changed, 3 insertions(+), 6 deletions(-)

diff --git a/packages/websocket/websocket.el b/packages/websocket/websocket.el
index 39a31c359..0533fb378 100644
--- a/packages/websocket/websocket.el
+++ b/packages/websocket/websocket.el
@@ -234,12 +234,9 @@ approximately 537M long."
            val nbytes))
   (if (= nbytes 8)
       (progn
-        (let ((hi-32bits (lsh val -32))
-              ;; Test for systems that don't have > 32 bits, and
-              ;; for those systems just return the value.
-              (low-32bits (if (= 0 (expt 2 32))
-                              val
-                            (logand #xffffffff val))))
+        (let* ((hi-32bits (lsh val -32))
+	       ;; This is just VAL on systems that don't have >= 32 bits.
+	       (low-32bits (- val (lsh hi-32bits 32))))
           (when (or (> hi-32bits 0) (> (lsh low-32bits -29) 0))
             (signal 'websocket-frame-too-large val))
           (bindat-pack `((:val vec 2 u32))
-- 
2.14.3

bug#31118: 27.0.50; Can't load/compile websocket in 32bit master

[Stefan Monnier]

> I fixed that by installing the attached into ELPA.
>
> I can't build ELPA from scratch on master now, for other reasons. The build
> fails as follows:
>
> In toplevel form:
> packages/counsel-ebdb/counsel-ebdb.el:29:1:Error: Cannot open load file: No
> such file or directory, ebdb
> make: *** [GNUmakefile:171: packages/counsel-ebdb/counsel-ebdb.elc] Error 1
>
> Is anybody building ELPA regularly?

Yes, I do.  Not sure why you can't find ebdb since it's in elpa.git.
Have you added the .../elpa/packages to your package-directory-list?

But I can tell you that the build fails a bit later because

    grep '#xffffffff' elpa/**/*.el

still finds other cases.


        Stefan

bug#31118: 27.0.50; Can't load/compile websocket in 32bit master

[Paul Eggert]

Stefan Monnier wrote:
>> I fixed that by installing the attached into ELPA.
>>
>> I can't build ELPA from scratch on master now, for other reasons. The build
>> fails as follows:
>>
>> In toplevel form:
>> packages/counsel-ebdb/counsel-ebdb.el:29:1:Error: Cannot open load file: No
>> such file or directory, ebdb
>> make: *** [GNUmakefile:171: packages/counsel-ebdb/counsel-ebdb.elc] Error 1
>>
>> Is anybody building ELPA regularly?
> 
> Yes, I do.  Not sure why you can't find ebdb since it's in elpa.git.
> Have you added the .../elpa/packages to your package-directory-list?

No, I tried to follow the instructions in README 
<http://git.savannah.gnu.org/cgit/emacs/elpa.git/tree/README>. They say that to 
install all the packages "in place" (which I assume is the simplest way to build 
ELPA) one should run "make externals" first. This fails from a fresh elpa 
checkout as follows:

$ make externals
emacs --batch -l admin/archive-contents.el \
     -f archive-add/remove/update-externals
No "emacs" subdir: will skip :core packages
Cloning branch auctex:
fatal: Not a valid object name: 'origin/externals/auctex'.
...
Cloning branch ebdb:
fatal: Not a valid object name: 'origin/externals/ebdb'.
...

and no doubt this explains the problem. I charged ahead and typed plain 'make' 
which failed as I mentioned earlier. What am I doing wrong? Should the README be 
updated? (As you can probably tell I've never used ELPA, and I must say that the 
startup overhead is offputting.)


> But I can tell you that the build fails a bit later because
> 
>      grep '#xffffffff' elpa/**/*.el
> 
> still finds other cases.
That command doesn't work for me:

$ grep '#xffffffff' elpa/**/*.el
grep: elpa/**/*.el: No such file or directory

And this one doesn't find anything for me, in the elpa directory:

grep -r '#xffffffff' .

Are we talking about the same repository? I'm using the Savannah ELPA repository 
under the emacs tree, as in the abovementioned URL.

bug#31118: 27.0.50; Can't load/compile websocket in 32bit master

[Paul Eggert]

Andreas Schwab wrote:
> How did you clone the repository?

I followed my usual practice:

git clone eggert@git.sv.gnu.org:/srv/git/emacs/elpa.git elpa-savannah
git clone elpa-savannah elpa

I do this so that I have a relatively-pristine repository (elpa-savannah) and a 
dirtier working one (elpa). Hmm, I now see that this practice does not work with 
ELPA for some reason. Too bad. I guess I'll not bother with the 
relatively-pristine repository for ELPA. Thanks for the help.

bug#31118: 27.0.50; Can't load/compile websocket in 32bit master

[Stefan Monnier]

> $ make externals
> emacs --batch -l admin/archive-contents.el \
>     -f archive-add/remove/update-externals
> No "emacs" subdir: will skip :core packages
> Cloning branch auctex:
> fatal: Not a valid object name: 'origin/externals/auctex'.
> ...
> Cloning branch ebdb:
> fatal: Not a valid object name: 'origin/externals/ebdb'.
> ...

Indeed, "make externals" assumes that the other branches are in your
repository, in the remote called "origin".  For some reason your
local repository doesn't seem to be setup this way.

> That command doesn't work for me:
>
> $ grep '#xffffffff' elpa/**/*.el
> grep: elpa/**/*.el: No such file or directory

The "**/" glob is a zsh extension (supported also by Eshell, AFAIK),
whose meaning is clear enough.

> And this one doesn't find anything for me, in the elpa directory:
> grep -r '#xffffffff' .

This is because of the above error (the other matches are in exwm,
i.e. in the origin/externals/exwm branch).


        Stefan

bug#31118: 27.0.50; Can't load/compile websocket in 32bit master

[Basil L. Contovounesios]

Stefan Monnier <monnier@IRO.UMontreal.CA> writes:

>> That command doesn't work for me:
>>
>> $ grep '#xffffffff' elpa/**/*.el
>> grep: elpa/**/*.el: No such file or directory
>
> The "**/" glob is a zsh extension (supported also by Eshell, AFAIK),
> whose meaning is clear enough.

FWIW, Bash 4.0 can also interpret "**" recursively via the globstar
shell option; see https://stackoverflow.com/a/28199633/3084001.

-- 
Basil

bug#31118: 27.0.50; Can't load/compile websocket in 32bit master

[Paul Eggert]

[-- Attachment #1: Type: text/plain, Size: 1368 bytes --]

On 04/11/2018 05:19 AM, Stefan Monnier wrote:
> "make externals" assumes that the other branches are in your
> repository, in the remote called "origin".  For some reason your
> local repository doesn't seem to be setup this way.

Yes, Andreas put me on the right track. I installed the attached to help 
others who might have a similar problem.

However, even when I clone directly from Savannah so that 'make 
externals' works, the resulting 'make' still fails from a fresh 
checkout. Here's the output of redoing the 'make'. (I can resend the log 
of the full 'make -j5' if you like, but it's 11,000 lines long....) Are 
the people doing periodic builds starting from a clean sheet, or is 
there some other mysterious thing that I am doing "wrong"?

$ make
Byte compiling packages/hyperbole/hinit.el
Unable to activate package ‘helm-ebdb’.
Required package ‘helm-1.0’ is unavailable
Unable to activate package ‘ebdb-i18n-chn’.
Required package ‘pyim-1.6.0’ is unavailable
Loading /home/eggert/src/gnu/elpa/packages/hyperbole/hui-window.el 
(source)...

In toplevel form:
packages/hyperbole/hinit.el:26:1:Error: "hui-window.el": ‘hmouse-alist’ 
must be defined before loading this.
make: *** [GNUmakefile:171: packages/hyperbole/hinit.elc] Error 1

Compilation exited abnormally with code 2 at Wed Apr 11 08:56:13


[-- Attachment #2: 0001-README-Don-t-clone-a-clone.patch --]
[-- Type: text/x-patch, Size: 761 bytes --]

From ed3aa83d0d4791d4a36085a9e8cd8ea41c18b014 Mon Sep 17 00:00:00 2001
From: Paul Eggert <eggert@cs.ucla.edu>
Date: Wed, 11 Apr 2018 09:09:25 -0700
Subject: [PATCH] * README: Don't clone a clone.

---
 README | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/README b/README
index bc74eddd9..067f4c769 100644
--- a/README
+++ b/README
@@ -10,6 +10,14 @@ and how to deploy the archive (either on elpa.gnu.org, or a local copy
 for testing purposes).
 
 
+* GETTING THE SOURCE
+
+Start with source that is cloned directly from Savannah.  See:
+
+https://savannah.gnu.org/git/?group=emacs
+
+and look for "ELPA".  Using a clone of a clone does not work.
+
 * DIRECTORY LAYOUT
 
 ** admin/    -- scripts for administering the package archive.
-- 
2.14.3

bug#31118: 27.0.50; Can't load/compile websocket in 32bit master

[Stefan Monnier]

> if you like, but it's 11,000 lines long....) Are the people doing periodic
> builds starting from a clean sheet, or is there some other mysterious thing
> that I am doing "wrong"?

I don't re-compile everything all the time, no.  It would be good to
have such a thing in a CI somewhere, ideally with some way to
automatically send the build failure notices to the right package maintainer.


        Stefan

bug#31118: 27.0.50; Can't load/compile websocket in 32bit master

[Glenn Morris]

Stefan Monnier wrote:

> I don't re-compile everything all the time, no.  It would be good to
> have such a thing in a CI somewhere, ideally with some way to
> automatically send the build failure notices to the right package maintainer.

Such a setup was posted once:
http://lists.gnu.org/archive/html/emacs-devel/2013-07/msg00305.html

bug#31118: 27.0.50; Can't load/compile websocket in 32bit master

[Paul Eggert]

[-- Attachment #1: Type: text/plain, Size: 503 bytes --]

Although I still can't build ELPA (see Bug#31118#31), I took a shot at 
making ELPA build for 32-bit Emacs on master by installing the attached 
patches into the xelb and exwm packages, respectively.

As I understand it, some parts of ELPA still assume 64-bit Emacs, but 
that's lower priority as it's not a regression. Also, one still cannot 
reliably build ELPA on 64-bit Emacs and run it on 32-bit Emacs (or vice 
versa), but people don't do that much and I suspect it wasn't really 
working either.

[-- Attachment #2: 0001-Port-to-32-bit-Emacs-master.patch --]
[-- Type: text/x-patch, Size: 8039 bytes --]

From 0a46cc62a3a82dcaa1d1bf0929a5207260f635e3 Mon Sep 17 00:00:00 2001
From: Paul Eggert <eggert@cs.ucla.edu>
Date: Sun, 15 Apr 2018 15:00:09 -0700
Subject: [PATCH] Port to 32-bit Emacs master

Rewrite #xFFFFFFFF to 4294967295. and similarly for other
hex constants intended to be floating-point on 32-bit Emacs.
* el_client.el (xelb-parse-enum):
Omit enums that do not fit in 32-bit Emacs.
* xcb-xinput.el (xcb:xinput:ModifierMask:Any):
* xcb-xkb.el (xcb:xkb:Control:IgnoreLockMods)
(xcb:xkb:Control:PerKeyRepeat, xcb:xkb:Control:ControlsEnabled):
Remove.
---
 el_client.el  |  8 +++++++-
 xcb-cursor.el |  2 +-
 xcb-icccm.el  |  2 +-
 xcb-types.el  | 28 ++++++++++++++--------------
 xcb-xinput.el |  2 --
 xcb-xkb.el    |  3 ---
 6 files changed, 23 insertions(+), 22 deletions(-)

diff --git a/el_client.el b/el_client.el
index 9b7077f98..393745ef0 100644
--- a/el_client.el
+++ b/el_client.el
@@ -351,7 +351,13 @@ an `xelb-auto-padding' attribute."
                         (if expression
                             (setq value (xelb-parse-expression expression))
                           (setq value (1+ value)))
-                        `(defconst ,name ,value))))
+			;; Omit the rare enums that do not fit in a fixnum in
+			;; 32-bit Emacs, so that the resulting .el and .elc
+			;; files are portable to 32-bit Emacs.  Admittedly
+			;; this is a kludge.
+			(unless (and ((integerp value)
+				      (not (<= -536870912 value 536870911))))
+			  `(defconst ,name ,value)))))
                   items))))
 
 (defun xelb-parse-typedef (node)
diff --git a/xcb-cursor.el b/xcb-cursor.el
index 12bbf27ac..71842a383 100644
--- a/xcb-cursor.el
+++ b/xcb-cursor.el
@@ -230,7 +230,7 @@
 
 (defconst xcb:cursor:-file-chunk-image-header 36
   "Header value of image-type chunk in Xcursor file.")
-(defconst xcb:cursor:-file-chunk-image-type #xFFFD0002
+(defconst xcb:cursor:-file-chunk-image-type 4294770690.
   "Type of image-type chunk in Xcursor file.")
 (defconst xcb:cursor:-file-chunk-image-version 1
   "Version of image-type chunk in Xcursor file.")
diff --git a/xcb-icccm.el b/xcb-icccm.el
index 613bed187..4599a7dbf 100644
--- a/xcb-icccm.el
+++ b/xcb-icccm.el
@@ -119,7 +119,7 @@ A valid timestamp (rather than `xcb:Time:CurrentTime') must be supplied.")
 (defclass xcb:icccm:-GetProperty (xcb:GetProperty)
   ((delete :initform 0)
    (long-offset :initform 0)
-   (long-length :initform 1000000000))  ;as long as possible
+   (long-length :initform 1000000000.))  ;as long as possible
   :documentation "Get an ICCCM property (request part).")
 
 (defclass xcb:icccm:-GetProperty~reply (xcb:GetProperty~reply)
diff --git a/xcb-types.el b/xcb-types.el
index f5c1a3c13..0c172b64d 100644
--- a/xcb-types.el
+++ b/xcb-types.el
@@ -155,8 +155,8 @@
                       (logand (lsh value -16) #xFF)
                       (logand (lsh value -8) #xFF)
                       (logand value #xFF))
-            (let* ((msdw (min #xFFFFFFFF (truncate value #x100000000)))
-                   (lsdw (min #xFFFFFFFF
+            (let* ((msdw (min 4294967295. (truncate value 4294967296.)))
+                   (lsdw (min 4294967295.
                               (truncate (- value (* msdw 4294967296.0))))))
               (vector (logand (lsh msdw -24) #xFF) (logand (lsh msdw -16) #xFF)
                       (logand (lsh msdw -8) #xFF) (logand msdw #xFF)
@@ -173,8 +173,8 @@
                       (logand (lsh value -40) #xFF)
                       (logand (lsh value -48) #xFF)
                       (logand (lsh value -56) #xFF))
-            (let* ((msdw (min #xFFFFFFFF (truncate value #x100000000)))
-                   (lsdw (min #xFFFFFFFF
+            (let* ((msdw (min 4294967295. (truncate value 4294967296.)))
+                   (lsdw (min 4294967295.
                               (truncate (- value (* msdw 4294967296.0))))))
               (vector (logand lsdw #xFF) (logand (lsh lsdw -8) #xFF)
                       (logand (lsh lsdw -16) #xFF) (logand (lsh lsdw -24) #xFF)
@@ -207,11 +207,11 @@
           (vector 0 0 0 0
                   (logand (lsh value -24) #xFF) (logand (lsh value -16) #xFF)
                   (logand (lsh value -8) #xFF) (logand value #xFF))
-        (let* ((msw (min #xFFFF (truncate value #x1000000000000)))
+        (let* ((msw (min #xFFFF (truncate value 281474976710656.)))
                (w1 (min #xFFFF
                         (truncate (setq value
                                         (- value (* msw 281474976710656.0)))
-                                  #x100000000)))
+                                  4294967296.)))
                (w2 (min #xFFFF
                         (truncate (setq value (- value (* w1 4294967296.0)))
                                   #x10000)))
@@ -226,11 +226,11 @@
           (vector (logand value #xFF) (logand (lsh value -8) #xFF)
                   (logand (lsh value -16) #xFF) (logand (lsh value -24) #xFF)
                   0 0 0 0)
-        (let* ((msw (min #xFFFF (truncate value #x1000000000000)))
+        (let* ((msw (min #xFFFF (truncate value 281474976710656.)))
                (w1 (min #xFFFF
                         (truncate (setq value
                                         (- value (* msw 281474976710656.0)))
-                                  #x100000000)))
+                                  4294967296.)))
                (w2 (min #xFFFF
                         (truncate (setq value (- value (* w1 4294967296.0)))
                                   #x10000)))
@@ -244,13 +244,13 @@
   "4 bytes signed integer => byte array (MSB first)."
   (xcb:-pack-u4 (if (>= value 0)
                     value
-                  (+ value #x100000000)))) ;treated as float for 32-bit
+                  (+ value 4294967296.)))) ;treated as float for 32-bit
 
 (defsubst xcb:-pack-i4-lsb (value)
   "4 bytes signed integer => byte array (LSB first)."
   (xcb:-pack-u4-lsb (if (>= value 0)
                         value
-                      (+ value #x100000000)))) ;treated as float for 32-bit
+                      (+ value 4294967296.)))) ;treated as float for 32-bit
 
 (defsubst xcb:-unpack-u1 (data offset)
   "Byte array => 1 byte unsigned integer."
@@ -361,16 +361,16 @@
 (defsubst xcb:-unpack-i4 (data offset)
   "Byte array => 4 bytes signed integer (MSB first)."
   (let ((value (xcb:-unpack-u4 data offset)))
-    (if (< value #x80000000)            ;treated as float for 32-bit
+    (if (< value 2147483648.)           ;treated as float for 32-bit
         value
-      (- value #x100000000))))          ;treated as float for 32-bit
+      (- value 4294967296.))))          ;treated as float for 32-bit
 
 (defsubst xcb:-unpack-i4-lsb (data offset)
   "Byte array => 4 bytes signed integer (LSB first)."
   (let ((value (xcb:-unpack-u4-lsb data offset)))
-    (if (< value #x80000000)            ;treated as float for 32-bit
+    (if (< value 2147483648.)           ;treated as float for 32-bit
         value
-      (- value #x100000000))))          ;treated as float for 32-bit
+      (- value 4294967296.))))          ;treated as float for 32-bit
 
 (defmacro xcb:-fieldref (field)
   "Evaluate a <fieldref> field."
diff --git a/xcb-xinput.el b/xcb-xinput.el
index b2ef72b8c..f4da64e37 100644
--- a/xcb-xinput.el
+++ b/xcb-xinput.el
@@ -1862,8 +1862,6 @@
 (defconst xcb:xinput:GrabType:FocusIn 3)
 (defconst xcb:xinput:GrabType:TouchBegin 4)
 
-(defconst xcb:xinput:ModifierMask:Any 2147483648)
-
 (defclass xcb:xinput:GrabModifierInfo
   (xcb:-struct)
   ((modifiers :initarg :modifiers :type xcb:CARD32)
diff --git a/xcb-xkb.el b/xcb-xkb.el
index 881f4df0f..1f26deb68 100644
--- a/xcb-xkb.el
+++ b/xcb-xkb.el
@@ -105,9 +105,6 @@
 
 (defconst xcb:xkb:Control:GroupsWrap 134217728)
 (defconst xcb:xkb:Control:InternalMods 268435456)
-(defconst xcb:xkb:Control:IgnoreLockMods 536870912)
-(defconst xcb:xkb:Control:PerKeyRepeat 1073741824)
-(defconst xcb:xkb:Control:ControlsEnabled 2147483648)
 
 (defconst xcb:xkb:AXOption:SKPressFB 1)
 (defconst xcb:xkb:AXOption:SKAcceptFB 2)
-- 
2.14.3


[-- Attachment #3: 0001-Port-to-32-bit-Emacs-on-master-branch.patch --]
[-- Type: text/x-patch, Size: 1260 bytes --]

From ff4ae82fd7ca9101da92f21c7f46f991da99a30e Mon Sep 17 00:00:00 2001
From: Paul Eggert <eggert@cs.ucla.edu>
Date: Sun, 15 Apr 2018 14:52:39 -0700
Subject: [PATCH] Port to 32-bit Emacs on master branch

---
 exwm-layout.el | 2 +-
 exwm.el        | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/exwm-layout.el b/exwm-layout.el
index b74f512a4..764259870 100644
--- a/exwm-layout.el
+++ b/exwm-layout.el
@@ -117,7 +117,7 @@
   (with-current-buffer (exwm--id->buffer id)
     (unless (or (exwm-layout--iconic-state-p)
                 (and exwm--floating-frame
-                     (eq #xffffffff exwm--desktop)))
+                     (eq 4294967295. exwm--desktop)))
       (exwm--log "Hide #x%x" id)
       (when exwm--floating-frame
         (let* ((container (frame-parameter exwm--floating-frame
diff --git a/exwm.el b/exwm.el
index 195ca7b80..17f73d8ca 100644
--- a/exwm.el
+++ b/exwm.el
@@ -154,7 +154,7 @@
       (when reply
         (setq desktop (slot-value reply 'value))
         (cond
-         ((eq desktop #xffffffff)
+         ((eq desktop 4294967295.)
           (unless (or (not exwm--floating-frame)
                       (eq exwm--frame exwm-workspace--current)
                       (and exwm--desktop
-- 
2.14.3

bug#31118: 27.0.50; Can't load/compile websocket in 32bit master

[Stefan Monnier]

> Although I still can't build ELPA (see Bug#31118#31), I took a shot at
> making ELPA build for 32-bit Emacs on master by installing the attached
> patches into the xelb and exwm packages, respectively.

I think a major problem remains: these problems in GNU ELPA packages
probably also affect other packages that we can't patch so easily.

So I think we should reconsider whether reading something like
#xffffffff in a 32bit build should signal an error.

More specifically, I think we should first make it emit a warning, while
keeping the old broken behavior.  Then in a later release we can make it
signal an error.


        Stefan

bug#31118: 27.0.50; Can't load/compile websocket in 32bit master

[Paul Eggert]

On 04/15/2018 06:26 PM, Stefan Monnier wrote:
> I think we should first make it emit a warning, while
> keeping the old broken behavior.

Do you mean 'read' should call 'display-warning' for now? I suppose we could 
make read-integer-overflow-as-float be a three-state variable: either (1) signal 
an error, or (2) call display-warning and yield a float, or (3) silently yield a 
float, and have (2) be the default for now. However, I worry that (2) might lead 
to further problems, e.g., display-warning is Lisp code that might in turn call 
'read' and loop recursively.

The code we're talking about had nonportable code like (eq desktop #xffffffff) 
that doesn't work on platforms with 30-bit fixnums anyway. If the goal is to 
make Elisp code safer on 32-bit platforms, --with-wide-int is a simpler and more 
reliable way to get there (of course bignums would be better but that's a much 
bigger project). Most Emacs development occurs on 64-bit platforms now, and it's 
becoming more and more of a pain to insist that programmers must hack on code to 
make it portable to platforms with 30-bit fixnums.

Admittedly --with-wide-hit is a ~30% CPU performance hit on my circa-2010 AMD 
desktop. If defaulting to --with-wide-int is too drastic, I hope that the 
already-existing read-integer-overflow-as-float flag is enough backstop for 
people who want to run nonportable code on platforms with 30-bit fixnums.

bug#31118: 27.0.50; Can't load/compile websocket in 32bit master

[Stefan Monnier]

> Do you mean 'read' should call 'display-warning' for now? I suppose we could
> make read-integer-overflow-as-float be a three-state variable: either (1)
> signal an error, or (2) call display-warning and yield a float, or (3)
> silently yield a float, and have (2) be the default for now. However,
> I worry that (2) might lead to further problems, e.g., display-warning is
> Lisp code that might in turn call 'read' and loop recursively.

We already solved this problem for old-style backquotes: don't emit the
warning directly, instead set a variable and let the caller check the
value of the variable to emit a warning if needed.

> The code we're talking about had nonportable code like (eq desktop
> #xffffffff) that doesn't work on platforms with 30-bit fixnums anyway.

I don't disagree with your change.  But the problems encountered with
code in GNU ELPA makes me think that we want to do it in two steps to
reduce the backward compatibility pain.

> Admittedly --with-wide-hit is a ~30% CPU performance hit on my circa-2010
> AMD desktop.

4 out of 5 of my machines predate your AMD desktop, and I use a 32bit
build on them.

> If defaulting to --with-wide-int is too drastic, I hope that the
> already-existing read-integer-overflow-as-float flag is enough
> backstop for people who want to run nonportable code on platforms with
> 30-bit fixnums.

If we want to signal an error by default, then I think it's important
for this error to give clear steps to set read-integer-overflow-as-float
(and in a message that's not too prone to disappear before the user
gets to see and act on it).


        Stefan

bug#31118: 27.0.50; Can't load/compile websocket in 32bit master

[Paul Eggert]

[-- Attachment #1: Type: text/plain, Size: 239 bytes --]

On 04/17/2018 05:18 AM, Stefan Monnier wrote:
> If we want to signal an error by default, then I think it's important
> for this error to give clear steps to set read-integer-overflow-as-float

Something like the attached patch, perhaps?


[-- Attachment #2: 0001-Tell-user-about-read-integer-overflow-as-float.patch --]
[-- Type: text/x-patch, Size: 1067 bytes --]

From eb2fa1a00c5c85aced40900b59183dc03d822958 Mon Sep 17 00:00:00 2001
From: Paul Eggert <eggert@cs.ucla.edu>
Date: Tue, 17 Apr 2018 16:52:04 -0700
Subject: [PATCH] Tell user about read-integer-overflow-as-float

* src/lread.c (string_to_number): Suggest
read-integer-overflow-as-float in signal message.
Suggested by Stefan Monnier (Bug#31118#58).
---
 src/lread.c | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/src/lread.c b/src/lread.c
index 6eda740540..16e34aff0c 100644
--- a/src/lread.c
+++ b/src/lread.c
@@ -3797,7 +3797,12 @@ string_to_number (char const *string, int base, int flags)
 	value = n;
 
       if (! (state & DOT_CHAR) && ! (flags & S2N_OVERFLOW_TO_FLOAT))
-	xsignal1 (Qoverflow_error, build_string (string));
+	{
+	  AUTO_STRING (fmt, ("%s is out of fixnum range; "
+			     "maybe set `read-integer-overflow-as-float'?"));
+	  xsignal1 (Qoverflow_error,
+		    CALLN (Fformat_message, fmt, build_string (string)));
+	}
     }
 
   /* Either the number uses float syntax, or it does not fit into a fixnum.
-- 
2.14.3

bug#31118: 27.0.50; Can't load/compile websocket in 32bit master

[Stefan Monnier]

> Something like the attached patch, perhaps?

That might be good enough.


        Stefan


> From eb2fa1a00c5c85aced40900b59183dc03d822958 Mon Sep 17 00:00:00 2001
> From: Paul Eggert <eggert@cs.ucla.edu>
> Date: Tue, 17 Apr 2018 16:52:04 -0700
> Subject: [PATCH] Tell user about read-integer-overflow-as-float
>
> * src/lread.c (string_to_number): Suggest
> read-integer-overflow-as-float in signal message.
> Suggested by Stefan Monnier (Bug#31118#58).
> ---
>  src/lread.c | 7 ++++++-
>  1 file changed, 6 insertions(+), 1 deletion(-)
>
> diff --git a/src/lread.c b/src/lread.c
> index 6eda740540..16e34aff0c 100644
> --- a/src/lread.c
> +++ b/src/lread.c
> @@ -3797,7 +3797,12 @@ string_to_number (char const *string, int base, int flags)
>  	value = n;
>  
>        if (! (state & DOT_CHAR) && ! (flags & S2N_OVERFLOW_TO_FLOAT))
> -	xsignal1 (Qoverflow_error, build_string (string));
> +	{
> +	  AUTO_STRING (fmt, ("%s is out of fixnum range; "
> +			     "maybe set `read-integer-overflow-as-float'?"));
> +	  xsignal1 (Qoverflow_error,
> +		    CALLN (Fformat_message, fmt, build_string (string)));
> +	}
>      }
>  
>    /* Either the number uses float syntax, or it does not fit into a fixnum.

bug#31118: 27.0.50; Can't load/compile websocket in 32bit master

[Paul Eggert]

[-- Attachment #1: Type: text/plain, Size: 161 bytes --]

On 04/18/2018 05:22 AM, Stefan Monnier wrote:
> That might be good enough.

OK, I installed the attached, which is slightly improved from what I 
sent earlier.


[-- Attachment #2: 0001-Tell-user-about-read-integer-overflow-as-float.patch --]
[-- Type: text/x-patch, Size: 1095 bytes --]

From edb67abcd778cde39c6d65e95c4b68d2b2e3505e Mon Sep 17 00:00:00 2001
From: Paul Eggert <eggert@cs.ucla.edu>
Date: Wed, 18 Apr 2018 12:38:19 -0700
Subject: [PATCH] Tell user about read-integer-overflow-as-float

* src/lread.c (string_to_number): Suggest
read-integer-overflow-as-float in signal message.
Suggested by Stefan Monnier (Bug#31118#58).
---
 src/lread.c | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/src/lread.c b/src/lread.c
index 6eda740540..9ddd8d5a7f 100644
--- a/src/lread.c
+++ b/src/lread.c
@@ -3797,7 +3797,12 @@ string_to_number (char const *string, int base, int flags)
 	value = n;
 
       if (! (state & DOT_CHAR) && ! (flags & S2N_OVERFLOW_TO_FLOAT))
-	xsignal1 (Qoverflow_error, build_string (string));
+	{
+	  AUTO_STRING (fmt, ("%s is out of fixnum range; "
+			     "maybe set `read-integer-overflow-as-float'?"));
+	  AUTO_STRING_WITH_LEN (arg, string, cp - string);
+	  xsignal1 (Qoverflow_error, CALLN (Fformat_message, fmt, arg));
+	}
     }
 
   /* Either the number uses float syntax, or it does not fit into a fixnum.
-- 
2.14.3

bug#31118: 27.0.50; Can't load/compile websocket in 32bit master

[Eli Zaretskii]

> From: Stefan Monnier <monnier@IRO.UMontreal.CA>
> Date: Wed, 11 Apr 2018 08:19:36 -0400
> Cc: 31118@debbugs.gnu.org
> 
> > $ make externals
> > emacs --batch -l admin/archive-contents.el \
> >     -f archive-add/remove/update-externals
> > No "emacs" subdir: will skip :core packages
> > Cloning branch auctex:

Btw, this part of archive-contents.el:

    (setq archive--use-worktree
          (list
           (ignore-errors
             (zerop (call-process "git" nil nil nil "worktree" "--help"))))))

is sub-optimal: on MS-Windows, Git help commands invoke the system Web
browser to show the HTML version of the docs, so the browser suddenly
pops up when this code runs.

I actually don't understand why --help is the best idea here; is there
something wrong with running "worktree list" instead?  I tested that
with Git v2.3.5, which doesn't support worktrees, and it exited with
non-zero code.

bug#31118: 27.0.50; Can't load/compile websocket in 32bit master

[Stefan Monnier]

> I actually don't understand why --help is the best idea here; is there
> something wrong with running "worktree list" instead?

I use "--help" because I tried it and it worked for me.
Feel free to replace it with "list".


        Stefan

bug#31118: 27.0.50; Can't load/compile websocket in 32bit master

[Eli Zaretskii]

> From: Stefan Monnier <monnier@IRO.UMontreal.CA>
> Cc: eggert@cs.ucla.edu, 31118@debbugs.gnu.org
> Date: Wed, 11 Apr 2018 14:38:45 -0400
> 
> > I actually don't understand why --help is the best idea here; is there
> > something wrong with running "worktree list" instead?
> 
> I use "--help" because I tried it and it worked for me.
> Feel free to replace it with "list".

Done.