Using the GNU GMP Library for Bignums in Emacs

Using the GNU GMP Library for Bignums in Emacs

[Siraphob (Ben) Phipathananunth]

Emacs Calc was written many years ago, and as a result its current
implementation implements bignums purely in Emacs Lisp. Its bignum
operations also use a lot of hacks (such as performing carry
operations). Arbitrary precision arithmetic could be faster if Emacs
had GNU GMP linked to it, with the relevant Emacs Lisp functions added
in C.

What is the consensus on linking the GNU GMP library to Emacs so that
packages such as Emacs Calc (and others) could benefit from using
native types (i.e. "mpz_t") rather than reinventing the wheel?

The benefits of linking GNU GMP are clear. It would make it easier to
extend the math capabilities of Emacs, while making it faster and less
error-prone. However, the downsides, if any exist, should be discussed
as well, to gauge whether pursuing such a task would be fruitful.

Re: Using the GNU GMP Library for Bignums in Emacs

[Eli Zaretskii]

> From: "Siraphob (Ben) Phipathananunth" <siraben@disroot.org>
> Date: Sat, 21 Apr 2018 21:15:08 +0700
> 
> Emacs Calc was written many years ago, and as a result its current
> implementation implements bignums purely in Emacs Lisp. Its bignum
> operations also use a lot of hacks (such as performing carry
> operations). Arbitrary precision arithmetic could be faster if Emacs
> had GNU GMP linked to it, with the relevant Emacs Lisp functions added
> in C.
> 
> What is the consensus on linking the GNU GMP library to Emacs so that
> packages such as Emacs Calc (and others) could benefit from using
> native types (i.e. "mpz_t") rather than reinventing the wheel?

I think the consensus is we want that, it's just a matter of someone
doing the job of making it happen.

The design should IMO be discussed up front, because we want not only
to be able to use bignums and arbitrary-precision floating-point
numbers in C, but also in Lisp.  How exactly to expose them to Lisp is
something we should talk about, I think.

Thanks.

Re: Using the GNU GMP Library for Bignums in Emacs

[Siraphob (Ben) Phipathananunth]

One approach would be to implement a minimal subset of functions
exposed in Emacs Lisp that would allow one to recreate the following
functions that are defined in calc.el :

Function Name              Arguments
============================================
math-bignum                (a)
math-bignum-big            (a)
math-div10-bignum          (a)
math-scale-left-bignum     (a n)
math-scale-right-bignum    (a n)
math-add-bignum            (a b)
math-sub-bignum            (a b)
math-mul-bignum            (a b)
math-mul-bignum-digit      (a d c)
math-div-bignum            (a b)
math-div-bignum-digit      (a b)
math-div-bignum-big        (a b alen blen)
math-div-bignum-part       (a b blen)
math-div-bignum-try        (a b c guess)
math-format-bignum         (a)
math-format-bignum-decimal (a)
math-read-bignum           (s)

There are other functions scattered throughout the calc sources, but
this is the rough idea of the type of functions we would need. This
list would have to be shortened, as when we migrate bignums we would
no longer need the following functions (at least, could be more):

math-mul-bignum-digit      (a d c)
math-div-bignum-digit      (a b)

These functions exist because bignums are implemented as a list of
digits.

I think that keeping the function names the same would be good (for
some backwards compatibility). The /actual/ value of the bignum
(internally, in C) would be a tagged pointer to the mpz_t data type
stored somewhere so that it could be marked for GC (very important to
mark for GC).

With respect to floating points, though, things get a little hairy.

As quoted from the GMP Library
(https://gmplib.org/manual/Floating_002dpoint-Functions.html):

 > The exponent of each float has fixed precision, one machine word on
 > most systems. In the current implementation the exponent is a count of
 > limbs, so for example on a 32-bit system this means a range of roughly
 > 2^-68719476768 to 2^68719476736, or on a 64-bit system this will be
 > much greater. Note however that mpf_get_str can only return an
 > exponent which fits an mp_exp_t and currently mpf_set_str doesn’t
 > accept exponents bigger than a long.

Fortunately, it links to another project called MPFR
http://www.mpfr.org/sample.html which allows us to computing floating
points at arbitrary precision.

However, I'm concerned that adding two different libraries could lead
to issues regarding interoperability (for example, a floating point
number that needs to be converted to a bignum, and vice versa). If
anyone has used MPFR + GMP in the past, please chime in.

GC shouldn't be ignored as well, because as the name implies, these
data objects would be very large.

Thanks,

Siraphob (Ben) Phipathananunth

Eli Zaretskii wrote:
>> From: "Siraphob (Ben) Phipathananunth" <siraben@disroot.org>
>> Date: Sat, 21 Apr 2018 21:15:08 +0700
>>
>> Emacs Calc was written many years ago, and as a result its current
>> implementation implements bignums purely in Emacs Lisp. Its bignum
>> operations also use a lot of hacks (such as performing carry
>> operations). Arbitrary precision arithmetic could be faster if Emacs
>> had GNU GMP linked to it, with the relevant Emacs Lisp functions added
>> in C.
>>
>> What is the consensus on linking the GNU GMP library to Emacs so that
>> packages such as Emacs Calc (and others) could benefit from using
>> native types (i.e. "mpz_t") rather than reinventing the wheel?
> 
> I think the consensus is we want that, it's just a matter of someone
> doing the job of making it happen.
> 
> The design should IMO be discussed up front, because we want not only
> to be able to use bignums and arbitrary-precision floating-point
> numbers in C, but also in Lisp.  How exactly to expose them to Lisp is
> something we should talk about, I think.
> 
> Thanks.
> 

Re: Using the GNU GMP Library for Bignums in Emacs

[Paul Eggert]

Siraphob (Ben) Phipathananunth wrote:
> One approach would be to implement a minimal subset of functions
> exposed in Emacs Lisp that would allow one to recreate the following
> functions that are defined in calc.el :

Surely we should just add bignum support to existing functions +, -, etc. 
Wouldn't that suffice for 'calc'? If not, why not? (Of course 'calc' would need 
to be changed to exploit the bignums properly, no matter how we add bignums.)

> With respect to floating points, though, things get a little hairy. 

This should be a separate task. Bignums alone are quite a large-enough project. 
I'm not even sure we should do rationals.

> The /actual/ value of the bignum
> (internally, in C) would be a tagged pointer to the mpz_t data type
> stored somewhere so that it could be marked for GC

For bignums I would think that Emacs shouldn't use the mpz_t data type, as this 
would complicate garbage collection. Emacs can use the mp_limb_t data type and 
stick with the mpn_* functions.

Re: Using the GNU GMP Library for Bignums in Emacs

[Eli Zaretskii]

> Cc: emacs-devel@gnu.org
> From: Paul Eggert <eggert@cs.ucla.edu>
> Date: Sat, 21 Apr 2018 08:23:33 -0700
> 
> Siraphob (Ben) Phipathananunth wrote:
> > One approach would be to implement a minimal subset of functions
> > exposed in Emacs Lisp that would allow one to recreate the following
> > functions that are defined in calc.el :
> 
> Surely we should just add bignum support to existing functions +, -, etc. 

Exactly my thoughts.

> > With respect to floating points, though, things get a little hairy. 
> 
> This should be a separate task. Bignums alone are quite a large-enough project. 
> I'm not even sure we should do rationals.

Right.

Re: Using the GNU GMP Library for Bignums in Emacs

[Siraphob (Ben) Phipathananunth]

Paul Eggert wrote:
 > Surely we should just add bignum support to existing functions +, -,
 > etc. Wouldn't that suffice for 'calc'? If not, why not? (Of course
 > 'calc' would need to be changed to exploit the bignums properly, no
 > matter how we add bignums.)

Of course we could to do that. Hopefully there isn't existing
Emacs Lisp code that relies on unsafe arithmetic /anywhere/. If the
functions + - * / operate on bignums (instead of dedicated bignum
functions), would that mean we drop 32/64 bit integers entirely?

Eli Zaretskii wrote:
 > The design should IMO be discussed up front, because we want not only
 > to be able to use bignums and arbitrary-precision floating-point
 > numbers in C, but also in Lisp.

I thought that Eli was talking about how we should interface bignums
to Emacs Lisp; the + - * / operators are defined in C source
code. Bignums would decrease performance in areas where the usual
32/64 bit integers are sufficient, and lead to higher memory usage. It
would make much more sense to have separate math functions for 32/64
bit numbers and for bignums. In doing so, it should be obvious to the
Emacs Lisp programmer when to use what.

Paul Eggert wrote:
 > This should be a separate task. Bignums alone are quite a large-enough
 > project. I'm not even sure we should do rationals.

Rationals would be a part of Emacs Calc, once we have bignums it
should be trivial to reimplement rationals.

Re: Using the GNU GMP Library for Bignums in Emacs

[Eli Zaretskii]

> Cc: emacs-devel@gnu.org
> From: "Siraphob (Ben) Phipathananunth" <siraben@disroot.org>
> Date: Sat, 21 Apr 2018 22:40:05 +0700
> 
> I thought that Eli was talking about how we should interface bignums
> to Emacs Lisp; the + - * / operators are defined in C source
> code.

I never thought we could consider not using + - * etc. with bignums,
so I didn't even raise that issue.

> Bignums would decrease performance in areas where the usual
> 32/64 bit integers are sufficient, and lead to higher memory usage. It
> would make much more sense to have separate math functions for 32/64
> bit numbers and for bignums. In doing so, it should be obvious to the
> Emacs Lisp programmer when to use what.

When the arguments are bignums, these operators should automatically
invoke the relevant GMP functions.  That is how we behave elsewhere in
Emacs; Emacs Lisp does support polymorphism on the level of primitive
functions.  E.g., that's how these operators support both Lisp
integers and Lisp floats (and markers, for that matter).

Re: Using the GNU GMP Library for Bignums in Emacs

[Paul Eggert]

Siraphob (Ben) Phipathananunth wrote:

> Hopefully there isn't existing
> Emacs Lisp code that relies on unsafe arithmetic /anywhere/.

I'm afraid your hope will be in vain....

And it depends on what we mean by "unsafe". Is it safe to assume that (eq (1+ 0) 
1) returns t, for example? The Scheme standard says "no" but we might decide 
that 'eq' should "work" for fixnums in Emacs Lisp. That sort of thing.

> If the functions + - * / operate on bignums (instead of dedicated bignum
> functions), would that mean we drop 32/64 bit integers entirely?

No, it'd mean we'd still have fixnums vs bignums internally, and most programs 
wouldn't care whether an integer is represented via fixnum or bignum, as it'd be 
an issue merely of efficiency. Some programs would probably still care though 
(for efficiency reasons), and GNU Calc quite possibly would be one such program.

> It would make much more sense to have separate math functions for 32/64
> bit numbers and for bignums. In doing so, it should be obvious to the
> Emacs Lisp programmer when to use what.

That's not how other Lisps work, by and large. They have just one integer type 
and one set of functions. The goal is to simplify the job of writing a program.

Re: Using the GNU GMP Library for Bignums in Emacs

[Tom Tromey]

>>>>> "Paul" == Paul Eggert <eggert@cs.ucla.edu> writes:

Paul> And it depends on what we mean by "unsafe". Is it safe to assume that
Paul> (eq (1+ 0) 1) returns t, for example? The Scheme standard says "no"
Paul> but we might decide that 'eq' should "work" for fixnums in Emacs
Paul> Lisp. That sort of thing.

I would like the "egal" idea to be at least considered: that is, for
immutable objects like bignums (and floats), have "eq" do a value
comparison, not an identity comparison.  Ideally we could dispense with
eql entirely.

The upside is that this makes programs simpler to reason about -- to my
mind the eq/eql distinction is letting an implementation leak through
the abstraction.

The downside is that this is less efficient.  (Currently my JIT emits
"eq" as a simple comparison, which is nice; but I think on the whole I'd
personally rather trade a bit of performance for the cleaner semantics.)

Tom

Re: Using the GNU GMP Library for Bignums in Emacs

[Stefan Monnier]

> Paul> And it depends on what we mean by "unsafe". Is it safe to assume that
> Paul> (eq (1+ 0) 1) returns t, for example? The Scheme standard says "no"
> Paul> but we might decide that 'eq' should "work" for fixnums in Emacs
> Paul> Lisp. That sort of thing.
> I would like the "egal" idea to be at least considered: that is, for
> immutable objects like bignums (and floats), have "eq" do a value
> comparison, not an identity comparison.  Ideally we could dispense with
> eql entirely.

I think I agree, tho I'd describe it differently: the way `eq` is
defined in Common-Lisp, I believe it is valid to make it an alias for
`eql`.

I think Emacs should move in this direction: define `eq` to be the same
as `eql`, then check every use of EQ in the C code to see if it will
necessarily behave identically to "EQL" and where it doesn't, replace it
with EQL.

Obviously, this will cost us some code size and performance, so we'd
also want to try and measure the cost to make sure it's not too serious.


        Stefan

Re: Using the GNU GMP Library for Bignums in Emacs

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > I think Emacs should move in this direction: define `eq` to be the same
  > as `eql`, then check every use of EQ in the C code to see if it will
  > necessarily behave identically to "EQL" and where it doesn't, replace it
  > with EQL.

This is a lot of work and will slow down some loops.
I would rather try the obvious approach: eq treats bignums like floats.


-- 
Dr Richard Stallman
President, Free Software Foundation (https://gnu.org, https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)
Skype: No way! See https://stallman.org/skype.html.

Re: Using the GNU GMP Library for Bignums in Emacs

[Stefan Monnier]

>   > I think Emacs should move in this direction: define `eq` to be the same
>   > as `eql`, then check every use of EQ in the C code to see if it will
>   > necessarily behave identically to "EQL" and where it doesn't, replace it
>   > with EQL.
>
> This is a lot of work and will slow down some loops.

It's hard to tell without measuring it first.  And I'd expect that
many/most of those loops can be made fast anyway (e.g. member/memql can
delegate to `memq` when the argument is such that eq/eql/equal behave
identically).

> I would rather try the obvious approach: eq treats bignums like floats.

While I'm writing this within the GMP thread, it's orthogonal.


        Stefan

Re: Using the GNU GMP Library for Bignums in Emacs

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > Of course we could to do that. Hopefully there isn't existing
  > Emacs Lisp code that relies on unsafe arithmetic /anywhere/. If the
  > functions + - * / operate on bignums (instead of dedicated bignum
  > functions), would that mean we drop 32/64 bit integers entirely?

To eliminate the current types for small integers would
require rewriting of much of the C code in Emacs.
It would be better to represent small integers as now,
and have a different structure for larger integers.

-- 
Dr Richard Stallman
President, Free Software Foundation (https://gnu.org, https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)
Skype: No way! See https://stallman.org/skype.html.

Re: Using the GNU GMP Library for Bignums in Emacs

[dancol]

> [[[ To any NSA and FBI agents reading my email: please consider    ]]]
> [[[ whether defending the US Constitution against all enemies,     ]]]
> [[[ foreign or domestic, requires you to follow Snowden's example. ]]]
>
>   > Of course we could to do that. Hopefully there isn't existing
>   > Emacs Lisp code that relies on unsafe arithmetic /anywhere/. If the
>   > functions + - * / operate on bignums (instead of dedicated bignum
>   > functions), would that mean we drop 32/64 bit integers entirely?
>
> To eliminate the current types for small integers would
> require rewriting of much of the C code in Emacs.
> It would be better to represent small integers as now,
> and have a different structure for larger integers.
>

I'd love to see Emacs get transparent bigint support. Python semantics are
fine, as is using a normal int representation at the C level. Adding
transparent bigints as Lisp types doesn't require us to increase various
Emacs core limits right away.

Re: Using the GNU GMP Library for Bignums in Emacs

[Philipp Stephani]

[-- Attachment #1: Type: text/plain, Size: 1661 bytes --]

<dancol@dancol.org> schrieb am So., 22. Apr. 2018 um 04:49 Uhr:

> > [[[ To any NSA and FBI agents reading my email: please consider    ]]]
> > [[[ whether defending the US Constitution against all enemies,     ]]]
> > [[[ foreign or domestic, requires you to follow Snowden's example. ]]]
> >
> >   > Of course we could to do that. Hopefully there isn't existing
> >   > Emacs Lisp code that relies on unsafe arithmetic /anywhere/. If the
> >   > functions + - * / operate on bignums (instead of dedicated bignum
> >   > functions), would that mean we drop 32/64 bit integers entirely?
> >
> > To eliminate the current types for small integers would
> > require rewriting of much of the C code in Emacs.
> > It would be better to represent small integers as now,
> > and have a different structure for larger integers.
> >
>
> I'd love to see Emacs get transparent bigint support. Python semantics are
> fine, as is using a normal int representation at the C level. Adding
> transparent bigints as Lisp types doesn't require us to increase various
> Emacs core limits right away.
>
>
>
We need to be very careful with transparent bigint support though, as a
naive design will break many invariants. For example, integers are
currently documented to use modular arithmetic (
https://www.gnu.org/software/emacs/manual/html_node/elisp/Integer-Type.html
,
https://www.gnu.org/software/emacs/manual/html_node/elisp/Integer-Basics.html),
and they are documented to be pure value types, i.e. same-valued integers
are the same object (
https://www.gnu.org/software/emacs/manual/html_node/elisp/Equality-Predicates.html).
Especially the second property is widely used.

[-- Attachment #2: Type: text/html, Size: 2354 bytes --]

Re: Using the GNU GMP Library for Bignums in Emacs

[Paul Eggert]

Philipp Stephani wrote:
> integers are currently documented to use modular arithmetic (

We'll need to change the documentation for modular arithmetic, since integers 
will no longer overflow.

> and they are documented to be pure value types, i.e. same-valued integers
> are the same objects. Especially the second property is widely used.

This property is used only for fixnums now (since bignums do not exist yet), and 
we can keep this property for fixnums. We could also keep it for bignums, by 
hashing all bignums, though I doubt whether it's worth the expense.

Re: Using the GNU GMP Library for Bignums in Emacs

[Daniel Colascione]

On 04/22/2018 10:43 AM, Paul Eggert wrote:
> Philipp Stephani wrote:
>> integers are currently documented to use modular arithmetic (
> 
> We'll need to change the documentation for modular arithmetic, since 
> integers will no longer overflow.

Bugs will inevitably arise. One possibility is just dealing with them; 
another might be to predicate bignum support on a lexical-binding-like flag.

>> and they are documented to be pure value types, i.e. same-valued integers
>> are the same objects. Especially the second property is widely used.
> 
> This property is used only for fixnums now (since bignums do not exist 
> yet), and we can keep this property for fixnums. We could also keep it 
> for bignums, by hashing all bignums, though I doubt whether it's worth 
> the expense.

That's a good idea, IMHO. It preserves identity without complicating eq 
itself.

Re: Using the GNU GMP Library for Bignums in Emacs

[Clément Pit-Claudel]

On 2018-04-22 14:04, Daniel Colascione wrote:
> On 04/22/2018 10:43 AM, Paul Eggert wrote:
>> Philipp Stephani wrote:
>>> integers are currently documented to use modular arithmetic (
>>
>> We'll need to change the documentation for modular arithmetic, since integers will no longer overflow.
> 
> Bugs will inevitably arise. One possibility is just dealing with them; another might be to predicate bignum support on a lexical-binding-like flag.

Or we could make them different types: the usual operators on fixnums would return fixnums, and the usual operators on bignums would return bignums. Mixing both would always return a bignum.

The difficulty is whether we should change existing Emacs function to return bignums.

Clément.

Re: Using the GNU GMP Library for Bignums in Emacs

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > https://www.gnu.org/software/emacs/manual/html_node/elisp/Integer-Basics.html),
  > and they are documented to be pure value types, i.e. same-valued integers
  > are the same object (
  > https://www.gnu.org/software/emacs/manual/html_node/elisp/Equality-Predicates.html).
  > Especially the second property is widely used.

This will continue to be true for all the integers that are currently
representable.  If we document that, maybe we won't have a lot of
places to fix.

  > Or we could make them different types: the usual operators on
  > fixnums would return fixnums, and the usual operators on bignums
  > would return bignums. Mixing both would always return a bignum.

Having two data types for integers would be a nuisance for many Lisp
programs.  It won't simplify anything, and it won't reduce the changes
needed in Lisp programs because of bignums.

-- 
Dr Richard Stallman
President, Free Software Foundation (https://gnu.org, https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)
Skype: No way! See https://stallman.org/skype.html.

Re: Using the GNU GMP Library for Bignums in Emacs

[Siraphob (Ben) Phipathananunth]

Richard Stallman wrote:
 > To eliminate the current types for small integers would
 > require rewriting of much of the C code in Emacs.
 > It would be better to represent small integers as now,
 > and have a different structure for larger integers.

 From what I understand, we would want to use fixnums by default in the
C code, and convert to bignums automatically (in lisp) when the number
exceeds the range of a fixnum, while retaining behavior as before,
using the regular math operations + - * / (and more) to interface this
to Lisp.

Paul Eggert wrote:
 > programs wouldn't care whether an integer is represented via fixnum or
 > bignum, as it'd be an issue merely of efficiency.

Would it not slow down computation to have to constantly convert
between the two types? (especially if the computation is switching
above/below the fixnum/bignum boundary). In such a case, a fix could
be to convert lisp numbers exceeding fixnum limits to bignums for the
rest of the number's life (until GC). This ensures memory usage is kept
low for fixnum computations.

Re: Using the GNU GMP Library for Bignums in Emacs

[Paul Eggert]

Siraphob (Ben) Phipathananunth wrote:
> Would it not slow down computation to have to constantly convert
> between the two types?

A bit perhaps, but it shouldn't be that big a deal. Most integer computation 
will likely be fixnum only, and the only slowdown there will be integer overflow 
checking that we currently aren't doing. With decent hardware and compiler, I'd 
guess this would cost us three machine instructions per Lisp arithmetic 
operation, including the conditional branch that is typically not taken. Hardly 
anybody will notice.
> In such a case, a fix could
> be to convert lisp numbers exceeding fixnum limits to bignums for the
> rest of the number's life (until GC). This ensures memory usage is kept
> low for fixnum computations.

Yes, the idea is to use bignums to represent numbers outside of fixnum range, 
and to use fixnums to represent numbers inside fixnum range. Bignums require 
garbage collection; fixnums do not.

Re: Using the GNU GMP Library for Bignums in Emacs

[Helmut Eller]

On Sun, Apr 22 2018, Paul Eggert wrote:

> Most integer
> computation will likely be fixnum only, and the only slowdown there
> will be integer overflow checking that we currently aren't doing. With
> decent hardware and compiler, I'd guess this would cost us three
> machine instructions per Lisp arithmetic operation, including the
> conditional branch that is typically not taken. Hardly anybody will
> notice.

Out of curiousity: what's the quickest way to detect overflow on
multiplication in ANSI C?  E.g.

  int64_t x, y, result;
  result = x * y;
  if (<how to detect overflow?>)

Helmut

Re: Using the GNU GMP Library for Bignums in Emacs

[Andreas Schwab]

On Apr 23 2018, Helmut Eller <eller.helmut@gmail.com> wrote:

> Out of curiousity: what's the quickest way to detect overflow on
> multiplication in ANSI C?  E.g.
>
>   int64_t x, y, result;
>   result = x * y;
>   if (<how to detect overflow?>)

There is no way to detect overflow after the fact because overflow
invokes undefined behaviour.  You need to either check the range
beforehand, or use special builtins offered by the compiler
(eg. __builtin_smul_overflow in GCC).

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510  2552 DF73 E780 A9DA AEC1
"And now for something completely different."

Re: Using the GNU GMP Library for Bignums in Emacs

[Paul Eggert]

Andreas Schwab wrote:
> There is no way to detect overflow after the fact because overflow
> invokes undefined behaviour.  You need to either check the range
> beforehand, or use special builtins offered by the compiler
> (eg. __builtin_smul_overflow in GCC).

All true, and Emacs lib/intprops.h has an INT_MULTIPLY_OVERFLOW macro that 
arranges for all that. On the x86-64 with GCC, it costs one additional 
instruction (typically a conditional branch that is not taken) to check for 
overflow in machine-word integer multiplication. Checking for fixnum overflow 
(as opposed to machine-word overflow) requires one more conditional branch after 
some quick bit-twiddling.

Re: Using the GNU GMP Library for Bignums in Emacs

[Helmut Eller]

Paul Eggert wrote:

> Andreas Schwab wrote:
>> There is no way to detect overflow after the fact because overflow
>> invokes undefined behaviour.  You need to either check the range
>> beforehand, or use special builtins offered by the compiler
>> (eg. __builtin_smul_overflow in GCC).
>
> All true, and Emacs lib/intprops.h has an INT_MULTIPLY_OVERFLOW macro
> that arranges for all that. On the x86-64 with GCC, it costs one
> additional instruction (typically a conditional branch that is not
> taken) to check for overflow in machine-word integer
> multiplication.

If I read the code in in data.c correctly, than Emacs uses the
INT_MULTIPLY_WRAPV macro which boils down to the __builtin_mul_overflow.
Which indeed produces nice machine code.  But the original question was
about ANSI C, which seems to require a division and 3 conditional jumps
for the range check (with gcc 6).

> Checking for fixnum overflow (as opposed to machine-word overflow)
> requires one more conditional branch after some quick bit-twiddling.

Does INT_MULTIPLY_WRAPV macro even perform fixnum overflow tests?

Anyway, I find it curios that the following two expression yield
different values:

(* (* most-positive-fixnum 2) 1.0) => -2.0

(* most-positive-fixnum 2 1.0) => 4.611686018427388e+18

Helmut

Re: Using the GNU GMP Library for Bignums in Emacs

[Paul Eggert]

On 04/23/2018 12:22 PM, Helmut Eller wrote:
> If I read the code in in data.c correctly, than Emacs uses the
> INT_MULTIPLY_WRAPV macro which boils down to the __builtin_mul_overflow.
> Which indeed produces nice machine code.  But the original question was
> about ANSI C, which seems to require a division and 3 conditional jumps
> for the range check (with gcc 6).

Yes, in general INT_MULTIPLY_WRAPV does require a division on platforms 
that do not support overflow-checking builtins. In practice, though, 
this tends to not be an issue, as the major compilers are moving in the 
direction of supporting such builtins.

> Does INT_MULTIPLY_WRAPV macro even perform fixnum overflow tests?

No, it checks only for machine-level overflow. We could come up with a 
variant that does both machine-level and fixnum-level checking 
simultaneously; I'm not sure it's worth the trouble, though, as most 
arithmetic operations are addition and subtraction and for these, 
fixnum-level checking suffixes.

> Anyway, I find it curios that the following two expression yield
> different values:
>
> (* (* most-positive-fixnum 2) 1.0) => -2.0
>
> (* most-positive-fixnum 2 1.0) => 4.611686018427388e+18

Cool, huh? That's done on purpose; see this Emacs commit:

https://git.savannah.gnu.org/cgit/emacs.git/commit/?id=0ae6bdee0085b05028108325b0a4ce979eadb24e

The idea is to produce a floating-point answer that is more 
mathematically-correct, while sticking to Common Lisp-style rules for 
contagion and argument processing.

Re: Using the GNU GMP Library for Bignums in Emacs

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  >  From what I understand, we would want to use fixnums by default in the
  > C code, and convert to bignums automatically (in lisp) when the number
  > exceeds the range of a fixnum, while retaining behavior as before,
  > using the regular math operations + - * / (and more) to interface this
  > to Lisp.

Yes, exactly.

  > Would it not slow down computation to have to constantly convert
  > between the two types? (especially if the computation is switching
  > above/below the fixnum/bignum boundary). In such a case, a fix could
  > be to convert lisp numbers exceeding fixnum limits to bignums for the
  > rest of the number's life (until GC). This ensures memory usage is kept
  > low for fixnum computations.

I am not sure that makes sense.  Emacs works with numbers, and
computes (or otherwise generates) new numbers, but it never changes an
existing number.

Numbers that are in the range of non-bignum integers MUST be
represented as non-bignum integers so that the existing C code can
work on them without change.  Some places ought to be able to handle
larger numbers, and we will need to change them,  However, many places
could continue to handle only small numbers.

-- 
Dr Richard Stallman
President, Free Software Foundation (https://gnu.org, https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)
Skype: No way! See https://stallman.org/skype.html.

Re: Using the GNU GMP Library for Bignums in Emacs

[Helmut Eller]

On Sat, Apr 21 2018, Paul Eggert wrote:

> Surely we should just add bignum support to existing functions +, -,
> etc.

I'm not so sure of that.  E.g. (format "%x" -1) => "3fffffffffffffff".
That doesn't make any sense on if -1 is a bignum.

Helmut

Re: Using the GNU GMP Library for Bignums in Emacs

[Paul Eggert]

Helmut Eller wrote:
> (format "%x" -1) => "3fffffffffffffff".

Any program that assumes that behavior is already unportable, since the 
expression returns "3fffffff" on 32-bit platforms.

The usual approach for this is to format negative numbers with a leading minus 
sign, so that (format "%x" -1) returns "-1". This is what Emac Lisp should have 
been doing anyway as it is more intuitive and more portable. Obviously there 
will be compatibility concerns here, and we'll need to address them.

Re: Using the GNU GMP Library for Bignums in Emacs

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > The usual approach for this is to format negative numbers with a leading minus 
  > sign, so that (format "%x" -1) returns "-1". This is what Emac Lisp should have

I think that change is likely to break things.  If you WANT a hex
string, you probably want it to be all hex, regardless of value.

I would expect that each use of %x probably has a certain number of bits
in mind.  Perhaps %x should take a number of bits and output modularly.
So %24x would output 24 bits' worth.  Always unsigned.




-- 
Dr Richard Stallman
President, Free Software Foundation (https://gnu.org, https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)
Skype: No way! See https://stallman.org/skype.html.

Re: Using the GNU GMP Library for Bignums in Emacs

[Paul Eggert]

Richard Stallman wrote:
>    > The usual approach for this is to format negative numbers with a leading minus
>    > sign, so that (format "%x" -1) returns "-1". This is what Emac Lisp should have
> 
> I think that change is likely to break things.  If you WANT a hex
> string, you probably want it to be all hex, regardless of value.

No doubt people occasionally want C-like formatting (i.e., print a negative 
number modulo 2**W where W is the word width), but once we have bignums the 
notion of the "word width" becomes dubious, and in practice it's cleaner and 
more useful to print hexadecimal integers in the usual mathematical way. That's 
the tradition in Common Lisp, in Python, and in every other language I know that 
has bignums and has the %x format or something similar.

> I would expect that each use of %x probably has a certain number of bits
> in mind.  Perhaps %x should take a number of bits and output modularly.
> So %24x would output 24 bits' worth.

We could invent a special syntax along those lines, though it would need to 
differ from "%24x" which already means pad with spaces to 24 characters. 
However, there's little point to a special syntax since one can easily achieve 
the same effect without it. For example, to format the low-order 24 bits of N, 
one can already use (format "%x" (logand N #xffffff)) and this will continue to 
work even if bignums are introduced.

Re: Using the GNU GMP Library for Bignums in Emacs

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > No doubt people occasionally want C-like formatting (i.e., print a
  > negative number modulo 2**W where W is the word width), but once
  > we have bignums the notion of the "word width" becomes dubious,

In a theoretical sense, yes -- but I doubt that that matters in
practice.  In practice, I think, the cases where one wants hexadecimal
are cases where there is a word width.

  > and in practice it's cleaner and more useful to print hexadecimal
  > integers in the usual mathematical way.

Would you tell me about some of the cases where you wanted hexadecimal
output and you wanted negative numbers to use a minus sign?

I ask because I am skeptical that such cases exist.

-- 
Dr Richard Stallman
President, Free Software Foundation (https://gnu.org, https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)
Skype: No way! See https://stallman.org/skype.html.

Re: Using the GNU GMP Library for Bignums in Emacs

[Paul Eggert]

Richard Stallman wrote:
> In practice, I think, the cases where one wants hexadecimal
> are cases where there is a word width.

Although that is true in traditional usage where word width is assumed, it's not 
true for most applications using bignums, the original context of this thread. 
In such applications, it's routine to have hexadecimal numbers that are wider 
than 32- or 64-bit words. And these numbers (when printed without a sign) are 
routinely treated as nonnegative, not as negative. For an example of this sort 
of application, see the OpenSSL C code that deals with bignums: e.g., BN_bn2hex 
generates a leading "-" when converting a negative number to a hexadecimal string.

Even in the standard C library (which lacks bignums), the %x printf format is 
supposed to be used only with unsigned integers. Nowadays GCC even optionally 
warns about using %x on signed integers.

> The reason to invent a special syntax is to get the commonly desired
> result in a simpler way.

If it turns out to be useful to have a shorthand printf format for formatting 
the least N bits of an argument, we can add it as needed. I have my doubts, 
though, as other bignum formatters seem to get along fine without such a feature.

Re: Using the GNU GMP Library for Bignums in Emacs

[Helmut Eller]

On Mon, Apr 23 2018, Paul Eggert wrote:

> Even in the standard C library (which lacks bignums), the %x printf
> format is supposed to be used only with unsigned integers. Nowadays
> GCC even optionally warns about using %x on signed integers.

Maybe %x should then print the (shortest) two's complement
representation for bignums.  In Common Lisp:

(defun 2comp (x)
  (format nil "~x" (logand x (1- (ash 1 (+ 1 (integer-length x)))))))

(2comp -1) => "1"
(2comp -2) => "2"
(2comp -3) => "5"
(2comp -15) => "11"
(2comp -16) => "10"

In practice, only integers smaller than most-negative-fixnum would use
this rule.

Helmut

Re: Using the GNU GMP Library for Bignums in Emacs

[Jefferson Carpenter]

On 4/24/2018 5:45 AM, Helmut Eller wrote:
> On Mon, Apr 23 2018, Paul Eggert wrote:
> 
>> Even in the standard C library (which lacks bignums), the %x printf
>> format is supposed to be used only with unsigned integers. Nowadays
>> GCC even optionally warns about using %x on signed integers.
> 
> Maybe %x should then print the (shortest) two's complement
> representation for bignums.  In Common Lisp:
> 
> (defun 2comp (x)
>    (format nil "~x" (logand x (1- (ash 1 (+ 1 (integer-length x)))))))
> 
> (2comp -1) => "1"
> (2comp -2) => "2"
> (2comp -3) => "5"
> (2comp -15) => "11"
> (2comp -16) => "10"
> 
> In practice, only integers smaller than most-negative-fixnum would use
> this rule.
> 
> Helmut
> 
> 

I don't think that would make sense, because you can't tell whether a 
given output string was generated from a positive or a negative number.

For instance, (2comp -6) = (2comp 10) = "A".

"Two's complement" isn't a representation of integers; "N-bit two's 
complement" is a representation of integers from -2^N to 2^N-1, for 
positive integer N.

So, I think that function should take another argument, the number of 
bits `n` to output.

Jefferson

Re: Using the GNU GMP Library for Bignums in Emacs

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > In such applications, it's routine to have hexadecimal numbers that are wider 
  > than 32- or 64-bit words.

What is the motive for using hexadecimal in a case like that?

  > Even in the standard C library (which lacks bignums), the %x printf format is 
  > supposed to be used only with unsigned integers.

I think that's a misleading statement of what it does in C.  The rule
is to use it with an unsigned _type_.  The type, not the value, is
supposed to be unsigned.

In practice, what this means is that the hex output is done _treating
the number as unsigned_.  So -1, which is 0xffffffff as a 32-bit int,
will print as ffffffff, not as -1.

I contend that we want the same behavior in Emacs Lisp, too.

  > If it turns out to be useful to have a shorthand printf format for formatting 
  > the least N bits of an argument, we can add it as needed. I have my doubts, 
  > though, as other bignum formatters seem to get along fine without such a feature.

What conclusions we can draw from them, about Emacs Lisp, depends on
the number of bits in a non-bignum integer for each of them.  In
Emacs, 0xffffffff will be a bignum.  Is it a bignum in those other
systems?  If they can treat 32-bit ints as short, many programs that
want to use hex will never operate on a bignum.

-- 
Dr Richard Stallman
President, Free Software Foundation (https://gnu.org, https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)
Skype: No way! See https://stallman.org/skype.html.

Re: Using the GNU GMP Library for Bignums in Emacs

[Paul Eggert]

On 04/24/2018 06:05 PM, Richard Stallman wrote:
>    > In such applications, it's routine to have hexadecimal numbers that are wider
>    > than 32- or 64-bit words.
>
> What is the motive for using hexadecimal in a case like that?

It varies. One motivation is a desire to communicate a number via text 
more-efficiently than base 10 would provide.

>    > Even in the standard C library (which lacks bignums), the %x printf format is
>    > supposed to be used only with unsigned integers.
>
> I think that's a misleading statement of what it does in C.  The rule
> is to use it with an unsigned _type_.  The type, not the value, is
> supposed to be unsigned.

In C, if the type is unsigned then the corresponding value is 
nonnegative. That is, in C there is no such thing as a negative value 
with an unsigned type. The %x format is supposed to be used only with 
unsigned types, i.e., only with nonnegative values.

> I contend that we want the same behavior in Emacs Lisp, too.

I certainly wouldn't want the behavior you suggest. Among other things, 
Emacs Lisp does not have unsigned types, and I'd rather not introduce 
such a concept into the language as it'd be needless complexity and an 
unnecessary divergence from other Lisps.

> In Emacs, 0xffffffff will be a bignum.

Whether 0xffffffff is a bignum will depend on the platform. On my 64-bit 
Emacs, 0xffffffff already is supported as a fixnum, and that wouldn't 
change if bignums were introduced to Emacs.

> Is it a bignum in those other systems?

It depends on the system, I expect (just as it would in Emacs).

Re: Using the GNU GMP Library for Bignums in Emacs

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > In C, if the type is unsigned then the corresponding value is 
  > nonnegative. That is, in C there is no such thing as a negative value 
  > with an unsigned type.

I know that (of course).  It looks like we are miscommunicating.

  > I certainly wouldn't want the behavior you suggest. Among other things, 
  > Emacs Lisp does not have unsigned types, and I'd rather not introduce 
  > such a concept into the language

I think you have misunderstood what I suggest.  I am talking about changing
the behavior of %x, nothing else.

I responded to a statement about what happens in C.  Did you misread that
as a proposal to change Emacs?

  > > In Emacs, 0xffffffff will be a bignum.

  > Whether 0xffffffff is a bignum will depend on the platform. On my 64-bit 
  > Emacs, 0xffffffff already is supported as a fixnum, and that wouldn't 
  > change if bignums were introduced to Emacs.

You're right about 64-bit systems.  I often forget they exist, since
I never use them.

My argument is valid nonetheless, because that quantity is a bignum on
_some_ systems.

If we were to support _only_ 64-but systems, this argument would cease
to be valid.

-- 
Dr Richard Stallman
President, Free Software Foundation (https://gnu.org, https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)
Skype: No way! See https://stallman.org/skype.html.

Re: Using the GNU GMP Library for Bignums in Emacs

[Paul Eggert]

On 04/25/2018 03:40 PM, Richard Stallman wrote:
> If we were to support_only_  64-bit systems, this argument would cease
> to be valid.

That is easy to arrange; just configure --with-wide-int. This causes 
Emacs to use 64-bit words on all platforms (or wider, at least in 
theory). If we made --with-wide-int the default (which I'm more and more 
thinking would be a good thing, for this and other reasons), then 
#xffffffff would work and would format the same on all platforms by 
default, and this would happen regardless of whether we also add bignums 
to Emacs.

However, I don't see why assuming 64 bits would mean that the argument 
would cease to be valid. Even with that assumption, we would continue 
have the same problem with numbers like #xffffffffffffffff that exceed 
Emacs fixnum range when words are 64 bits.

> I responded to a statement about what happens in C.  Did you misread that
> as a proposal to change Emacs?
Yes, because after your response (which talked about what happens in C), 
you wrote "I contend that we want the same behavior in Emacs Lisp, too." 
Sorry if I misunderstood you.

I think part of the problem here, is that once we have bignums then 
there will no problem representing unsigned values that are too large to 
fit into (say) 30-bit signed integers, so there will be no problem in 
having %x treat integers like Common Lisp and Python do: programs that 
want to do I/O of numbers as hexadecimal strings will be able to read 
and print them with %x and they'll all work just fine even if the 
numbers contain 8 or 16 or more hexadecimal digits. The problem arises 
in Emacs Lisp only because it is limited to fixnums now.

Re: Using the GNU GMP Library for Bignums in Emacs

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > > If we were to support_only_  64-bit systems, this argument would cease
  > > to be valid.

  > That is easy to arrange; just configure --with-wide-int. This causes 
  > Emacs to use 64-bit words on all platforms

That may be correct, but it doesn't respond to the point at hand.
The point at hand is that Emacs does support 32-bit Lisp objecs in
some cases. and that a 32-bit number _in 32-bit builds_ will not
be a fixnum.

If you're suggesting that I personally change to a 64-bit build, that
doesn't respond to the issue at hand.  It's not about what happens for
me in particular.

If you're suggesting to eliminate support for 32-bit builds,
I agree that that would eliminate the issue.  But I think that change
would have substantial disadvantages.

  > However, I don't see why assuming 64 bits would mean that the argument 
  > would cease to be valid. Even with that assumption, we would continue 
  > have the same problem with numbers like #xffffffffffffffff that exceed 
  > Emacs fixnum range when words are 64 bits.

I expect that %x is used only for values meant to interact
with the operating system in certain ways, and that _most of_ those
values will never be more than 32 bits.

I could be mistaken in this, but it is a factual question.
Where else do people use %x?


-- 
Dr Richard Stallman
President, Free Software Foundation (https://gnu.org, https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)
Skype: No way! See https://stallman.org/skype.html.

Re: Using the GNU GMP Library for Bignums in Emacs

[Michael Welsh Duggan]

Richard Stallman <rms@gnu.org> writes:

>   > However, I don't see why assuming 64 bits would mean that the argument 
>   > would cease to be valid. Even with that assumption, we would continue 
>   > have the same problem with numbers like #xffffffffffffffff that exceed 
>   > Emacs fixnum range when words are 64 bits.
>
> I expect that %x is used only for values meant to interact
> with the operating system in certain ways, and that _most of_ those
> values will never be more than 32 bits.
>
> I could be mistaken in this, but it is a factual question.
> Where else do people use %x?

Well, although I admit I most often use hexadecimal numbers to represent
machine addresses (including on 64-bit addresses, of course), I also use
them to represent memory offsets.  I only bring this up because this is
a case when a negative hexadecimal number (-#x1000 or -0x1000) is a more
useful representation than two's complement.

I admit that this may be considered a niche use, however, and I am
seldom working with offsets of sizes greater than 32 bits in size.  I
will further admit that I have never needed this in Emacs, but have run
into this problem with writing C and C++ code, since I have been doing a
lot of work lately on static binary analysis.

-- 
Michael Welsh Duggan
(md5i@md5i.com)

Re: Using the GNU GMP Library for Bignums in Emacs

[Stefan Monnier]

> a case when a negative hexadecimal number (-#x1000 or -0x1000) is
                                             ^^^^^^^
                                             #x-1000

-- Stefan

Re: Using the GNU GMP Library for Bignums in Emacs

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > Well, although I admit I most often use hexadecimal numbers to represent
  > machine addresses (including on 64-bit addresses, of course), I also use
  > them to represent memory offsets.

I think that implies we need a way to specify for %x to use the proper
width for an address on the machine in use.

  >   I only bring this up because this is
  > a case when a negative hexadecimal number (-#x1000 or -0x1000) is a more
  > useful representation than two's complement.

If it's useful in Emacs, we should support that.  But we may as well take
stock of the need before deciding whether to implement this.

-- 
Dr Richard Stallman
President, Free Software Foundation (https://gnu.org, https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)
Skype: No way! See https://stallman.org/skype.html.

Re: Using the GNU GMP Library for Bignums in Emacs

[Paul Eggert]

Richard Stallman wrote:
> I expect that %x is used only for values meant to interact
> with the operating system in certain ways

Hexadecimal is used for many other purposes. For example, the shell command 'git 
log' at the top level of Emacs currently outputs a first line containing the 
hexadecimal string bca6c4348077c8c0b368503b16378867b6d49659 which represents an 
integer containing 160 bits, the integer width of the SHA-1 checksums used by 
Git. Although Emacs cannot now process such a number directly, with bignums it 
will be able to and %x is the natural way to format such numbers.

Re: Using the GNU GMP Library for Bignums in Emacs

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > log' at the top level of Emacs currently outputs a first line containing the 
  > hexadecimal string bca6c4348077c8c0b368503b16378867b6d49659 which represents an 
  > integer containing 160 bits, the integer width of the SHA-1 checksums used by 
  > Git. Although Emacs cannot now process such a number directly, with bignums it 
  > will be able to and %x is the natural way to format such numbers.

If the first 20 bits were zero, would you want the output to start
with 00000, or would you want it to be shorter?

-- 
Dr Richard Stallman
President, Free Software Foundation (https://gnu.org, https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)
Skype: No way! See https://stallman.org/skype.html.

Re: Using the GNU GMP Library for Bignums in Emacs

[Paul Eggert]

On 04/30/2018 08:01 PM, Richard Stallman wrote:
> If the first 20 bits were zero, would you want the output to start
> with 00000, or would you want it to be shorter?

For Git checksums I'd want leading zeros without other decorations like 
#x, since that's how Git itself prints those numbers and I'd prefer to 
be compatible.

Re: Using the GNU GMP Library for Bignums in Emacs

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > For Git checksums I'd want leading zeros without other decorations like 
  > #x, since that's how Git itself prints those numbers and I'd prefer to 
  > be compatible.

That means the feature would need to specify the number of hex digits
desired.

Is that the case for all use of hex output?
Is there any counterexample?

-- 
Dr Richard Stallman
President, Free Software Foundation (https://gnu.org, https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)
Skype: No way! See https://stallman.org/skype.html.

Re: Using the GNU GMP Library for Bignums in Emacs

[Paul Eggert]

Richard Stallman wrote:
> That means the feature would need to specify the number of hex digits
> desired.

We already have such a feature. For example, (format "%09x" 257) returns
"000000101", which is 257 hexadecimal, expressed using leading zeros as needed 
so that at least 9 digits are used. Once Emacs supports bignums, the same 
feature can print 40-hex-digit SHA-1 checksums by using (format "%040x" n).

Re: Using the GNU GMP Library for Bignums in Emacs

[Helmut Eller]

On Wed, May 02 2018, Paul Eggert wrote:

> Richard Stallman wrote:
>> That means the feature would need to specify the number of hex digits
>> desired.
>
> We already have such a feature. For example, (format "%09x" 257) returns
> "000000101", which is 257 hexadecimal, expressed using leading zeros
> as needed so that at least 9 digits are used. Once Emacs supports
> bignums, the same feature can print 40-hex-digit SHA-1 checksums by
> using (format "%040x" n).

I think the bitwise operations and the resulting negative numbers are
the problematic part:

E.g.

(format "%016x" (lognot 257)) => "3ffffffffffffefe"

but the 16-hex-digit string should be "fffffffffffffefe".

Helmut

Re: Using the GNU GMP Library for Bignums in Emacs

[Eli Zaretskii]

> From: Helmut Eller <eller.helmut@gmail.com>
> Date: Thu, 03 May 2018 08:26:05 +0200
> Cc: Paul Eggert <eggert@cs.ucla.edu>, rms@gnu.org
> 
> I think the bitwise operations and the resulting negative numbers are
> the problematic part:
> 
> E.g.
> 
> (format "%016x" (lognot 257)) => "3ffffffffffffefe"
> 
> but the 16-hex-digit string should be "fffffffffffffefe".

But a 16-hex-digit fffffffffffffefe is not a fixnum, right?

Re: Using the GNU GMP Library for Bignums in Emacs

[Paul Eggert]

On 05/03/2018 10:49 AM, Eli Zaretskii wrote:
>> (format "%016x" (lognot 257)) => "3ffffffffffffefe"
>>
>> but the 16-hex-digit string should be "fffffffffffffefe".
> But a 16-hex-digit fffffffffffffefe is not a fixnum, right?

Exactly right. If we have bignums, this will be straightforward; it'll 
work like Common Lisp, Python, etc. and it'll be obvious what to do. The 
problems mentioned in this thread mainly come from the fact that Emacs 
does not have bignums now and that some code may depend on fixnum 
overflow wrapping around.

Here's another way to think about it. (format "%016x" (lognot 257)) is 
already nonportable, as it yields "000000003ffffefe" on 32-bit platforms 
and "3ffffffffffffefe" on 64-bit platforms. Once we add bignums, the 
same expression should yield "-000000000000102" on all platforms. Code 
intended to be portable between old and new regimes already needs to 
worry about nonportability in this area, and it shouldn't be hard to 
adjust (if it needs adjusting at all). Code intended to run only under 
the new regime will be able to safely assume the expression yields 
"-000000000000102" on all platforms, regardless of whether the 
underlying platform is 32- or 64- or some other number of bits, and in 
the long run this will reduce the number of portability issues in Elisp 
code.

Re: Using the GNU GMP Library for Bignums in Emacs

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

I don't think people want (format "%x" (lognot 257)) to output "-102".
Not on any platform.  It isn't what code expects, and it isn't useful.

Perhaps, with bignums,  lognot  should take a width operand, measured in bits.
So (lognot 257 32) would produce 0xfffffefe (a positive number)
and (lognot 257 64) would produce 0xfffffffffffffefe (a positive number).

Another idea: %Nx with a negative number should output its 2's
complement representation in 4N bits.

-- 
Dr Richard Stallman
President, Free Software Foundation (https://gnu.org, https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)
Skype: No way! See https://stallman.org/skype.html.

Re: Using the GNU GMP Library for Bignums in Emacs

[Ken Raeburn]

On May 4, 2018, at 00:26, Richard Stallman <rms@gnu.org> wrote:
> 
> [[[ To any NSA and FBI agents reading my email: please consider    ]]]
> [[[ whether defending the US Constitution against all enemies,     ]]]
> [[[ foreign or domestic, requires you to follow Snowden's example. ]]]
> 
> I don't think people want (format "%x" (lognot 257)) to output "-102".
> Not on any platform.  It isn't what code expects, and it isn't useful.
> 
> Perhaps, with bignums,  lognot  should take a width operand, measured in bits.
> So (lognot 257 32) would produce 0xfffffefe (a positive number)
> and (lognot 257 64) would produce 0xfffffffffffffefe (a positive number).
> 
> Another idea: %Nx with a negative number should output its 2's
> complement representation in 4N bits.

That's already taken: A leading “-“ means to left-justify the output if the specified width is wider than required.

ELISP> (format "%-16x" 300)
"12c             "
ELISP> (format "%16x" 300)
"             12c"
ELISP> 

Re: Using the GNU GMP Library for Bignums in Emacs

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > > Another idea: %Nx with a negative number should output its 2's
  > > complement representation in 4N bits.

  > That's already taken: A leading “-“ means to left-justify the output if the specified width is wider than required.

We are miscommunicating.  I'm talking about what %Nx should do when
outputting an argument that is negative, as in (format "%16x" -63).

Sorry that wasn't clear.

You could try p/x -63 in GDB to see what I mean.

-- 
Dr Richard Stallman
President, Free Software Foundation (https://gnu.org, https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)
Skype: No way! See https://stallman.org/skype.html.

Re: Using the GNU GMP Library for Bignums in Emacs

[Ken Raeburn]

On May 5, 2018, at 23:12, Richard Stallman <rms@gnu.org> wrote:
> 
> [[[ To any NSA and FBI agents reading my email: please consider    ]]]
> [[[ whether defending the US Constitution against all enemies,     ]]]
> [[[ foreign or domestic, requires you to follow Snowden's example. ]]]
> 
>>> Another idea: %Nx with a negative number should output its 2's
>>> complement representation in 4N bits.
> 
>> That's already taken: A leading “-“ means to left-justify the output if the specified width is wider than required.
> 
> We are miscommunicating.  I'm talking about what %Nx should do when
> outputting an argument that is negative, as in (format "%16x" -63).
> 
> Sorry that wasn't clear.
> 
> You could try p/x -63 in GDB to see what I mean.

Oh, I see. Yes, I misunderstood. Apologies for my confusion.

So, in this case, would it truncate the output or use a wider field, if the value couldn’t “properly” be shown in the specified size?
E.g., -5000 would display in hex as …ffec78; with a format “%2x” should it return “78” or should it get enough digits to show the minimum number of bits needed to express the real value, much like formatting with “%2d” would still give you “-5000” instead of chopping off relevant digits?

Maybe we can supply both bit count and minimum field width in a format string… “%32!8x” to display the lowest 32 bits in 8 columns but with leading spaces rather than zeros? Or swap the positions of the numbers. And maybe one could be optional; “%32!x” could mean either “%32!8x” (eight columns, leading spaces) or “%32!1x” (minimum one column, i.e., no leading spaces or zeros added).

It would be logical to consider whether the same format extensions would be useful with %o or %d (or others?) being used to format bignums.

Ken

Re: Using the GNU GMP Library for Bignums in Emacs

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > So, in this case, would it truncate the output or use a wider
  > field, if the value couldn’t “properly” be shown in the specified
  > size?

I think we want the option of truncating (which includes truncating a
negative number as 2s complement), and the option of signaling an
error when the value does not fit (when it would require truncating).

It would be ok also to have the option of using extra digits, as many
as needed.  That option would have to print a minus sign if the number
is negative, since treating it as positive would require infinitely
many digits.

I am not sure that third option is really useful, but if it is, let's
have it.

  > It would be logical to consider whether the same format extensions
  > would be useful with %o or %d (or others?) being used to format
  > bignums.

I don't think these features are useful for decimal output.  (Has
anyone ever wanted them?)  For octal, I am not sure.  Maybe octal
should be treated like hex.



-- 
Dr Richard Stallman
President, Free Software Foundation (https://gnu.org, https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)
Skype: No way! See https://stallman.org/skype.html.

Re: Using the GNU GMP Library for Bignums in Emacs

[Andreas Schwab]

On Mai 04 2018, Richard Stallman <rms@gnu.org> wrote:

> Another idea: %Nx with a negative number should output its 2's
> complement representation in 4N bits.

That's what the precision is for.

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510  2552 DF73 E780 A9DA AEC1
"And now for something completely different."

Re: Using the GNU GMP Library for Bignums in Emacs

[Helmut Eller]

On Thu, May 03 2018, Eli Zaretskii wrote:

>> (format "%016x" (lognot 257)) => "3ffffffffffffefe"
>> 
>> but the 16-hex-digit string should be "fffffffffffffefe".
>
> But a 16-hex-digit fffffffffffffefe is not a fixnum, right?

Well, (lognot 257) is fixnum.  And if we explicity specify the length,
then the 16-hex-digit string "fffffffffffffefe" represents the same
fixnum as the 4-hex-digit string "fefe".

Helmut

Re: Using the GNU GMP Library for Bignums in Emacs

[Helmut Eller]

On Thu, May 03 2018, Eli Zaretskii wrote:

[...]
> Maybe we are miscommunicating.  fffffffffffffefe cannot be a fixnum
> because it invades the bits reserved for the type tag.  Emacs fixnums
> are limited to smaller values, as I'm sure you know.

If we interpret "fffffffffffffefe" as the 64 bit two's complement
representation of an integer then it corresponds to the integer -258.
If we interpret "fefe" as the 16 bit two's complement representation of
an integer then we also get the integer -258.  And -258 is a fixnum.
Still convinced that "fffffffffffffefe cannot be a fixnum"?.

Helmut

Re: Using the GNU GMP Library for Bignums in Emacs

[Paul Eggert]

On 05/03/2018 01:30 PM, Helmut Eller wrote:
> Still convinced that "fffffffffffffefe cannot be a fixnum"?.

No, Eli's right. Emacs has never interpreted fffffffffffffefe as a 
fixnum on any practical platform, as far as I know. In Elisp code on 
practical platforms, #xfffffffffffffefe is a fixnum neither in master 
(where it is an overflow error) nor in Emacs 25 (where it is typically 
converted silently to 1.8446744073709552e+19, with some loss of 
information). If Emacs gets bignums, #xfffffffffffffefe will be lossless 
and will have the same meaning as 18446744073709551358, which is the 
standard interpretation of this hexadecimal number in C and in most 
other languages.

Re: Using the GNU GMP Library for Bignums in Emacs

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > We already have such a feature. For example, (format "%09x" 257) returns
  > "000000101", which is 257 hexadecimal, expressed using leading zeros as needed 
  > so that at least 9 digits are used. Once Emacs supports bignums, the same 
  > feature can print 40-hex-digit SHA-1 checksums by using (format "%040x" n).

Perhaps this way everything would work with all-positive hex numbers.

However, I think we also want a feature to signal an error if the
number doesn't fall in the range that the %x construct can represent.
If the Git hash code is 40 hex digits, any attempt to use as a Git hash code
a number that won't fit in 40 hex digits implies that there has already been
an error.

I suggest %!040x for this.  The ! would mean, "Signal an error if the number
doesn't fit in the specified number of digits."  It could be implemented
for all %-specs, but if that is a pain in the neck, it could be implemented
only for %x.


-- 
Dr Richard Stallman
President, Free Software Foundation (https://gnu.org, https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)
Skype: No way! See https://stallman.org/skype.html.

Re: Using the GNU GMP Library for Bignums in Emacs

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > We could invent a special syntax along those lines, though it would need to 
  > differ from "%24x" which already means pad with spaces to 24 characters. 
  > However, there's little point to a special syntax since one can easily achieve 
  > the same effect without it. For example, to format the low-order 24 bits of N, 
  > one can already use (format "%x" (logand N #xffffff)) and this will continue to

The reason to invent a special syntax is to get the commonly desired
result in a simpler way.  If I am right that this is what people
want, why make it hard?

-- 
Dr Richard Stallman
President, Free Software Foundation (https://gnu.org, https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)
Skype: No way! See https://stallman.org/skype.html.

Re: Using the GNU GMP Library for Bignums in Emacs

[Stefan Monnier]

>> Surely we should just add bignum support to existing functions +, -,
>> etc.
> I'm not so sure of that.  E.g. (format "%x" -1) => "3fffffffffffffff".
> That doesn't make any sense on if -1 is a bignum.

Note that (format "%x" -1) does not necessarily use functions
in "+, -, etc..."

There are several steps to adding GMP support.  Some of those steps
might involve non-trivial decisions.  But the first few steps should be
straightforward enough:
A- add a new "bignum" type
B- add new operations on them (this might be enough to start using them in Calc)
C- add support for bignums to some of the pre-existing functions (e.g. +)
D- add support for bignums to more of the pre-existing functions
E- add support for bignums "everywhere" where numbers are usually allowed.

Anything before D/E should be fairly straightforward.
I consider "support for read&print" to fall somewhere in D or maybe E.


        Stefan

Re: Using the GNU GMP Library for Bignums in Emacs

[Tom Tromey]

>>>>> "Stefan" == Stefan Monnier <monnier@iro.umontreal.ca> writes:

Stefan> There are several steps to adding GMP support.  Some of those steps
Stefan> might involve non-trivial decisions.  But the first few steps should be
Stefan> straightforward enough:
Stefan> A- add a new "bignum" type
Stefan> B- add new operations on them (this might be enough to start using them in Calc)
Stefan> C- add support for bignums to some of the pre-existing functions (e.g. +)
Stefan> D- add support for bignums to more of the pre-existing functions
Stefan> E- add support for bignums "everywhere" where numbers are usually allowed.

Stefan> Anything before D/E should be fairly straightforward.
Stefan> I consider "support for read&print" to fall somewhere in D or maybe E.

I've got some of this working.

I've added configury for GMP and added a new bignum type.

I've made some operations work -- comparisons, abs, float.  I haven't
made arith_driver bignum-aware yet, so no "+", "-", etc.  This doesn't
look too hard.

Some part of "format" work but I haven't tried to handle all the cases
discussed in this thread.


A few things need to be discussed.

1. Should the C core change the names of things like NUMBERP to FIXNUMP?
   Similarly should make_number -> make_fixnum?
   I could do this with sed I suppose.  In the meantime I've tweaked the
   necessary spots in a somewhat ugly way.

2. What is (type-of 5)?  Right now 'integer, but arguably perhaps
   'fixnum is correct.  Gerd's branch did this.

3. Should intbig_to_lisp et al be converted to use bignums?
   I'm inclined to say no, for compatibility.

4. Should it be possible to make a "low-valued bignum"?  For testing I
   have a make-bignum function that turns a fixnum into a like-valued
   bignum.  However, maybe this should just not be possible.  (And most
   likely make-bignum should just be deleted before merging any of
   this.)


I've pushed this to my github but I could push it to a branch on
savannah if that's useful.  I don't know the current branch naming
convention, so I'd need guidance there.

Tom

Re: Using the GNU GMP Library for Bignums in Emacs

[John Wiegley]

>>>>> "TT" == Tom Tromey <tom@tromey.com> writes:

TT> 2. What is (type-of 5)? Right now 'integer, but arguably perhaps 'fixnum
TT> is correct. Gerd's branch did this.

Are we still going to have 'integer as a separate type? If not, then I'd stick
with integer, even if it now means fixnum.

-- 
John Wiegley                  GPG fingerprint = 4710 CF98 AF9B 327B B80F
http://newartisans.com                          60E1 46C4 BD1A 7AC1 4BA2

Re: Using the GNU GMP Library for Bignums in Emacs

[Tom Tromey]

Tom> I've made some operations work -- comparisons, abs, float.  I haven't
Tom> made arith_driver bignum-aware yet, so no "+", "-", etc.  This doesn't
Tom> look too hard.

Last night I converted arith_driver, but this breaks bootstrap.
I haven't debugged this yet.

Tom

Re: Using the GNU GMP Library for Bignums in Emacs

[Paul Eggert]

Tom Tromey wrote:

> 1. Should the C core change the names of things like NUMBERP to FIXNUMP?
>     Similarly should make_number -> make_fixnum?

Yes, we should keep things clear in the C core, and this would definitely be 
clearer. Please do it as a separate commit, though.

> 2. What is (type-of 5)?  Right now 'integer, but arguably perhaps
>     'fixnum is correct.  Gerd's branch did this.

'integer', for the same reason that (type-of 5) doesn't return 'natnum'. fixnum 
and natnum are both subtypes of integer, and the basic type is integer.  The 
other ways of doing this would cause more trouble to user code.

> 3. Should intbig_to_lisp et al be converted to use bignums?
>     I'm inclined to say no, for compatibility.

Yes, that function should now return only fixnums and bignums (assuming GMP is 
available). Of course the reverse functions that converts lisp to C integer also 
needs to work with bignums in C range.

> 4. Should it be possible to make a "low-valued bignum"?

No, as too much Lisp code depends on eq acting like = on fixnums. Although this 
will not work for bignums (unless we take Stefan's suggestion...), it should 
continue to work on integers that can fit into fixnum range.

Thanks for working on this!

Re: Using the GNU GMP Library for Bignums in Emacs

[Tom Tromey]

Tom> 1. Should the C core change the names of things like NUMBERP to FIXNUMP?
Tom> Similarly should make_number -> make_fixnum?

Paul> Yes, we should keep things clear in the C core, and this would
Paul> definitely be clearer. Please do it as a separate commit, though.

Yes, definitely.  I ran this and got a big patch:

  sed -i 's/INTEGERP/FIXNUMP/g' *.[ch]
  sed -i 's/make_number/make_fixnum/g' *.[ch]
  sed -i 's/CHECK_NUMBER/CHECK_FIXNUM/g' *.[ch]

I wasn't sure if I should try to rename NUMBERP or NATNUMP.
On the one hand they are really checking only for fixnums.
On the other hand, the logical names seem like they'd be unwieldy.

Also I was not sure what to do with calls like:

#define lisp_h_CHECK_FIXNUM(x) CHECK_TYPE (FIXNUMP (x), Qintegerp, x)

Here it is using 'integerp, but that will be a bit false.



Also I found the bug with arith_driver and now the branch can bootstrap
again.

Tom

Re: Using the GNU GMP Library for Bignums in Emacs

[Paul Eggert]

Tom Tromey wrote:

> I was not sure what to do with calls like:
> 
> #define lisp_h_CHECK_FIXNUM(x) CHECK_TYPE (FIXNUMP (x), Qintegerp, x)

That last line should use Qfixnump, surely.

> I ran this and got a big patch:
> 
>   sed -i 's/INTEGERP/FIXNUMP/g' *.[ch]
I doubt whether this is the best option. Often, the code should actually be 
checking for integers, not for fixnums. So the existing code needs to be 
examined more carefully, with only some INTEGERP instances changed to FIXNUMP. 
(Code should prefer INTEGERP to FIXNUMP, for generality, but in some cases 
FIXNUMP will have to do.)

>   sed -i 's/make_number/make_fixnum/g' *.[ch]

This should be OK, at least for now. At some point we may want to have 
make_fixnum do proper overflow checking.

>   sed -i 's/CHECK_NUMBER/CHECK_FIXNUM/g' *.[ch]

CHECK_NUMBER has always been a misnomer of course, since it excludes floats. But 
these uses need to be fixed by hand; some changed to CHECK_INTEGER, and others 
to CHECK_FIXNUM, depending on what check is wanted.  In that sense this is like 
the INTEGERP/FIXNUMP issue.

> I wasn't sure if I should try to rename NUMBERP or NATNUMP.
> On the one hand they are really checking only for fixnums.
> On the other hand, the logical names seem like they'd be unwieldy.

NUMBERP and NATNUMP should not exclude bignums, since their Lisp counterparts 
don't. If there is a need for "fixnum or float" (not that likely) or for 
"nonnegative fixnum" (more likely), I suggest the names FIXNUM_OR_FLOATP and 
FIXNATP.

Re: Using the GNU GMP Library for Bignums in Emacs

[Tom Tromey]

>> #define lisp_h_CHECK_FIXNUM(x) CHECK_TYPE (FIXNUMP (x), Qintegerp, x)

Paul> That last line should use Qfixnump, surely.

The reason I hesitated is that it seems possible for some Lisp code to
notice this change.

This is also why I wasn't sure that intbig_to_lisp et al should be
converted.  (This one seems even a little worse since it'll be possible
to build without GMP.)

>> sed -i 's/INTEGERP/FIXNUMP/g' *.[ch]

Paul> I doubt whether this is the best option. Often, the code should
Paul> actually be checking for integers, not for fixnums. So the existing
Paul> code needs to be examined more carefully, with only some INTEGERP
Paul> instances changed to FIXNUMP. (Code should prefer INTEGERP to FIXNUMP,
Paul> for generality, but in some cases FIXNUMP will have to do.)

I probably should have explained a little more.

The basic idea behind the patch series is to do the conversion
incrementally.  So, the first patch is just these sed-based renamings --
the resulting Emacs works the same.

Next comes the configury; then the bignum type and type predicate.
After this come some patches to change various operations: eql,
comparisons, arithmetic, printing, the reader, Ffloat, ...

Some of these subsequent patches introduce things like the new INTEGERP:

    INLINE bool
    INTEGERP (Lisp_Object x)
    {
      return FIXNUMP (x) || BIGNUMP (x);
    }

... or the new make_number, etc.

So, the examination you want is done, just not all in a single patch.  I
think this will make review simpler for whoever does that.  Also this
avoids having to convert absolutely everything, which I am sure I can't
do alone.


It's not all so clean right now, so if you do happen to look the
patches, don't be worried, I plan to rework the whole series to be
cleaner before submitting it.


I made a few more things work tonight.  My "minimal viable feature"
to-do list is down to:

* documentation
* natnump
* since we can't have small bignums we can simplify eql a bit
* logcount
* check how rounding should be done for division and fix the code
* lsh / ash
* logb
* % and mod

Any ideas on things that might be missing from this list?

I wonder if Emacs should (provide 'bignum) when bignums are compiled in.

I still haven't looked at using the mpn_ API.

Tom

Re: Using the GNU GMP Library for Bignums in Emacs

[Paul Eggert]

Tom Tromey wrote:
> Any ideas on things that might be missing from this list?

Seems OK for a first cut.

> I wonder if Emacs should (provide 'bignum) when bignums are compiled in.

Not sure it's needed. It should be easy to tell via other means whether bignums 
are present, e.g, (natnump (1+ most-positive-fixnum))

> I still haven't looked at using the mpn_ API.

We need to do that before installing this in master, as Emacs must be in charge 
of bignum storage allocation. But it isn't needed for a first cut.

Re: Using the GNU GMP Library for Bignums in Emacs

[Tom Tromey]

>>>>> "Paul" == Paul Eggert <eggert@cs.ucla.edu> writes:

>> I still haven't looked at using the mpn_ API.

Paul> We need to do that before installing this in master, as Emacs must be
Paul> in charge of bignum storage allocation. But it isn't needed for a
Paul> first cut.

I looked into this a little.

The mpn API seems very hard to use.  Using it would mean reimplementing
many of the functions provided by GMP.

On the other hand, I see that Guile is using the mpz API.  So, maybe in
practice it is safe enough to just set custom allocation functions and
let them throw.

Tom

Re: Using the GNU GMP Library for Bignums in Emacs

[Paul Eggert]

Tom Tromey wrote:
> On the other hand, I see that Guile is using the mpz API.  So, maybe in
> practice it is safe enough to just set custom allocation functions and
> let them throw.

Guile treats memory allocation failure like stack overflow, in the sense that it 
relies on undefined behavior in both cases, I guess on the theory that it's 
typically better to try to keep running (with undefined behavior) than to give 
up and exit right away. GNU Emacs already takes a similar attitude toward stack 
overflow, so perhaps it could do the same for memory exhaustion within mpz 
routines.  (Emacs could soften the blow by using a reserve, as I think it does 
already.)

With all that in mind, I guess it's OK to use the mpz API after all.

> one idea would be to put mini-gmp.c into the Emacs source tree, but
> only compile it if the real GMP library is not found.

That sounds like a good idea, given the above. We could also do this if the 
installed GMP library is too old for Emacs.

Re: Using the GNU GMP Library for Bignums in Emacs

[Tom Tromey]

Paul> With all that in mind, I guess it's OK to use the mpz API after all.

I'll make this change shortly.

>> one idea would be to put mini-gmp.c into the Emacs source tree, but
>> only compile it if the real GMP library is not found.

Paul> That sounds like a good idea, given the above. We could also do this
Paul> if the installed GMP library is too old for Emacs.

I did this, and removed all the non-GMP code paths from my branch.
I haven't yet updated the docs or fixed up the tests (they still act as
if bignums might not work).

There's still one failing test.

After those things I guess this is ready to land.
How should I submit the patches for review?

Tom

Re: Using the GNU GMP Library for Bignums in Emacs

[Paul Eggert]

Tom Tromey wrote:
> After those things I guess this is ready to land.
> How should I submit the patches for review?

If it's a relatively compact set, you can just use 'git format-patch' send off 
an email to bug-gnu-emacs, with the patches attached. If it's a long set, we'd 
probably be better off creating a branch for it on savannah. Maybe you can start 
with the former and we can evolve to the latter as needed.

Re: Using the GNU GMP Library for Bignums in Emacs

[Tom Tromey]

>>>>> "Paul" == Paul Eggert <eggert@cs.ucla.edu> writes:

Paul> Tom Tromey wrote:
>> After those things I guess this is ready to land.
>> How should I submit the patches for review?

Paul> If it's a relatively compact set, you can just use 'git format-patch'
Paul> send off an email to bug-gnu-emacs, with the patches attached. If it's
Paul> a long set, we'd probably be better off creating a branch for it on
Paul> savannah. Maybe you can start with the former and we can evolve to the
Paul> latter as needed.

Right now it is 24 patches.  I tend to split things up a bit to make
them easier to follow.  The only exception is the first patch, which is
the bulk renaming; since this was done by "sed" it may not require quite
as much reading.

So, once the final things are fixed up, I will write the ChangeLog
entries and push it to savannah and send a note here.

Tom

Re: Using the GNU GMP Library for Bignums in Emacs

[Tom Tromey]

Tom> I haven't yet updated the docs or fixed up the tests (they still act as
Tom> if bignums might not work).

One thing I've noticed is that the documentation would be simpler if
'eq' worked for bignums.  As it is there are a couple of spots that try
to explain the difference between small and large integers.

Tom

Re: Using the GNU GMP Library for Bignums in Emacs

[Tom Tromey]

>>>>> "Tom" == Tom Tromey <tom@tromey.com> writes:

Tom> There's still one failing test.

This turned out to be a giant thinko on my part.
Not sure if that's good or bad, but at least we're one step closer.

Tom

Re: Using the GNU GMP Library for Bignums in Emacs

[Tom Tromey]

>>>>> "Paul" == Paul Eggert <eggert@cs.ucla.edu> writes:

>> #define lisp_h_CHECK_FIXNUM(x) CHECK_TYPE (FIXNUMP (x), Qintegerp, x)

Paul> That last line should use Qfixnump, surely.

Fixing this made me realize that some calls to DEFVAR_PER_BUFFER had to
be updated to use Qfixnump rather than Qintegerp.  (With the
restrictions to be lifted piecemeal in the future.)

Also the bytecode interpreter needed some updates; and I changed the
emacs-module API to allow bigger-than-fixnum ints to be converted.

I've implemented all my to-do items, except:

* Update documentation
* Use the mpn_ API.  This does not look fun :-(

I still haven't tried cons_to_unsigned et al, out of compatibility
fears.

I've cleaned up the series quite a bit so it is in decent shape if you
want to read through it:

git@github.com:tromey/emacs.git feature/bignum

I tend to rebase and force-push for development branches like this, so
beware of that.  No ChangeLogs yet I'm afraid.

Tom

Re: Using the GNU GMP Library for Bignums in Emacs

[Tom Tromey]

>>>>> "Tom" == Tom Tromey <tom@tromey.com> writes:

Tom> * Update documentation
Tom> * Use the mpn_ API.  This does not look fun :-(

I made an attempt at updating the documentation, but my heart isn't
really in it, so I tend to doubt that I did all that good of a job.
Also, the issue of whether GMP is required affects the docs quite a bit.
Maybe once the decisions have settled, someone else could look at the
docs.

I replied about the mpn thing already.

I did find a couple more items for the to-do list, though nothing too
pressing:

* cl-isqrt should probably be updated, maybe just sqrt should work on
  integers and use GMP (?)
* expt still isn't converted

Tom

Re: Using the GNU GMP Library for Bignums in Emacs

[Stefan Monnier]

>   sed -i 's/INTEGERP/FIXNUMP/g' *.[ch]
>   sed -i 's/make_number/make_fixnum/g' *.[ch]
>   sed -i 's/CHECK_NUMBER/CHECK_FIXNUM/g' *.[ch]

Reminds me that there's also Coccinelle for such systematic changes.


        Stefan

Re: Using the GNU GMP Library for Bignums in Emacs

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > Yes, that function should now return only fixnums and bignums (assuming GMP is 
  > available).

In what situations would GMP not be available?
Doesn't it run on all our target platforms?

-- 
Dr Richard Stallman
President, Free Software Foundation (https://gnu.org, https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: Using the GNU GMP Library for Bignums in Emacs

[Paul Eggert]

Richard Stallman wrote:
> In what situations would GMP not be available?

On operating systems where GMP is not in the base installation. One example is 
Solaris 10; I'm sure there are others. We could strongly suggest installing GMP 
before building Emacs, as we already do for a GnuTLS and few other libraries; 
but at least at first I doubt whether we'd want to absolutely require GMP, just 
as we don't require GnuTLS.

Re: Using the GNU GMP Library for Bignums in Emacs

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > On operating systems where GMP is not in the base
  > installation. One example is Solaris 10; I'm sure there are
  > others. We could strongly suggest installing GMP before building
  > Emacs, as we already do for a GnuTLS and few other libraries; but
  > at least at first I doubt whether we'd want to absolutely require
  > GMP, just as we don't require GnuTLS.

I don't think we should go to very much trouble
just to make installation on Solaris 10 easier.

GMP and GNUtls are not comparable issues.  Bignums will have a
pervasive effect on the Emacs C code.  I expect TLS affects only a
small part of the code.  I expect that supporting with and without TLS
is a small job compared with supporting with and without bignums.

-- 
Dr Richard Stallman
President, Free Software Foundation (https://gnu.org, https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: Using the GNU GMP Library for Bignums in Emacs

[Tom Tromey]

>>>>> "Richard" == Richard Stallman <rms@gnu.org> writes:

Richard> In what situations would GMP not be available?
Richard> Doesn't it run on all our target platforms?

I think it does, and if not could be made to.

The reason I have made this optional so far is that introducing a new
build-time dependency seems like a pain.  However, if it isn't really a
pain, then I think Emacs would be better if this were just used
unconditionally.

It's easy to remove the #ifdefs at any time of course, so the decision
doesn't have to be made immediately.

Tom

Re: Using the GNU GMP Library for Bignums in Emacs

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > The reason I have made this optional so far is that introducing a new
  > build-time dependency seems like a pain.  However, if it isn't really a
  > pain, then I think Emacs would be better if this were just used
  > unconditionally.

I think it will be even more of a pain to support Emacs both with and
without GMO.

How much of a pain would it be to package GMP with Emacs?

-- 
Dr Richard Stallman
President, Free Software Foundation (https://gnu.org, https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: Using the GNU GMP Library for Bignums in Emacs

[Tom Tromey]

>>>>> "Richard" == Richard Stallman <rms@gnu.org> writes:

Richard> How much of a pain would it be to package GMP with Emacs?

I looked into this and I think one possible answer is "mini-gmp".

This is a single C file that is part of GMP.  It provides the same "mpz"
interface that these patches are currently using, though at a
performance cost.

So, one idea would be to put mini-gmp.c into the Emacs source tree, but
only compile it if the real GMP library is not found.

Tom

Re: Using the GNU GMP Library for Bignums in Emacs

[Stefan Monnier]

>> 2. What is (type-of 5)?  Right now 'integer, but arguably perhaps
>>     'fixnum is correct.  Gerd's branch did this.
> 'integer', for the same reason that (type-of 5) doesn't return
> 'natnum'. fixnum and natnum are both subtypes of integer, and the basic type
> is integer.  The other ways of doing this would cause more trouble to
> user code.

I think it'd be wrong for (type-of 5) to return the same value as
(type-of <BIGNUM>), especially as long as `eq` doesn't treat bignum
the same as fixnums.

Also, for purposes of cl-generic's dispatch it'd be better for type-of
to return `natnum` or `fixnum` when applicable.

Grepping through Elisp code, `type-of` is very rarely used and it's even
more rare for those uses to depend on the value returned for small
integers, but I did find a few such cases, so it's indeed safer to just
keep returning `integer` for small integers (and probably some other value
for bignums, such as `bignum`).


        Stefan

Re: Using the GNU GMP Library for Bignums in Emacs

[Paul Eggert]

Stefan Monnier wrote:
> Also, for purposes of cl-generic's dispatch it'd be better for type-of
> to return `natnum` or `fixnum` when applicable.
Isn't the issue of how finely type-of distinguishes integers separable from the 
issue of whether to have bignums? type-of could return 'natnum' now, even 
without bignums, but it doesn't. If we decide that type-of should return some 
value other than 'integer' for some integers, how would we decide what value to 
return? For example, if (natnump X) and (bignump X) both succeed, should type-of 
return 'natnum' or 'bignum' or both?

All in all I expect Elisp would be a bit easier to use and explain if it treats 
'integer' as the fundamental type, and 'characterp', 'fixnump', 'bignump', 
'natnump' as predicates that succeed on only some members of that type.

> it's indeed safer to just
> keep returning `integer` for small integers (and probably some other value
> for bignums, such as `bignum`).

Why the latter? That is, when would it be useful in existing code for (type-of 
X) to return a value other than 'integer' when (integerp X) returns t?

Re: Using the GNU GMP Library for Bignums in Emacs

[Stefan Monnier]

>> Also, for purposes of cl-generic's dispatch it'd be better for type-of
>> to return `natnum` or `fixnum` when applicable.
> Isn't the issue of how finely type-of distinguishes integers separable from
> the issue of whether to have bignums? type-of could return 'natnum' now,
> even without bignums, but it doesn't.

Exactly.

I was pointing out that type-of already provides suboptimal results in
this respect, but also that there's some grey area when it comes to
decide what it should return.

The main problem is due to subtyping: natnum is (currently) a subtype of
fixnum which is a subtype of integer, so 5 is all three at the same time
and hence (type-of 5) could return any of those three and claim to
be correct.

In general, we'd want type-of to return the "most precise" type, so
you could argue that `natnum` is the better option.  Note that `natnum`
is not a valid type in CommonLisp, tho, so `fixnum` would probably be
a better option (especially if you want to consider positive bignums as
natnums, in which case natnum is not a subtype of fixnum).

> If we decide that type-of should return some value other than
> 'integer' for some integers, how would we decide what value to return?
> For example, if (natnump X) and (bignump X) both succeed, should
> type-of return 'natnum' or 'bignum' or both?

Based on the above (the fact that natnum is neither a subtype nor
a supertype of fixnum or bignum, and the fact that it's not even a type
in Common Lisp), I think we can ignore `natnum` for `type-of` purposes.

> All in all I expect Elisp would be a bit easier to use and explain if it
> treats 'integer' as the fundamental type, and 'characterp', 'fixnump',
> 'bignump', 'natnump' as predicates that succeed on only some members of
> that type.

As long as `eq` treats bignums differently from fixnums, I think type-of
should return different values for the two (ideally, I'd vote for
`bignum` and `fixnum` respectively).

When/if we completely hide the difference between the two (i.e. all
integers behave like bignums and fixnums are just an implementation
detail for optimization purposes), then I'd agree that `type-of` can
return just `integer` for those numbers regardless of their
internal representation.  But at that moment I'd also argue that
`fixnump` and `bignump` should disappear.

>> it's indeed safer to just keep returning `integer` for small integers
>> (and probably some other value for bignums, such as `bignum`).
> Why the latter? That is, when would it be useful in existing code for
> (type-of X) to return a value other than 'integer' when (integerp X)
> returns t?

Because it's generally better to return a more precise type than a more
generic one.
E.g. (type-of '(1 2)) returns `cons` not `list` and (type-of [1 2])
returns `vector` rather than `sequence`.


        Stefan

Re: Using the GNU GMP Library for Bignums in Emacs

[Helmut Eller]

On Mon, Jul 09 2018, Stefan Monnier wrote:

> In general, we'd want type-of to return the "most precise" type, so
> you could argue that `natnum` is the better option.  Note that `natnum`
> is not a valid type in CommonLisp, tho, so `fixnum` would probably be
> a better option (especially if you want to consider positive bignums as
> natnums, in which case natnum is not a subtype of fixnum).

The most precise type for 5 would be (eql 5) or (integer 5 5).  But I
guess for generic function dispatch it would more useful to have
something like a "representation type".  Or even "type codes" that are
ordered in some fashion.

Helmut

Re: Using the GNU GMP Library for Bignums in Emacs

[Stefan Monnier]

> The most precise type for 5 would be (eql 5) or (integer 5 5).

Indeed, and we don't want that.

> But I guess for generic function dispatch it would more useful to have
> something like a "representation type".

That's right.  The most precise possible, but not too precise.


        Stefan

Re: Using the GNU GMP Library for Bignums in Emacs

[Paul Eggert]

On 07/09/2018 08:41 PM, Stefan Monnier wrote:
> As long as `eq` treats bignums differently from fixnums, I think type-of
> should return different values for the two

I don't see how this folds into your earlier point about cl-generic's 
dispatch. I thought the idea of cl-generic was to define a generic 
function that finds the most specialized method for its arguments. (I'm 
no expert in cl-generic, so please bear with me if I'm wrong.)

If so, why would someone want to specialize a method just for fixnums, 
as opposed to a method for integers? The basic operations +, -, * etc. 
aren't specialized for fixnums. Are you anticipating adding specialized 
variants fixnum-+, fixnum--, fixnum-*, etc., and then having people 
write methods with all this in mind? Or maybe having the bytecode 
optimizer use them?

If so, I suspect that approach won't be needed for Emacs Lisp. Elisp 
programs are not typically intended for that sort of microptimization -- 
otherwise we'd already have operations like integer-+, integer--, 
integer-*, etc. And the complexity of the differing semantics (fixnum-+ 
would wrap around, whereas + would not) would probably not be worth the 
relatively minor performance benefits.

If not, then I'm still not following the advantage of having (type-of 5) 
return something other than 'integer'.

Re: Using the GNU GMP Library for Bignums in Emacs

[Helmut Eller]

On Tue, Jul 10 2018, Paul Eggert wrote:

> If so, why would someone want to specialize a method just for fixnums,
> as opposed to a method for integers? 

Some important functions only for fixnums but not for bignums.
E.g. goto-char or aref.

Helmut

Re: Using the GNU GMP Library for Bignums in Emacs

[Paul Eggert]

On 07/10/2018 09:48 AM, Helmut Eller wrote:
>> If so, why would someone want to specialize a method just for fixnums,
>> as opposed to a method for integers?
> Some important functions only for fixnums but not for bignums.
> E.g. goto-char or aref.

Both of these functions have limits that are not relevant to fixnums; 
they have limits based on something else. For example, goto-char is 
limited to point-min..point-max, and aref is limited to 0..length-1. We 
will want both functions to work on bignums, since we will want to allow 
buffer sizes and bool vector lengths that do not fit into fixnums.

And even if these functions' limits were relevant to fixnums, I don't 
see why one would want to specialize a method just for fixnums as 
opposed to settling for specializing it to integers. The method would 
accept an integer, and then if it used a fixnum-only goto-char or aref 
that would signal an error, just as it would for any other out-of-range 
integer. There would be no need to specialize the method to fixnums.

So I'm still not seeing the actual use cases that would suggest that 
(type-of 5) should not continue to return 'integer'.

Re: Using the GNU GMP Library for Bignums in Emacs

[Helmut Eller]

On Tue, Jul 10 2018, Paul Eggert wrote:

>> Some important functions only for fixnums but not for bignums.
>> E.g. goto-char or aref.
>
> Both of these functions have limits that are not relevant to fixnums;
> they have limits based on something else. For example, goto-char is
> limited to point-min..point-max, and aref is limited to
> 0..length-1. We will want both functions to work on bignums, since we
> will want to allow buffer sizes and bool vector lengths that do not
> fit into fixnums.

Speak for yourself.  I certainly don't want goto-char or aref to work on
bignums.  I bed 200 dollars that Emacs will not support buffer sizes or
array lengths beyond most-positive-fixnum within the next year.

And I would be surprised if Emacs would switch to bignums for functions
like, current-time of file-attributes, where it would actually make
sense to use bignums.

> And even if these functions' limits were relevant to fixnums, I don't
> see why one would want to specialize a method just for fixnums as
> opposed to settling for specializing it to integers. The method would
> accept an integer, and then if it used a fixnum-only goto-char or aref
> that would signal an error, just as it would for any other
> out-of-range integer. There would be no need to specialize the method
> to fixnums.

If we follow that argumentation, then no method would ever need any
specialization.  The method would accept any type and then if something
doesn't work for the actual type it would signal an error.

> So I'm still not seeing the actual use cases that would suggest that
> (type-of 5) should not continue to return 'integer'.

To write reasonable programs, programmers need to have reasonable models
of how computers work and in particular they need to now how data types
are physically represented.  That's why some programmers want know if a
sequence is a list or a vector or if a number is a fixnum, a flonum, or
a bignum.

Helmut

Re: Using the GNU GMP Library for Bignums in Emacs

[Stefan Monnier]

> Speak for yourself.  I certainly don't want goto-char or aref to work on
> bignums.  I bed 200 dollars that Emacs will not support buffer sizes or
> array lengths beyond most-positive-fixnum within the next year.

bignums don't make much sense for array sizes and indices, indeed
(except maybe for boolean-vectors), but for buffer sizes they would
allow buffers >512MB (tho still <4GB or likely even <2GB) on 32bit
systems without having to pay for the --wide-int tax.

> To write reasonable programs, programmers need to have reasonable models
> of how computers work and in particular they need to now how data types
> are physically represented.  That's why some programmers want know if a
> sequence is a list or a vector or if a number is a fixnum, a flonum, or
> a bignum.

In case there was still a doubt: I fully agree.


        Stefan

Re: Using the GNU GMP Library for Bignums in Emacs

[Helmut Eller]

On Tue, Jul 10 2018, Stefan Monnier wrote:

>> Speak for yourself.  I certainly don't want goto-char or aref to work on
>> bignums.  I bed 200 dollars that Emacs will not support buffer sizes or
>> array lengths beyond most-positive-fixnum within the next year.
>
> bignums don't make much sense for array sizes and indices, indeed
> (except maybe for boolean-vectors), but for buffer sizes they would
> allow buffers >512MB (tho still <4GB or likely even <2GB) on 32bit
> systems without having to pay for the --wide-int tax.

We'll see if somebody spends his time on this.  It seems like a waste of
time to me, now as 64 bit systems are so common.

Helmut

Re: Using the GNU GMP Library for Bignums in Emacs

[Stefan Monnier]

> We'll see if somebody spends his time on this.  It seems like a waste of
> time to me, now as 64 bit systems are so common.

My guess is that it will happen not because someone really cares about
it, but because a "Lisp_Object to EMACS_INT" conversion function (which
also accepts bignums as long as they fit within EMACS_INT) will be
needed elsewhere anyway and once you have that, you might as well call
it XINT so it will "magically work".


        Stefan

Re: Using the GNU GMP Library for Bignums in Emacs

[Tom Tromey]

>>>>> "Helmut" == Helmut Eller <eller.helmut@gmail.com> writes:

Helmut> And I would be surprised if Emacs would switch to bignums for functions
Helmut> like, current-time of file-attributes, where it would actually make
Helmut> sense to use bignums.

This was discussed elsewhere in the thread - that is what the
INTEGER_TO_CONS stuff was about.  Paul wanted this change:

https://lists.gnu.org/archive/html/emacs-devel/2018-07/msg00160.html

I'm maybe a bit more cautious about it.

Tom

Re: Using the GNU GMP Library for Bignums in Emacs

[Paul Eggert]

Tom Tromey wrote:
> Paul wanted this change:
> 
> https://lists.gnu.org/archive/html/emacs-devel/2018-07/msg00160.html
> 
> I'm maybe a bit more cautious about it.

It'd be fine to do bignums first, and change intbig_to_lisp to use them later.

Re: Using the GNU GMP Library for Bignums in Emacs

[Stefan Monnier]

> If so, why would someone want to specialize a method just for fixnums, as
> opposed to a method for integers?

cl-generic has no trouble dispatching on a supertype of that returned by
`type-of`.   IOW if `type-of` returns `fixnum` and `bignum` cl-generic
has no difficulty finding that these are applicable for methods that
stipulate an `integer` argument (or a `number` argument for that
matter).  But the other way around is a lot more difficult, so as long
as fixnums and bignums are visibly two different types, I think it makes
more sense for `type-of` to return different values for those two cases.

> The basic operations +, -, * etc. aren't specialized for fixnums.

Actually, they are, but this is done by manual type-testing in the
C code, not with cl-defmethod.

> Are you anticipating adding specialized variants fixnum-+, fixnum--,
> fixnum-*, etc., and then having people write methods with all this in
> mind? Or maybe having the bytecode optimizer use them?

Not at all, no.

> If not, then I'm still not following the advantage of having (type-of 5)
> return something other than 'integer'.

For backward compatibility reasons, I agree that (type-of 5) should
continue returning `integer` (tho returning `fixnum` would be better
otherwise), but (type-of <BIGNUM>) doesn't suffer from such a backward
compatibility requirement, so it would be preferable for it to return
`bignum`.

I wonder on the other hand, why you think it'd be bad for (type-of <BIGNUM>)
to return `bignum` instead of `integer`.


        Stefan

Re: Using the GNU GMP Library for Bignums in Emacs

[Tom Tromey]

>>>>> "Stefan" == Stefan Monnier <monnier@IRO.UMontreal.CA> writes:

Stefan> but (type-of <BIGNUM>) doesn't suffer from such a backward
Stefan> compatibility requirement, so it would be preferable for it to return
Stefan> `bignum`.

I read this thread and I still don't know what a use case for this would
be.  Do you have something in mind?

Stefan> I wonder on the other hand, why you think it'd be bad for (type-of <BIGNUM>)
Stefan> to return `bignum` instead of `integer`.

Well, for one thing, it's simpler to explain that there are just
integers.  In fact after working on the docs for this I think it would
be better if bignums were interned so that eq would work.

Tom

Re: Using the GNU GMP Library for Bignums in Emacs

[Paul Eggert]

> I read this thread and I still don't know what a use case for this would be.

Yes, I'm not seeing it either.

> it's simpler to explain that there are just integers.

Yes, and this is a significant win.

If interning bignums (so that eq works on them) is what it takes to satify 
Stefan's concerns, then I'm all for interning them. The idea is to avoid 
unnecessary distinctions between fixnums and bignums, so that Lisp programmers 
can focus on their problems rather than worry about integer representation. 
Although it's OK to have user-visible predicates fixnump and bignump for the few 
programmers who really need to know, I'm not seeing use cases for why these 
predicates should be any more special than characterp or natnump are.


 >> why would someone want to specialize a method just for fixnums, as
 >> opposed to a method for integers?
 >
 > cl-generic has no trouble dispatching on a supertype of that returned by
 > `type-of`.

That's fine, but I was asking for use cases not implementation motivation. THat 
is, I was wondering why a Lisp programmer would want to use cl-generic that way. 
Having cl-generic distinguish fixnums from bignums sounds like more a solution 
looking for a problem than a truly useful feature.

 >> The basic operations +, -, * etc. aren't specialized for fixnums.
 >
 > Actually, they are, but this is done by manual type-testing in the
 > C code, not with cl-defmethod.

Yes, of course: with basic operations it's done under the hood, and Lisp 
programmers don't need to worry about the details. This is a reasonable 
programming style for Emacs Lisp, and it'd be a good thing if this style were 
used as consistently as possible, to keep things simpler.

Re: Using the GNU GMP Library for Bignums in Emacs

[Stefan Monnier]

>> it's simpler to explain that there are just integers.
> Yes, and this is a significant win.

I'm not sure I understand why a return value of `bignum` for `type-of`
would complicate the doc.  AFAICT all it takes is:

    diff --git a/doc/lispref/objects.texi b/doc/lispref/objects.texi
    index b94de80b65..3ca5f6399f 100644
    --- a/doc/lispref/objects.texi
    +++ b/doc/lispref/objects.texi
    @@ -2034,7 +2034,7 @@ Type Predicates
     
     @defun type-of object
     This function returns a symbol naming the primitive type of
    -@var{object}.  The value is one of the symbols @code{bool-vector},
    +@var{object}.  The value is one of the symbols @code{bignum}, @code{bool-vector},
     @code{buffer}, @code{char-table}, @code{compiled-function},
     @code{condition-variable}, @code{cons}, @code{finalizer},
     @code{float}, @code{font-entity}, @code{font-object},

There's no need to explain the difference between `integer` and `bignum`
here, just like we don't explain the difference between `subr` and
`compiled-function`.

> If interning bignums (so that eq works on them) is what it takes to satify
> Stefan's concerns, then I'm all for interning them. The idea is to avoid
> unnecessary distinctions between fixnums and bignums, so that Lisp
> programmers can focus on their problems rather than worry about integer
> representation. Although it's OK to have user-visible predicates fixnump and
> bignump for the few programmers who really need to know, I'm not seeing use
> cases for why these predicates should be any more special than characterp or
> natnump are.
[...]
> That's fine, but I was asking for use cases not implementation
> motivation. THat is, I was wondering why a Lisp programmer would want to use
> cl-generic that way. Having cl-generic distinguish fixnums from bignums
> sounds like more a solution looking for a problem than a truly
> useful feature.

I think I'm missing a reality check here.  We are talking about
`type-of`, right?  Have you looked at how many times it's used in Elisp?
It's a rather obscure function (and indeed the lispref recommends to
use predicates instead: type-of might be handy to implement other things
(like cl-generic) but is otherwise rather inconvenient to use).

So, yes, the motivation is thin because the decision will be irrelevant
in 99.99% of the cases.  If you're really dead-set on having it
hide the difference between fixnums and bignums, I won't oppose it.

I just mentioned my preference based on the fact that cl-generic is
(AFAIK) by far the most common user of `type-of` nowadays.


        Stefan

Re: Using the GNU GMP Library for Bignums in Emacs

[Paul Eggert]

Stefan Monnier wrote:

> There's no need to explain the difference between `integer` and `bignum`
> here, just like we don't explain the difference between `subr` and
> `compiled-function`.

subr and compiled-functions are distinct (nonintersecting) types, whereas bignum 
is a subset of integer, so these are quite different cases. The manual says 
"each object belongs to one and only one primitive type; ‘type-of’ tells you 
which one". But if type-of returned 'integer' and bignums are integers (which 
they should be), this statement in the manual would become false, and would need 
to be rewritten. I don't see this as a win.

> I think I'm missing a reality check here.  We are talking about
> `type-of`, right?  Have you looked at how many times it's used in Elisp?

True, it's pretty rare. And it is a relic of the old-fashioned days when types 
were all disjoint. (Maybe we should remove 'type-of'? :-)

I'm not all that worried about type-of. I am more worried about the idea that 
bignums are second-class integers. When at all possible, bignums should be 
treated the same way that any other (large) integer is treated. For example, 
aref should report that a bignum index is out of range (if it is indeed out of 
range) rather than reporting a type error.

Re: Using the GNU GMP Library for Bignums in Emacs

[Stefan Monnier]

> subr and compiled-functions are distinct (nonintersecting) types, whereas
> bignum is a subset of integer, so these are quite different cases. The
> manual says "each object belongs to one and only one primitive type;
> ‘type-of’ tells you which one". But if type-of returned 'integer' and
> bignums are integers (which they should be), this statement in the manual
> would become false, and would need to be rewritten. I don't see this
> as a win.

Actually, the incorrect statement would be when (type-of 5) returns
`integer` since `integer` wouldn't be a primitive type any more.

> True, it's pretty rare. And it is a relic of the old-fashioned days when
> types were all disjoint. (Maybe we should remove 'type-of'? :-)

FWIW, I dislike type-of and would be happy to get rid of it (and
replace it with a cl-generic-specific function instead which would fix
some of its shortcomings for that use-case).

> I'm not all that worried about type-of. I am more worried about the idea
> that bignums are second-class integers. When at all possible, bignums should
> be treated the same way that any other (large) integer is treated.

Fully agreed.  I just see `type-of` as a function that reports about the
actual representation so it's OK for it to reveal differences which will
rarely affect normal code.


        Stefan

Re: Using the GNU GMP Library for Bignums in Emacs

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > Well, for one thing, it's simpler to explain that there are just
  > integers.  In fact after working on the docs for this I think it would
  > be better if bignums were interned so that eq would work.

Simplifying the documentation is often a good reason to change the
Emacs Lisp language.  But it is not automatically decisive.  In this
case, there may be important advantages on the other side too.

-- 
Dr Richard Stallman
President, Free Software Foundation (https://gnu.org, https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: Using the GNU GMP Library for Bignums in Emacs

[Tom Tromey]

>>>>> "Richard" == Richard Stallman <rms@gnu.org> writes:

Richard> [[[ To any NSA and FBI agents reading my email: please consider    ]]]
Richard> [[[ whether defending the US Constitution against all enemies,     ]]]
Richard> [[[ foreign or domestic, requires you to follow Snowden's example. ]]]

>> Well, for one thing, it's simpler to explain that there are just
>> integers.  In fact after working on the docs for this I think it would
>> be better if bignums were interned so that eq would work.

Richard> Simplifying the documentation is often a good reason to change the
Richard> Emacs Lisp language.  But it is not automatically decisive.  In this
Richard> case, there may be important advantages on the other side too.

Yeah, I just don't know what those advantages are; or maybe they were
explained and I missed and/or did not understand them.


Speaking of downsides, I think there is at least one downside of making
'eq' work for bignums, which is that it is slower.

One way to implement this would be to intern each mpz_t in some sort of
weak hash table.  This isn't hard to implement but it means a hash
lookup whenever converting a bignum to a Lisp object.

Another way would be to add a special case to eq, though as you said
earlier, this would slow down eq.

I don't know which would be better and I haven't implemented either one.
Instead I plan to push my branch to savannah soon, and then I will send
out an email and everybody can take a look at it and see what they think
about the documentation and the implementation.

Tom

Re: Using the GNU GMP Library for Bignums in Emacs

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > Richard> Simplifying the documentation is often a good reason to change the
  > Richard> Emacs Lisp language.  But it is not automatically decisive.  In this
  > Richard> case, there may be important advantages on the other side too.

  > Yeah, I just don't know what those advantages are; or maybe they were
  > explained and I missed and/or did not understand them.

It is this:

  > Speaking of downsides, I think there is at least one downside of making
  > 'eq' work for bignums, which is that it is slower.

I don't know HOW bad this will be.  (It would be nice to find out by
experiment.)  That's why I said there _may_ be important advantages on
the other side.  I don't know how important they are.

-- 
Dr Richard Stallman
President, Free Software Foundation (https://gnu.org, https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: Using the GNU GMP Library for Bignums in Emacs

[Stefan Monnier]

> I don't know HOW bad this will be.  (It would be nice to find out by
> experiment.)  That's why I said there _may_ be important advantages on
> the other side.  I don't know how important they are.

I recently posted a patch which makes EQ behave like `eql` attached.
I just tried to see its impact on the ELisp compilation time
(i.e. I did `rm **/*.elc; make`).

The original time on my benchmark machine was in the bracket
58.77s..59.13s, and with the patch, the time increased to a bracket of
61.03s..61.68s.

So basically a 4% slow down.


        Stefan


diff --git a/src/fns.c b/src/fns.c
index 27c16c6072..4c23bb2488 100644
--- a/src/fns.c
+++ b/src/fns.c
@@ -2173,10 +2173,7 @@ DEFUN ("eql", Feql, Seql, 2, 2, 0,
 Floating-point numbers of equal value are `eql', but they may not be `eq'.  */)
   (Lisp_Object obj1, Lisp_Object obj2)
 {
-  if (FLOATP (obj1))
-    return equal_no_quit (obj1, obj2) ? Qt : Qnil;
-  else
-    return EQ (obj1, obj2) ? Qt : Qnil;
+  return EQL (obj1, obj2) ? Qt : Qnil;
 }
 
 DEFUN ("equal", Fequal, Sequal, 2, 2, 0,
@@ -2272,7 +2269,7 @@ internal_equal (Lisp_Object o1, Lisp_Object o2, enum equal_kind equal_kind,
 	}
     }
 
-  if (EQ (o1, o2))
+  if (lisp_h_EQ (o1, o2))
     return true;
   if (XTYPE (o1) != XTYPE (o2))
     return false;
@@ -3766,7 +3763,7 @@ cmpfn_user_defined (struct hash_table_test *ht,
    in a Lisp integer.  */
 
 static EMACS_UINT
-hashfn_eq (struct hash_table_test *ht, Lisp_Object key)
+hashfn__eq (struct hash_table_test *ht, Lisp_Object key)
 {
   return XHASH (key) ^ XTYPE (key);
 }
@@ -3788,7 +3785,13 @@ hashfn_equal (struct hash_table_test *ht, Lisp_Object key)
 static EMACS_UINT
 hashfn_eql (struct hash_table_test *ht, Lisp_Object key)
 {
-  return FLOATP (key) ? hashfn_equal (ht, key) : hashfn_eq (ht, key);
+  return FLOATP (key) ? hashfn_equal (ht, key) : hashfn__eq (ht, key);
+}
+
+static EMACS_UINT
+hashfn_eq (struct hash_table_test *ht, Lisp_Object key)
+{
+  return hashfn_eql (ht, key);
 }
 
 /* Value is a hash code for KEY for use in hash table H which uses as
diff --git a/src/lisp.h b/src/lisp.h
index 0bb7c488f2..26fa477ce4 100644
--- a/src/lisp.h
+++ b/src/lisp.h
@@ -361,7 +361,7 @@ typedef EMACS_INT Lisp_Word;
 #define lisp_h_INTEGERP(x) ((XTYPE (x) & (Lisp_Int0 | ~Lisp_Int1)) == Lisp_Int0)
 #define lisp_h_MARKERP(x) (MISCP (x) && XMISCTYPE (x) == Lisp_Misc_Marker)
 #define lisp_h_MISCP(x) (XTYPE (x) == Lisp_Misc)
-#define lisp_h_NILP(x) EQ (x, Qnil)
+#define lisp_h_NILP(x) lisp_h_EQ (x, Qnil)
 #define lisp_h_SET_SYMBOL_VAL(sym, v) \
    (eassert ((sym)->u.s.redirect == SYMBOL_PLAINVAL), \
     (sym)->u.s.val.value = (v))
@@ -421,7 +421,7 @@ typedef EMACS_INT Lisp_Word;
 # define CHECK_SYMBOL(x) lisp_h_CHECK_SYMBOL (x)
 # define CHECK_TYPE(ok, predicate, x) lisp_h_CHECK_TYPE (ok, predicate, x)
 # define CONSP(x) lisp_h_CONSP (x)
-# define EQ(x, y) lisp_h_EQ (x, y)
+# define EQ(x, y) EQL (x, y)
 # define FLOATP(x) lisp_h_FLOATP (x)
 # define INTEGERP(x) lisp_h_INTEGERP (x)
 # define MARKERP(x) lisp_h_MARKERP (x)
@@ -1141,10 +1141,21 @@ make_natnum (EMACS_INT n)
 
 /* Return true if X and Y are the same object.  */
 
+extern bool equal_no_quit (Lisp_Object o1, Lisp_Object o2);
+
+INLINE bool
+EQL (Lisp_Object x, Lisp_Object y)
+{
+  if (lisp_h_FLOATP (x))
+    return equal_no_quit (x, y);
+  else
+    return lisp_h_EQ (x, y);
+}
+
 INLINE bool
 (EQ) (Lisp_Object x, Lisp_Object y)
 {
-  return lisp_h_EQ (x, y);
+  return EQL (x, y);
 }
 
 /* True if the possibly-unsigned integer I doesn't fit in a Lisp fixnum.  */
@@ -3626,7 +3637,6 @@ extern Lisp_Object merge (Lisp_Object, Lisp_Object, Lisp_Object);
 extern Lisp_Object do_yes_or_no_p (Lisp_Object);
 extern Lisp_Object concat2 (Lisp_Object, Lisp_Object);
 extern Lisp_Object concat3 (Lisp_Object, Lisp_Object, Lisp_Object);
-extern bool equal_no_quit (Lisp_Object, Lisp_Object);
 extern Lisp_Object nconc2 (Lisp_Object, Lisp_Object);
 extern Lisp_Object assq_no_quit (Lisp_Object, Lisp_Object);
 extern Lisp_Object assoc_no_quit (Lisp_Object, Lisp_Object);

Re: Using the GNU GMP Library for Bignums in Emacs

[Paul Eggert]

[-- Attachment #1: Type: text/plain, Size: 939 bytes --]

Stefan Monnier wrote:
> I recently posted a patch which makes EQ behave like `eql` attached.
> I just tried to see its impact on the ELisp compilation time
> (i.e. I did `rm **/*.elc; make`).

I tried to reproduce this on my machine, and ran into some trouble (the code had 
warnings that caused compilation to fail). Although your patch is evidently 
intended only as a quick benchmark and not as an actual change, I worry that the 
benchmark isn't realistic enough, as some usage of EQ in C code will need to 
change to Feql or equivalent. Also, the C code will need to change how hashing 
works since XHASH etc. must be consistent with eq.

Looking into this a bit more, I discovered that eql currently operates 
incorrectly on NaNs, as it's inconsistent with how hash tables work. I installed 
the attached patch into master to fix that.

I'll try to look into doing a benchmark like yours with all the above in mind.

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-Fix-bug-with-eql-etc.-on-NaNs.patch --]
[-- Type: text/x-patch; name="0001-Fix-bug-with-eql-etc.-on-NaNs.patch", Size: 7709 bytes --]

From c70d22f70b77b053d01c7380122d166ecb728610 Mon Sep 17 00:00:00 2001
From: Paul Eggert <eggert@cs.ucla.edu>
Date: Wed, 18 Jul 2018 03:16:54 -0700
Subject: [PATCH] Fix bug with eql etc. on NaNs

Fix a bug where eql, sxhash-eql, memql, and make-hash-table
were not consistent on NaNs.  Likewise for equal,
sxhash-equal, member, and make-hash-table.  Some of these
functions ignored NaN significands, whereas others treated
them as significant.  It's more logical to treat significands
as significant, and this typically makes eql a bit more
efficient on floats, with just one integer comparison instead
of one to three floating-point comparisons.
* doc/lispref/numbers.texi (Float Basics): Document that
NaNs are never numerically equal, but might be eql.
* src/fns.c (WORDS_PER_DOUBLE): Move to top level of this file.
(union double_and_words): Now named, and at the top level of this file.
(same_float): New function.
(Fmemql, Feql, internal_equal, cmpfn_eql): Use it, so that
the corresponding functions treat NaNs consistently.
(sxhash_float): Simplify based on above-mentioned changes.

* test/src/fns-tests.el (fns-tests-equality-nan): New test.
---
 doc/lispref/numbers.texi |  9 +++++--
 src/fns.c                | 68 +++++++++++++++++++++++++-----------------------
 test/src/fns-tests.el    | 11 ++++++++
 3 files changed, 53 insertions(+), 35 deletions(-)

diff --git a/doc/lispref/numbers.texi b/doc/lispref/numbers.texi
index 2fed2b6..6c51b84 100644
--- a/doc/lispref/numbers.texi
+++ b/doc/lispref/numbers.texi
@@ -232,13 +232,18 @@ Float Basics
 @cindex negative infinity
 @cindex infinity
 @cindex NaN
+@findex eql
+@findex sxhash-eql
   The @acronym{IEEE} floating-point standard supports positive
 infinity and negative infinity as floating-point values.  It also
 provides for a class of values called NaN, or ``not a number'';
 numerical functions return such values in cases where there is no
 correct answer.  For example, @code{(/ 0.0 0.0)} returns a NaN@.
-Although NaN values carry a sign, for practical purposes there is no other
-significant difference between different NaN values in Emacs Lisp.
+A NaN is never numerically equal to any value, not even to itself.
+NaNs carry a sign and a significand, and non-numeric functions like
+@code{eql} and @code{sxhash-eql} treat two NaNs as equal when their
+signs and significands agree.  Significands of NaNs are
+machine-dependent and are not directly visible to Emacs Lisp.
 
 Here are read syntaxes for these special floating-point values:
 
diff --git a/src/fns.c b/src/fns.c
index c171784..10997da 100644
--- a/src/fns.c
+++ b/src/fns.c
@@ -1419,6 +1419,29 @@ DEFUN ("elt", Felt, Selt, 2, 2, 0,
   return Faref (sequence, n);
 }
 
+enum { WORDS_PER_DOUBLE = (sizeof (double) / sizeof (EMACS_UINT)
+                          + (sizeof (double) % sizeof (EMACS_UINT) != 0)) };
+union double_and_words
+{
+  double val;
+  EMACS_UINT word[WORDS_PER_DOUBLE];
+};
+
+/* Return true if X and Y are the same floating-point value.
+   This looks at X's and Y's representation, since (unlike '==')
+   it returns true if X and Y are the same NaN.  */
+static bool
+same_float (Lisp_Object x, Lisp_Object y)
+{
+  union double_and_words
+    xu = { .val = XFLOAT_DATA (x) },
+    yu = { .val = XFLOAT_DATA (y) };
+  EMACS_UINT neql = 0;
+  for (int i = 0; i < WORDS_PER_DOUBLE; i++)
+    neql |= xu.word[i] ^ yu.word[i];
+  return !neql;
+}
+
 DEFUN ("member", Fmember, Smember, 2, 2, 0,
        doc: /* Return non-nil if ELT is an element of LIST.  Comparison done with `equal'.
 The value is actually the tail of LIST whose car is ELT.  */)
@@ -1457,7 +1480,7 @@ The value is actually the tail of LIST whose car is ELT.  */)
   FOR_EACH_TAIL (tail)
     {
       Lisp_Object tem = XCAR (tail);
-      if (FLOATP (tem) && equal_no_quit (elt, tem))
+      if (FLOATP (tem) && same_float (elt, tem))
 	return tail;
     }
   CHECK_LIST_END (tail, list);
@@ -2175,7 +2198,7 @@ Floating-point numbers of equal value are `eql', but they may not be `eq'.  */)
   (Lisp_Object obj1, Lisp_Object obj2)
 {
   if (FLOATP (obj1))
-    return equal_no_quit (obj1, obj2) ? Qt : Qnil;
+    return FLOATP (obj2) && same_float (obj1, obj2) ? Qt : Qnil;
   else
     return EQ (obj1, obj2) ? Qt : Qnil;
 }
@@ -2266,13 +2289,7 @@ internal_equal (Lisp_Object o1, Lisp_Object o2, enum equal_kind equal_kind,
   switch (XTYPE (o1))
     {
     case Lisp_Float:
-      {
-	double d1 = XFLOAT_DATA (o1);
-	double d2 = XFLOAT_DATA (o2);
-	/* If d is a NaN, then d != d. Two NaNs should be `equal' even
-	   though they are not =.  */
-	return d1 == d2 || (d1 != d1 && d2 != d2);
-      }
+      return same_float (o1, o2);
 
     case Lisp_Cons:
       if (equal_kind == EQUAL_NO_QUIT)
@@ -3706,24 +3723,20 @@ HASH_INDEX (struct Lisp_Hash_Table *h, ptrdiff_t idx)
   return XINT (AREF (h->index, idx));
 }
 
-/* Compare KEY1 which has hash code HASH1 and KEY2 with hash code
-   HASH2 in hash table H using `eql'.  Value is true if KEY1 and
-   KEY2 are the same.  */
+/* Compare KEY1 and KEY2 in hash table HT using `eql'.  Value is true
+   if KEY1 and KEY2 are the same.  KEY1 and KEY2 must not be eq.  */
 
 static bool
 cmpfn_eql (struct hash_table_test *ht,
 	   Lisp_Object key1,
 	   Lisp_Object key2)
 {
-  return (FLOATP (key1)
-	  && FLOATP (key2)
-	  && XFLOAT_DATA (key1) == XFLOAT_DATA (key2));
+  return FLOATP (key1) && FLOATP (key2) && same_float (key1, key2);
 }
 
 
-/* Compare KEY1 which has hash code HASH1 and KEY2 with hash code
-   HASH2 in hash table H using `equal'.  Value is true if KEY1 and
-   KEY2 are the same.  */
+/* Compare KEY1 and KEY2 in hash table HT using `equal'.  Value is
+   true if KEY1 and KEY2 are the same.  */
 
 static bool
 cmpfn_equal (struct hash_table_test *ht,
@@ -3734,9 +3747,8 @@ cmpfn_equal (struct hash_table_test *ht,
 }
 
 
-/* Compare KEY1 which has hash code HASH1, and KEY2 with hash code
-   HASH2 in hash table H using H->user_cmp_function.  Value is true
-   if KEY1 and KEY2 are the same.  */
+/* Compare KEY1 and KEY2 in hash table HT using HT->user_cmp_function.
+   Value is true if KEY1 and KEY2 are the same.  */
 
 static bool
 cmpfn_user_defined (struct hash_table_test *ht,
@@ -4328,18 +4340,8 @@ static EMACS_UINT
 sxhash_float (double val)
 {
   EMACS_UINT hash = 0;
-  enum {
-    WORDS_PER_DOUBLE = (sizeof val / sizeof hash
-			+ (sizeof val % sizeof hash != 0))
-  };
-  union {
-    double val;
-    EMACS_UINT word[WORDS_PER_DOUBLE];
-  } u;
-  int i;
-  u.val = val;
-  memset (&u.val + 1, 0, sizeof u - sizeof u.val);
-  for (i = 0; i < WORDS_PER_DOUBLE; i++)
+  union double_and_words u = { .val = val };
+  for (int i = 0; i < WORDS_PER_DOUBLE; i++)
     hash = sxhash_combine (hash, u.word[i]);
   return SXHASH_REDUCE (hash);
 }
diff --git a/test/src/fns-tests.el b/test/src/fns-tests.el
index d9cca55..e4b9cbe 100644
--- a/test/src/fns-tests.el
+++ b/test/src/fns-tests.el
@@ -23,6 +23,17 @@
 
 (require 'cl-lib)
 
+;; Test that equality predicates work correctly on NaNs when combined
+;; with hash tables based on those predicates.  This was not the case
+;; for eql in Emacs 26.
+(ert-deftest fns-tests-equality-nan ()
+  (dolist (test (list #'eq #'eql #'equal))
+    (let* ((h (make-hash-table :test test))
+           (nan 0.0e+NaN)
+           (-nan (- nan)))
+      (puthash nan t h)
+      (should (eq (funcall test nan -nan) (gethash -nan h))))))
+
 (ert-deftest fns-tests-reverse ()
   (should-error (reverse))
   (should-error (reverse 1))
-- 
2.7.4

Re: Using the GNU GMP Library for Bignums in Emacs

[Stefan Monnier]

> I tried to reproduce this on my machine, and ran into some trouble (the code
> had warnings that caused compilation to fail). Although your patch is
> evidently intended only as a quick benchmark and not as an actual change,

Indeed (tho I have this patch installed in my "regular Emacs", so I use
it daily).

> I worry that the benchmark isn't realistic enough, as some usage of EQ in
> C code will need to change to Feql or equivalent.

I don't understand what you mean: the patch changes `EQ` itself:

    -# define EQ(x, y) lisp_h_EQ (x, y)
    +# define EQ(x, y) EQL (x, y)

where EQL is the same as Feql (except it returns a boolean instead of
a Lisp_Object).  Some uses of EQ admittedly don't need to be
redirected to EQL, so we could improve the patch to reduce its cost, but
I expect the benefit would be small.

> Also, the C code will need to change how hashing works since XHASH
> etc. must be consistent with eq.

The patch does that already, AFAIK.


        Stefan

Making 'eq' == 'eql' in bignum branch

[Paul Eggert]

[-- Attachment #1: Type: text/plain, Size: 2228 bytes --]

Ten days ago we were discussing the possibility of changing eq to be equivalent 
to eql in the bignum branch, to avoid possible compability issues where 'eq' no 
longer agrees with 'eql' (or with '=') on integer arguments. On July 18 Stefan 
wrote:

> Some uses of EQ admittedly don't need to be
> redirected to EQL, so we could improve the patch to reduce its cost, but
> I expect the benefit would be small.

It's more than "some", if we count static occurrences. Almost all uses of EQ in 
C code don't need to be redirected to EQL, because one can easily tell 
statically that at least one of the arguments must be a non-float, or that it's 
OK to use pointer comparison for some other reason. I wrote a patch along those 
lines (see attached; it's against the master branch), and when compiled with 
normal -O2 optimization (Fedora 28 x86-64, AMD Phenom II X4 910e, user+system 
time, average of 3 runs), "cd lisp; make compile-always" suffers only a 2.3% 
slowdown, better than the 4% slowdown Stefan mentioned with his simpler patch.

Stefan, have you thought about hashing floating-point objects instead, so that 
comparing pointers suffices for eql, and eq becomes equivalent to eql in a 
different way? That may well have better performance for typical Emacs 
applications, since they typically don't create a lot of floating-point numbers. 
Also, it'd be less error-prone than the attached patch, which involves static 
analysis that would typically be done by hand.


 >> Also, the C code will need to change how hashing works since XHASH
 >> etc. must be consistent with eq.
 >
 > The patch does that already, AFAIK.

I still see a problem there in your simpler patch, since cmpfn_eql isn't used in 
the eq case, when hashing floating-point values.

 >> I worry that the benchmark isn't realistic enough, as some usage of EQ in
 >> C code will need to change to Feql or equivalent.
 >
 > I don't understand what you mean: the patch changes `EQ` itself:
 >
 >      -# define EQ(x, y) lisp_h_EQ (x, y)
 >      +# define EQ(x, y) EQL (x, y)
 >
 > where EQL is the same as Feql (except it returns a boolean instead of
 > a Lisp_Object).

You're right. Sorry, I missed that.

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-Make-eq-act-like-eql.patch --]
[-- Type: text/x-patch; name="0001-Make-eq-act-like-eql.patch", Size: 31868 bytes --]

From 91c48f71f4e1a86eceae39642bc88b2781e9bbe4 Mon Sep 17 00:00:00 2001
From: Paul Eggert <eggert@cs.ucla.edu>
Date: Fri, 27 Jul 2018 13:41:10 -0700
Subject: [PATCH] Make eq act like eql

This patch changes the Lisp function eq to act like eql.
The goal is to simplify the introduction of bignums, so
that eq will still be equivalent to eql on integers.
This patch tests this by doing so for floating-point numbers.
* lisp/subr.el (memq, sxhash-eq):
Now aliases for the corresponding eql fns.
* src/bytecode.c (exec_byte_code):
* src/category.c (word_boundary_p):
* src/chartab.c (sub_char_table_ref_and_range)
(char_table_ref_and_range, optimize_sub_char_table)
(map_sub_char_table, uniprop_encode_value_run_length)
(unicode_encode_value_numeric):
* src/data.c (Feq, set_internal, set_default_internal):
* src/dispnew.c (Fframe_or_buffer_changed_p):
* src/emacs-module.c (module_eq):
* src/eval.c (defvaralias, macroexpand, Fthrow)
(process_quit_flag):
* src/fns.c (Fmember, Fassq, Frassq, Fdelq, Fplist_get)
(Fplist_put, Fplist_member, hash_lookup, hash_remove_from_table):
* src/textprop.c (interval_has_all_properties)
(interval_has_some_properties, interval_has_some_properties_list)
(property_value, set_properties, add_properties, remove_properties)
(Fnext_single_char_property_change)
(Fprevious_single_char_property_change)
(Fnext_single_property_change)
(Fprevious_single_property_change, Ftext_property_any)
(Ftext_property_not_all, copy_text_properties, text_property_list):
Use EQL, not EQ, on values not known to be safe for EQ.
* src/fns.c (WORDS_PER_DOUBLE, union double_and_words)
(same_float): Move to src/lisp.h.
(Fmemql): Remove; now written in Lisp.
(Feql): Rewrite in terms of EQL.
(assq_no_quit): Add an assertion that the key is not
a float.
(cmpfn_eql, hashfn_eq): Remove; all uses removed.
(sxhash_float): Now extern.  Accept Lisp float, not double.
All uses changed.
(Fsxhash_eql): Remove; all uses removed.
(Fmake_hash_table): Do the same thing for eql as for eq.
* src/lisp.h (lisp_h_XHASH): Look at contents of floats.
(same_float): Now INLINE.
(EQL): New function.
* src/window.c (window_wants_mode_line, window_wants_header_line):
Rewrite to avoid GCC warning.
---
 lisp/subr.el       |   3 ++
 src/bytecode.c     |   4 +-
 src/category.c     |   4 +-
 src/chartab.c      |  18 +++----
 src/data.c         |   7 +--
 src/dispnew.c      |   2 +-
 src/emacs-module.c |   2 +-
 src/eval.c         |  10 ++--
 src/fns.c          | 123 +++++++++------------------------------------
 src/lisp.h         |  43 +++++++++++++++-
 src/textprop.c     |  38 +++++++-------
 src/window.c       |  10 ++--
 12 files changed, 116 insertions(+), 148 deletions(-)

diff --git a/lisp/subr.el b/lisp/subr.el
index 6b30371a86..2ffaa79762 100644
--- a/lisp/subr.el
+++ b/lisp/subr.el
@@ -680,6 +680,9 @@ assoc-default
       (setq tail (cdr tail)))
     value))
 
+(defalias 'memql 'memq)
+(defalias 'sxhash-eql 'sxhash-eq)
+
 (defun member-ignore-case (elt list)
   "Like `member', but ignore differences in case and text representation.
 ELT must be a string.  Upper-case and lower-case letters are treated as equal.
diff --git a/src/bytecode.c b/src/bytecode.c
index 772cc982f9..8e4eeddc89 100644
--- a/src/bytecode.c
+++ b/src/bytecode.c
@@ -518,7 +518,7 @@ exec_byte_code (Lisp_Object bytestr, Lisp_Object vector, Lisp_Object maxdepth,
 	CASE (Beq):
 	  {
 	    Lisp_Object v1 = POP;
-	    TOP = EQ (v1, TOP) ? Qt : Qnil;
+	    TOP = EQL (v1, TOP) ? Qt : Qnil;
 	    NEXT;
 	  }
 
@@ -1418,7 +1418,7 @@ exec_byte_code (Lisp_Object bytestr, Lisp_Object vector, Lisp_Object maxdepth,
                   ? make_number (h->test.hashfn (&h->test, v1)) : Qnil;
 
                 for (i = h->count; 0 <= --i; )
-                  if (EQ (v1, HASH_KEY (h, i))
+                  if (EQL (v1, HASH_KEY (h, i))
                       || (h->test.cmpfn
                           && EQ (hash_code, HASH_HASH (h, i))
                           && h->test.cmpfn (&h->test, v1, HASH_KEY (h, i))))
diff --git a/src/category.c b/src/category.c
index 62bb7f1a6c..c84b747c13 100644
--- a/src/category.c
+++ b/src/category.c
@@ -397,8 +397,8 @@ word_boundary_p (int c1, int c2)
   Lisp_Object tail;
   bool default_result;
 
-  if (EQ (CHAR_TABLE_REF (Vchar_script_table, c1),
-	  CHAR_TABLE_REF (Vchar_script_table, c2)))
+  if (EQL (CHAR_TABLE_REF (Vchar_script_table, c1),
+	   CHAR_TABLE_REF (Vchar_script_table, c2)))
     {
       tail = Vword_separating_categories;
       default_result = 0;
diff --git a/src/chartab.c b/src/chartab.c
index 89983503ac..d169400cbd 100644
--- a/src/chartab.c
+++ b/src/chartab.c
@@ -283,7 +283,7 @@ sub_char_table_ref_and_range (Lisp_Object table, int c, int *from, int *to,
       else if (NILP (this_val))
 	this_val = defalt;
 
-      if (! EQ (this_val, val))
+      if (! EQL (this_val, val))
 	{
 	  *from = c + 1;
 	  break;
@@ -304,7 +304,7 @@ sub_char_table_ref_and_range (Lisp_Object table, int c, int *from, int *to,
 						 is_uniprop);
       else if (NILP (this_val))
 	this_val = defalt;
-      if (! EQ (this_val, val))
+      if (! EQL (this_val, val))
 	{
 	  *to = c - 1;
 	  break;
@@ -356,7 +356,7 @@ char_table_ref_and_range (Lisp_Object table, int c, int *from, int *to)
       else if (NILP (this_val))
 	this_val = tbl->defalt;
 
-      if (! EQ (this_val, val))
+      if (! EQL (this_val, val))
 	{
 	  *from = c + 1;
 	  break;
@@ -376,7 +376,7 @@ char_table_ref_and_range (Lisp_Object table, int c, int *from, int *to)
 						 tbl->defalt, is_uniprop);
       else if (NILP (this_val))
 	this_val = tbl->defalt;
-      if (! EQ (this_val, val))
+      if (! EQL (this_val, val))
 	{
 	  *to = c - 1;
 	  break;
@@ -684,7 +684,7 @@ optimize_sub_char_table (Lisp_Object table, Lisp_Object test)
 	}
       if (optimizable
 	  && (NILP (test) ? NILP (Fequal (this, elt)) /* defaults to `equal'. */
-	      : EQ (test, Qeq) ? !EQ (this, elt)      /* Optimize `eq' case.  */
+	      : EQ (test, Qeq) ? !EQL (this, elt)     /* Optimize `eq' case.  */
 	      : NILP (call2 (test, this, elt))))
 	optimizable = 0;
     }
@@ -791,7 +791,7 @@ map_sub_char_table (void (*c_function) (Lisp_Object, Lisp_Object, Lisp_Object),
 	{
 	  if (NILP (this))
 	    this = XCHAR_TABLE (top)->defalt;
-	  if (!EQ (val, this))
+	  if (!EQL (val, this))
 	    {
 	      bool different_value = 1;
 
@@ -811,7 +811,7 @@ map_sub_char_table (void (*c_function) (Lisp_Object, Lisp_Object, Lisp_Object),
 		      val = map_sub_char_table (c_function, function,
 						parent, arg, val, range,
 						parent);
-		      if (EQ (val, this))
+		      if (EQL (val, this))
 			different_value = 0;
 		    }
 		}
@@ -1223,7 +1223,7 @@ uniprop_encode_value_run_length (Lisp_Object table, Lisp_Object value)
   int i, size = ASIZE (XCHAR_TABLE (table)->extras[4]);
 
   for (i = 0; i < size; i++)
-    if (EQ (value, value_table[i]))
+    if (EQL (value, value_table[i]))
       break;
   if (i == size)
     wrong_type_argument (build_string ("Unicode property value"), value);
@@ -1242,7 +1242,7 @@ uniprop_encode_value_numeric (Lisp_Object table, Lisp_Object value)
 
   CHECK_NUMBER (value);
   for (i = 0; i < size; i++)
-    if (EQ (value, value_table[i]))
+    if (EQL (value, value_table[i]))
       break;
   value = make_number (i);
   if (i == size)
diff --git a/src/data.c b/src/data.c
index c8beeda720..9777010e7f 100644
--- a/src/data.c
+++ b/src/data.c
@@ -186,7 +186,8 @@ DEFUN ("eq", Feq, Seq, 2, 2, 0,
        attributes: const)
   (Lisp_Object obj1, Lisp_Object obj2)
 {
-  if (EQ (obj1, obj2))
+  /* EQL not EQ, as Lisp eq is equivalent to Lisp eql.  */
+  if (EQL (obj1, obj2))
     return Qt;
   return Qnil;
 }
@@ -1316,7 +1317,7 @@ set_internal (Lisp_Object symbol, Lisp_Object newval, Lisp_Object where,
     {
     case SYMBOL_NOWRITE:
       if (NILP (Fkeywordp (symbol))
-          || !EQ (newval, Fsymbol_value (symbol)))
+          || !EQL (newval, Fsymbol_value (symbol)))
         xsignal1 (Qsetting_constant, symbol);
       else
         /* Allow setting keywords to their own value.  */
@@ -1659,7 +1660,7 @@ set_default_internal (Lisp_Object symbol, Lisp_Object value,
     {
     case SYMBOL_NOWRITE:
       if (NILP (Fkeywordp (symbol))
-          || !EQ (value, Fsymbol_value (symbol)))
+          || !EQL (value, Fsymbol_value (symbol)))
         xsignal1 (Qsetting_constant, symbol);
       else
         /* Allow setting keywords to their own value.  */
diff --git a/src/dispnew.c b/src/dispnew.c
index fc6f9e2263..ae23c39e52 100644
--- a/src/dispnew.c
+++ b/src/dispnew.c
@@ -5896,7 +5896,7 @@ pass nil for VARIABLE.  */)
 	goto changed;
       if (idx == ASIZE (state))
 	goto changed;
-      if (!EQ (AREF (state, idx++), BVAR (XBUFFER (buf), read_only)))
+      if (!EQL (AREF (state, idx++), BVAR (XBUFFER (buf), read_only)))
 	goto changed;
       if (idx == ASIZE (state))
 	goto changed;
diff --git a/src/emacs-module.c b/src/emacs-module.c
index 5b9f6629e7..8fae36ec4f 100644
--- a/src/emacs-module.c
+++ b/src/emacs-module.c
@@ -510,7 +510,7 @@ static bool
 module_eq (emacs_env *env, emacs_value a, emacs_value b)
 {
   MODULE_FUNCTION_BEGIN_NO_CATCH (false);
-  return EQ (value_to_lisp (a), value_to_lisp (b));
+  return EQL (value_to_lisp (a), value_to_lisp (b));
 }
 
 static intmax_t
diff --git a/src/eval.c b/src/eval.c
index 5964dd1867..7a6e9bea6c 100644
--- a/src/eval.c
+++ b/src/eval.c
@@ -628,8 +628,8 @@ The return value is BASE-VARIABLE.  */)
     set_internal (base_variable, find_symbol_value (new_alias),
                   Qnil, SET_INTERNAL_BIND);
   else if (!NILP (Fboundp (new_alias))
-           && !EQ (find_symbol_value (new_alias),
-                   find_symbol_value (base_variable)))
+           && !EQL (find_symbol_value (new_alias),
+		    find_symbol_value (base_variable)))
     call2 (intern ("display-warning"),
            list3 (intern ("defvaralias"), intern ("losing-value"), new_alias),
            CALLN (Fformat_message,
@@ -1067,7 +1067,7 @@ definitions to shadow the loaded ones for use in file byte-compilation.  */)
 	}
       {
 	Lisp_Object newform = apply1 (expander, XCDR (form));
-	if (EQ (form, newform))
+	if (EQL (form, newform))
 	  break;
 	else
 	  form = newform;
@@ -1186,7 +1186,7 @@ Both TAG and VALUE are evalled.  */
       {
 	if (c->type == CATCHER_ALL)
           unwind_to_catch (c, Fcons (tag, value));
-	if (c->type == CATCHER && EQ (c->tag_or_ch, tag))
+	if (c->type == CATCHER && EQL (c->tag_or_ch, tag))
 	  unwind_to_catch (c, value);
       }
   xsignal2 (Qno_catch, tag, value);
@@ -1532,7 +1532,7 @@ process_quit_flag (void)
   Vquit_flag = Qnil;
   if (EQ (flag, Qkill_emacs))
     Fkill_emacs (Qnil);
-  if (EQ (Vthrow_on_input, flag))
+  if (EQL (Vthrow_on_input, flag))
     Fthrow (Vthrow_on_input, Qt);
   quit ();
 }
diff --git a/src/fns.c b/src/fns.c
index 5247140ead..9a5fbdcdd8 100644
--- a/src/fns.c
+++ b/src/fns.c
@@ -1441,29 +1441,6 @@ DEFUN ("elt", Felt, Selt, 2, 2, 0,
   return Faref (sequence, n);
 }
 
-enum { WORDS_PER_DOUBLE = (sizeof (double) / sizeof (EMACS_UINT)
-                          + (sizeof (double) % sizeof (EMACS_UINT) != 0)) };
-union double_and_words
-{
-  double val;
-  EMACS_UINT word[WORDS_PER_DOUBLE];
-};
-
-/* Return true if X and Y are the same floating-point value.
-   This looks at X's and Y's representation, since (unlike '==')
-   it returns true if X and Y are the same NaN.  */
-static bool
-same_float (Lisp_Object x, Lisp_Object y)
-{
-  union double_and_words
-    xu = { .val = XFLOAT_DATA (x) },
-    yu = { .val = XFLOAT_DATA (y) };
-  EMACS_UINT neql = 0;
-  for (int i = 0; i < WORDS_PER_DOUBLE; i++)
-    neql |= xu.word[i] ^ yu.word[i];
-  return !neql;
-}
-
 DEFUN ("member", Fmember, Smember, 2, 2, 0,
        doc: /* Return non-nil if ELT is an element of LIST.  Comparison done with `equal'.
 The value is actually the tail of LIST whose car is ELT.  */)
@@ -1484,31 +1461,12 @@ The value is actually the tail of LIST whose car is ELT.  */)
 {
   Lisp_Object tail = list;
   FOR_EACH_TAIL (tail)
-    if (EQ (XCAR (tail), elt))
+    if (EQL (XCAR (tail), elt))
       return tail;
   CHECK_LIST_END (tail, list);
   return Qnil;
 }
 
-DEFUN ("memql", Fmemql, Smemql, 2, 2, 0,
-       doc: /* Return non-nil if ELT is an element of LIST.  Comparison done with `eql'.
-The value is actually the tail of LIST whose car is ELT.  */)
-  (Lisp_Object elt, Lisp_Object list)
-{
-  if (!FLOATP (elt))
-    return Fmemq (elt, list);
-
-  Lisp_Object tail = list;
-  FOR_EACH_TAIL (tail)
-    {
-      Lisp_Object tem = XCAR (tail);
-      if (FLOATP (tem) && same_float (elt, tem))
-	return tail;
-    }
-  CHECK_LIST_END (tail, list);
-  return Qnil;
-}
-
 DEFUN ("assq", Fassq, Sassq, 2, 2, 0,
        doc: /* Return non-nil if KEY is `eq' to the car of an element of LIST.
 The value is actually the first element of LIST whose car is KEY.
@@ -1517,18 +1475,20 @@ Elements of LIST that are not conses are ignored.  */)
 {
   Lisp_Object tail = list;
   FOR_EACH_TAIL (tail)
-    if (CONSP (XCAR (tail)) && EQ (XCAR (XCAR (tail)), key))
+    if (CONSP (XCAR (tail)) && EQL (XCAR (XCAR (tail)), key))
       return XCAR (tail);
   CHECK_LIST_END (tail, list);
   return Qnil;
 }
 
-/* Like Fassq but never report an error and do not allow quits.
-   Use only on objects known to be non-circular lists.  */
+/* Like Fassq but never report an error and do not allow quits and use
+   EQ not EQL.  Use only on objects known to be non-circular lists,
+   and where KEY is not a float.  */
 
 Lisp_Object
 assq_no_quit (Lisp_Object key, Lisp_Object list)
 {
+  eassert (!FLOATP (key));
   for (; ! NILP (list); list = XCDR (list))
     if (CONSP (XCAR (list)) && EQ (XCAR (XCAR (list)), key))
       return XCAR (list);
@@ -1581,7 +1541,7 @@ The value is actually the first element of LIST whose cdr is KEY.  */)
 {
   Lisp_Object tail = list;
   FOR_EACH_TAIL (tail)
-    if (CONSP (XCAR (tail)) && EQ (XCDR (XCAR (tail)), key))
+    if (CONSP (XCAR (tail)) && EQL (XCDR (XCAR (tail)), key))
       return XCAR (tail);
   CHECK_LIST_END (tail, list);
   return Qnil;
@@ -1621,7 +1581,7 @@ argument.  */)
   FOR_EACH_TAIL (tail)
     {
       Lisp_Object tem = XCAR (tail);
-      if (EQ (elt, tem))
+      if (EQL (elt, tem))
 	{
 	  if (NILP (prev))
 	    list = XCDR (tail);
@@ -2084,7 +2044,7 @@ properties on the list.  This function never signals an error.  */)
     {
       if (! CONSP (XCDR (tail)))
 	break;
-      if (EQ (prop, XCAR (tail)))
+      if (EQL (prop, XCAR (tail)))
 	return XCAR (XCDR (tail));
       tail = XCDR (tail);
       if (EQ (tail, li.tortoise))
@@ -2123,7 +2083,7 @@ The PLIST is modified by side effects.  */)
       if (! CONSP (XCDR (tail)))
 	break;
 
-      if (EQ (prop, XCAR (tail)))
+      if (EQL (prop, XCAR (tail)))
 	{
 	  Fsetcar (XCDR (tail), val);
 	  return plist;
@@ -2221,10 +2181,7 @@ This differs from numeric comparison: (eql 0.0 -0.0) returns nil and
 \(eql 0.0e+NaN 0.0e+NaN) returns t, whereas `=' does the opposite.  */)
   (Lisp_Object obj1, Lisp_Object obj2)
 {
-  if (FLOATP (obj1))
-    return FLOATP (obj2) && same_float (obj1, obj2) ? Qt : Qnil;
-  else
-    return EQ (obj1, obj2) ? Qt : Qnil;
+  return EQL (obj1, obj2) ? Qt : Qnil;
 }
 
 DEFUN ("equal", Fequal, Sequal, 2, 2, 0,
@@ -2959,7 +2916,7 @@ The value is actually the tail of PLIST whose car is PROP.  */)
   Lisp_Object tail = plist;
   FOR_EACH_TAIL (tail)
     {
-      if (EQ (XCAR (tail), prop))
+      if (EQL (XCAR (tail), prop))
 	return tail;
       tail = XCDR (tail);
       if (! CONSP (tail))
@@ -3747,18 +3704,6 @@ HASH_INDEX (struct Lisp_Hash_Table *h, ptrdiff_t idx)
   return XINT (AREF (h->index, idx));
 }
 
-/* Compare KEY1 and KEY2 in hash table HT using `eql'.  Value is true
-   if KEY1 and KEY2 are the same.  KEY1 and KEY2 must not be eq.  */
-
-static bool
-cmpfn_eql (struct hash_table_test *ht,
-	   Lisp_Object key1,
-	   Lisp_Object key2)
-{
-  return FLOATP (key1) && FLOATP (key2) && same_float (key1, key2);
-}
-
-
 /* Compare KEY1 and KEY2 in hash table HT using `equal'.  Value is
    true if KEY1 and KEY2 are the same.  */
 
@@ -3770,7 +3715,6 @@ cmpfn_equal (struct hash_table_test *ht,
   return !NILP (Fequal (key1, key2));
 }
 
-
 /* Compare KEY1 and KEY2 in hash table HT using HT->user_cmp_function.
    Value is true if KEY1 and KEY2 are the same.  */
 
@@ -3782,16 +3726,6 @@ cmpfn_user_defined (struct hash_table_test *ht,
   return !NILP (call2 (ht->user_cmp_function, key1, key2));
 }
 
-/* Value is a hash code for KEY for use in hash table H which uses
-   `eq' to compare keys.  The hash code returned is guaranteed to fit
-   in a Lisp integer.  */
-
-static EMACS_UINT
-hashfn_eq (struct hash_table_test *ht, Lisp_Object key)
-{
-  return XHASH (key) ^ XTYPE (key);
-}
-
 /* Value is a hash code for KEY for use in hash table H which uses
    `equal' to compare keys.  The hash code returned is guaranteed to fit
    in a Lisp integer.  */
@@ -3809,7 +3743,7 @@ hashfn_equal (struct hash_table_test *ht, Lisp_Object key)
 static EMACS_UINT
 hashfn_eql (struct hash_table_test *ht, Lisp_Object key)
 {
-  return FLOATP (key) ? hashfn_equal (ht, key) : hashfn_eq (ht, key);
+  return FLOATP (key) ? hashfn_equal (ht, key) : XHASH (key) ^ XTYPE (key);
 }
 
 /* Value is a hash code for KEY for use in hash table H which uses as
@@ -3820,14 +3754,14 @@ static EMACS_UINT
 hashfn_user_defined (struct hash_table_test *ht, Lisp_Object key)
 {
   Lisp_Object hash = call1 (ht->user_hash_function, key);
-  return hashfn_eq (ht, hash);
+  return hashfn_eql (ht, hash);
 }
 
 struct hash_table_test const
   hashtest_eq = { LISPSYM_INITIALLY (Qeq), LISPSYM_INITIALLY (Qnil),
-		  LISPSYM_INITIALLY (Qnil), 0, hashfn_eq },
+		  LISPSYM_INITIALLY (Qnil), 0, hashfn_eql },
   hashtest_eql = { LISPSYM_INITIALLY (Qeql), LISPSYM_INITIALLY (Qnil),
-		   LISPSYM_INITIALLY (Qnil), cmpfn_eql, hashfn_eql },
+		   LISPSYM_INITIALLY (Qnil), 0, hashfn_eql },
   hashtest_equal = { LISPSYM_INITIALLY (Qequal), LISPSYM_INITIALLY (Qnil),
 		     LISPSYM_INITIALLY (Qnil), cmpfn_equal, hashfn_equal };
 
@@ -4063,7 +3997,7 @@ hash_lookup (struct Lisp_Hash_Table *h, Lisp_Object key, EMACS_UINT *hash)
   start_of_bucket = hash_code % ASIZE (h->index);
 
   for (i = HASH_INDEX (h, start_of_bucket); 0 <= i; i = HASH_NEXT (h, i))
-    if (EQ (key, HASH_KEY (h, i))
+    if (EQL (key, HASH_KEY (h, i))
 	|| (h->test.cmpfn
 	    && hash_code == XUINT (HASH_HASH (h, i))
 	    && h->test.cmpfn (&h->test, key, HASH_KEY (h, i))))
@@ -4120,7 +4054,7 @@ hash_remove_from_table (struct Lisp_Hash_Table *h, Lisp_Object key)
        0 <= i;
        i = HASH_NEXT (h, i))
     {
-      if (EQ (key, HASH_KEY (h, i))
+      if (EQL (key, HASH_KEY (h, i))
 	  || (h->test.cmpfn
 	      && hash_code == XUINT (HASH_HASH (h, i))
 	      && h->test.cmpfn (&h->test, key, HASH_KEY (h, i))))
@@ -4360,9 +4294,10 @@ sxhash_string (char const *ptr, ptrdiff_t len)
 
 /* Return a hash for the floating point value VAL.  */
 
-static EMACS_UINT
-sxhash_float (double val)
+EMACS_UINT
+sxhash_float (Lisp_Object obj)
 {
+  double val = XFLOAT_DATA (obj);
   EMACS_UINT hash = 0;
   union double_and_words u = { .val = val };
   for (int i = 0; i < WORDS_PER_DOUBLE; i++)
@@ -4481,7 +4416,7 @@ sxhash (Lisp_Object obj, int depth)
       break;
 
     case Lisp_Float:
-      hash = sxhash_float (XFLOAT_DATA (obj));
+      hash = sxhash_float (obj);
       break;
 
     default:
@@ -4501,14 +4436,6 @@ DEFUN ("sxhash-eq", Fsxhash_eq, Ssxhash_eq, 1, 1, 0,
        doc: /* Return an integer hash code for OBJ suitable for `eq'.
 If (eq A B), then (= (sxhash-eq A) (sxhash-eq B)).  */)
   (Lisp_Object obj)
-{
-  return make_number (hashfn_eq (NULL, obj));
-}
-
-DEFUN ("sxhash-eql", Fsxhash_eql, Ssxhash_eql, 1, 1, 0,
-       doc: /* Return an integer hash code for OBJ suitable for `eql'.
-If (eql A B), then (= (sxhash-eql A) (sxhash-eql B)).  */)
-  (Lisp_Object obj)
 {
   return make_number (hashfn_eql (NULL, obj));
 }
@@ -4574,9 +4501,7 @@ usage: (make-hash-table &rest KEYWORD-ARGS)  */)
   /* See if there's a `:test TEST' among the arguments.  */
   i = get_key_arg (QCtest, nargs, args, used);
   test = i ? args[i] : Qeql;
-  if (EQ (test, Qeq))
-    testdesc = hashtest_eq;
-  else if (EQ (test, Qeql))
+  if (EQ (test, Qeq) || EQ (test, Qeql))
     testdesc = hashtest_eql;
   else if (EQ (test, Qequal))
     testdesc = hashtest_equal;
@@ -5226,7 +5151,6 @@ syms_of_fns (void)
   DEFSYM (Qkey_and_value, "key-and-value");
 
   defsubr (&Ssxhash_eq);
-  defsubr (&Ssxhash_eql);
   defsubr (&Ssxhash_equal);
   defsubr (&Smake_hash_table);
   defsubr (&Scopy_hash_table);
@@ -5344,7 +5268,6 @@ this variable.  */);
   defsubr (&Selt);
   defsubr (&Smember);
   defsubr (&Smemq);
-  defsubr (&Smemql);
   defsubr (&Sassq);
   defsubr (&Sassoc);
   defsubr (&Srassq);
diff --git a/src/lisp.h b/src/lisp.h
index 96de60e467..62996b7ee5 100644
--- a/src/lisp.h
+++ b/src/lisp.h
@@ -377,7 +377,7 @@ typedef EMACS_INT Lisp_Word;
 #define lisp_h_XCDR(c) XCONS (c)->u.s.u.cdr
 #define lisp_h_XCONS(a) \
    (eassert (CONSP (a)), XUNTAG (a, Lisp_Cons, struct Lisp_Cons))
-#define lisp_h_XHASH(a) XUINT (a)
+#define lisp_h_XHASH(a) (FLOATP (a) ? sxhash_float (a) : XUINT (a))
 #ifndef GC_CHECK_CONS_LIST
 # define lisp_h_check_cons_list() ((void) 0)
 #endif
@@ -615,6 +615,8 @@ enum CHECK_LISP_OBJECT_TYPE { CHECK_LISP_OBJECT_TYPE = false };
 /* Forward declarations.  */
 
 /* Defined in this file.  */
+INLINE bool FLOATP (Lisp_Object);
+INLINE double XFLOAT_DATA (Lisp_Object);
 INLINE void set_sub_char_table_contents (Lisp_Object, ptrdiff_t,
 					      Lisp_Object);
 
@@ -625,6 +627,8 @@ extern void char_table_set (Lisp_Object, int, Lisp_Object);
 /* Defined in data.c.  */
 extern _Noreturn void wrong_type_argument (Lisp_Object, Lisp_Object);
 
+/* Defined in fns.c.  */
+EMACS_UINT sxhash_float (Lisp_Object);
 
 #ifdef CANNOT_DUMP
 enum { might_dump = false };
@@ -1125,7 +1129,13 @@ make_natnum (EMACS_INT n)
   return USE_LSB_TAG ? make_number (n) : XIL (n + (int0 << VALBITS));
 }
 
-/* Return true if X and Y are the same object.  */
+/* Return true if X and Y are the same object.  Unlike EQL, this
+   distinguishes floats with the same values but different addresses.
+   The C name EQ is a misnomer, and dates back to when Lisp eq behaved
+   like C EQ does now.  Nowadays Lisp eq is equivalent to the Lisp eql
+   and the C EQL.  C EQ is meant for low-level C code, e.g., for
+   efficiency in places where at least one argument is a
+   non-float.  */
 
 INLINE bool
 (EQ) (Lisp_Object x, Lisp_Object y)
@@ -1133,6 +1143,35 @@ INLINE bool
   return lisp_h_EQ (x, y);
 }
 
+enum { WORDS_PER_DOUBLE = (sizeof (double) / sizeof (EMACS_UINT)
+			   + (sizeof (double) % sizeof (EMACS_UINT) != 0)) };
+union double_and_words
+{
+  double val;
+  EMACS_UINT word[WORDS_PER_DOUBLE];
+};
+
+/* Return true if X and Y are the same floating-point value.
+   This looks at X's and Y's representation, since (unlike '==')
+   it returns true if X and Y are the same NaN.  */
+INLINE bool
+same_float (Lisp_Object x, Lisp_Object y)
+{
+  union double_and_words
+    xu = { .val = XFLOAT_DATA (x) },
+    yu = { .val = XFLOAT_DATA (y) };
+  EMACS_UINT neql = 0;
+  for (int i = 0; i < WORDS_PER_DOUBLE; i++)
+    neql |= xu.word[i] ^ yu.word[i];
+  return !neql;
+}
+
+INLINE bool
+EQL (Lisp_Object x, Lisp_Object y)
+{
+  return FLOATP (x) ? FLOATP (y) && same_float (x, y) : EQ (x, y);
+}
+
 /* True if the possibly-unsigned integer I doesn't fit in a Lisp fixnum.  */
 
 #define FIXNUM_OVERFLOW_P(i) \
diff --git a/src/textprop.c b/src/textprop.c
index fe5b61e2dd..8ff2d625cc 100644
--- a/src/textprop.c
+++ b/src/textprop.c
@@ -237,11 +237,11 @@ interval_has_all_properties (Lisp_Object plist, INTERVAL i)
 
       /* Go through I's plist, looking for sym1 */
       for (tail2 = i->plist; CONSP (tail2); tail2 = Fcdr (XCDR (tail2)))
-	if (EQ (sym1, XCAR (tail2)))
+	if (EQL (sym1, XCAR (tail2)))
 	  {
 	    /* Found the same property on both lists.  If the
 	       values are unequal, return false.  */
-	    if (! EQ (Fcar (XCDR (tail1)), Fcar (XCDR (tail2))))
+	    if (! EQL (Fcar (XCDR (tail1)), Fcar (XCDR (tail2))))
 	      return false;
 
 	    /* Property has same value on both lists; go to next one.  */
@@ -271,7 +271,7 @@ interval_has_some_properties (Lisp_Object plist, INTERVAL i)
 
       /* Go through i's plist, looking for tail1 */
       for (tail2 = i->plist; CONSP (tail2); tail2 = Fcdr (XCDR (tail2)))
-	if (EQ (sym, XCAR (tail2)))
+	if (EQL (sym, XCAR (tail2)))
 	  return true;
     }
 
@@ -293,7 +293,7 @@ interval_has_some_properties_list (Lisp_Object list, INTERVAL i)
 
       /* Go through i's plist, looking for tail1 */
       for (tail2 = i->plist; CONSP (tail2); tail2 = XCDR (XCDR (tail2)))
-	if (EQ (sym, XCAR (tail2)))
+	if (EQL (sym, XCAR (tail2)))
 	  return true;
     }
 
@@ -310,7 +310,7 @@ property_value (Lisp_Object plist, Lisp_Object prop)
   Lisp_Object value;
 
   while (PLIST_ELT_P (plist, value))
-    if (EQ (XCAR (plist), prop))
+    if (EQL (XCAR (plist), prop))
       return XCAR (value);
     else
       plist = XCDR (value);
@@ -334,8 +334,8 @@ set_properties (Lisp_Object properties, INTERVAL interval, Lisp_Object object)
       for (sym = interval->plist;
 	   PLIST_ELT_P (sym, value);
 	   sym = XCDR (value))
-	if (! EQ (property_value (properties, XCAR (sym)),
-		  XCAR (value)))
+	if (! EQL (property_value (properties, XCAR (sym)),
+		   XCAR (value)))
 	  {
 	    record_property_change (interval->position, LENGTH (interval),
 				    XCAR (sym), XCAR (value),
@@ -388,7 +388,7 @@ add_properties (Lisp_Object plist, INTERVAL i, Lisp_Object object,
 
       /* Go through I's plist, looking for sym1 */
       for (tail2 = i->plist; CONSP (tail2); tail2 = Fcdr (XCDR (tail2)))
-	if (EQ (sym1, XCAR (tail2)))
+	if (EQL (sym1, XCAR (tail2)))
 	  {
 	    Lisp_Object this_cdr;
 
@@ -398,7 +398,7 @@ add_properties (Lisp_Object plist, INTERVAL i, Lisp_Object object,
 
 	    /* The properties have the same value on both lists.
 	       Continue to the next property.  */
-	    if (EQ (val1, Fcar (this_cdr)))
+	    if (EQL (val1, Fcar (this_cdr)))
 	      break;
 
 	    /* Record this change in the buffer, for undo purposes.  */
@@ -473,7 +473,7 @@ remove_properties (Lisp_Object plist, Lisp_Object list, INTERVAL i, Lisp_Object
       Lisp_Object sym = XCAR (tail1);
 
       /* First, remove the symbol if it's at the head of the list */
-      while (CONSP (current_plist) && EQ (sym, XCAR (current_plist)))
+      while (CONSP (current_plist) && EQL (sym, XCAR (current_plist)))
 	{
 	  if (BUFFERP (object))
 	    record_property_change (i->position, LENGTH (i),
@@ -489,7 +489,7 @@ remove_properties (Lisp_Object plist, Lisp_Object list, INTERVAL i, Lisp_Object
       while (! NILP (tail2))
 	{
 	  Lisp_Object this = XCDR (XCDR (tail2));
-	  if (CONSP (this) && EQ (sym, XCAR (this)))
+	  if (CONSP (this) && EQL (sym, XCAR (this)))
 	    {
 	      if (BUFFERP (object))
 		record_property_change (i->position, LENGTH (i),
@@ -822,7 +822,7 @@ last valid position in OBJECT.  */)
 	      }
 
 	    value = Fget_char_property (position, prop, object);
-	    if (!EQ (value, initial_value))
+	    if (!EQL (value, initial_value))
 	      break;
 	  }
 
@@ -914,7 +914,7 @@ first valid position in OBJECT.  */)
 		    = Fget_char_property (make_number (XFASTINT (position) - 1),
 					  prop, object);
 
-		  if (!EQ (value, initial_value))
+		  if (!EQL (value, initial_value))
 		    break;
 		}
 	    }
@@ -1024,7 +1024,7 @@ past position LIMIT; return LIMIT if nothing is found before LIMIT.  */)
   here_val = textget (i->plist, prop);
   next = next_interval (i);
   while (next
-	 && EQ (here_val, textget (next->plist, prop))
+	 && EQL (here_val, textget (next->plist, prop))
 	 && (NILP (limit) || next->position < XFASTINT (limit)))
     next = next_interval (next);
 
@@ -1126,7 +1126,7 @@ back past position LIMIT; return LIMIT if nothing is found until LIMIT.  */)
   here_val = textget (i->plist, prop);
   previous = previous_interval (i);
   while (previous
-	 && EQ (here_val, textget (previous->plist, prop))
+	 && EQL (here_val, textget (previous->plist, prop))
 	 && (NILP (limit)
 	     || (previous->position + LENGTH (previous) > XFASTINT (limit))))
     previous = previous_interval (previous);
@@ -1752,7 +1752,7 @@ markers).  If OBJECT is a string, START and END are 0-based indices into it.  */
     {
       if (i->position >= e)
 	break;
-      if (EQ (textget (i->plist, property), value))
+      if (EQL (textget (i->plist, property), value))
 	{
 	  pos = i->position;
 	  if (pos < XINT (start))
@@ -1789,7 +1789,7 @@ markers).  If OBJECT is a string, START and END are 0-based indices into it.  */
     {
       if (i->position >= e)
 	break;
-      if (! EQ (textget (i->plist, property), value))
+      if (! EQL (textget (i->plist, property), value))
 	{
 	  if (i->position > s)
 	    s = i->position;
@@ -1922,7 +1922,7 @@ copy_text_properties (Lisp_Object start, Lisp_Object end, Lisp_Object src,
       if (! NILP (prop))
 	while (! NILP (plist))
 	  {
-	    if (EQ (Fcar (plist), prop))
+	    if (EQL (Fcar (plist), prop))
 	      {
 		plist = list2 (prop, Fcar (Fcdr (plist)));
 		break;
@@ -1992,7 +1992,7 @@ text_property_list (Lisp_Object object, Lisp_Object start, Lisp_Object end, Lisp
 
 	  if (!NILP (prop))
 	    for (; CONSP (plist); plist = Fcdr (XCDR (plist)))
-	      if (EQ (XCAR (plist), prop))
+	      if (EQL (XCAR (plist), prop))
 		{
 		  plist = list2 (prop, Fcar (XCDR (plist)));
 		  break;
diff --git a/src/window.c b/src/window.c
index 422b06a49f..a220e61af8 100644
--- a/src/window.c
+++ b/src/window.c
@@ -4888,13 +4888,14 @@ window_wants_mode_line (struct window *w)
 {
   Lisp_Object window_mode_line_format =
     window_parameter (w, Qmode_line_format);
+  Lisp_Object buf = WINDOW_BUFFER (w);
 
-  return ((WINDOW_LEAF_P (w)
+  return ((!NILP (buf)
 	   && !MINI_WINDOW_P (w)
 	   && !WINDOW_PSEUDO_P (w)
 	   && !EQ (window_mode_line_format, Qnone)
 	   && (!NILP (window_mode_line_format)
-	       || !NILP (BVAR (XBUFFER (WINDOW_BUFFER (w)), mode_line_format)))
+	       || !NILP (BVAR (XBUFFER (buf), mode_line_format)))
 	   && WINDOW_PIXEL_HEIGHT (w) > WINDOW_FRAME_LINE_HEIGHT (w))
 	  ? 1
 	  : 0);
@@ -4919,13 +4920,14 @@ window_wants_header_line (struct window *w)
 {
   Lisp_Object window_header_line_format =
     window_parameter (w, Qheader_line_format);
+  Lisp_Object buf = WINDOW_BUFFER (w);
 
-  return ((WINDOW_LEAF_P (w)
+  return ((!NILP (buf)
 	   && !MINI_WINDOW_P (w)
 	   && !WINDOW_PSEUDO_P (w)
 	   && !EQ (window_header_line_format, Qnone)
 	   && (!NILP (window_header_line_format)
-	       || !NILP (BVAR (XBUFFER (WINDOW_BUFFER (w)), header_line_format)))
+	       || !NILP (BVAR (XBUFFER (buf), header_line_format)))
 	   && (WINDOW_PIXEL_HEIGHT (w)
 	       > (window_wants_mode_line (w)
 		  ? 2 * WINDOW_FRAME_LINE_HEIGHT (w)
-- 
2.17.1

Re: Making 'eq' == 'eql' in bignum branch

[Stefan Monnier]

> suffers only a 2.3% slowdown, better than the 4% slowdown Stefan mentioned
> with his simpler patch.

I'm afraid you might be comparing apples and pears, since the relative
cost may vary substantially depending on the CPU's architecture.
Have you tried to measure the slowdown of my patch on your machine?

> Stefan, have you thought about hashing floating-point objects instead, so
> that comparing pointers suffices for eql, and eq becomes equivalent to eql
> in a different way?

No I have not considered this trade-off carefully.  It's true that
floats aren't used very often, but it would make floats enormously more
expensive both in CPU and memory resources.  Currently, they cost
8 bytes of allocated heap, and currently our hash-tables cost about
6 words per entry (i.e. 48 bytes per entry on 64bit systems).
Clearly we could cut that down for hash-consing uses (where key==value),
but it's still pretty bad.

> I still see a problem there in your simpler patch, since cmpfn_eql isn't
> used in the eq case, when hashing floating-point values.

Good spotting, thanks.


        Stefan

Re: Making 'eq' == 'eql' in bignum branch

[Paul Eggert]

Stefan Monnier wrote:

> I'm afraid you might be comparing apples and pears, since the relative
> cost may vary substantially depending on the CPU's architecture.
> Have you tried to measure the slowdown of my patch on your machine?

It's definitely apples-to-pears, as your patch hashes incorrectly and that might 
be a performance advantage.

For what it's worth I tried to do the measurement again, and this time got 
different numbers. Again, this is Fedora 28 x86-64, AMD Phenom II X4 910e, 
user+system time, average of 3 runs for "cd lisp; time make compile-always", 
otherwise-idle system. However, this time I started with master commit 
db80851a1f10d73f0b2c12299c3d77716bb8425a, and I did 4 runs and ignored the first 
one to avoid any startup overheads involved. This time I got: master 460.587 s, 
my patch 465.373 s (1% slower than master), your patch 469.619 s (2% slower than 
master).

>> have you thought about hashing floating-point objects instead, so
>> that comparing pointers suffices for eql, and eq becomes equivalent to eql
>> in a different way?
> 
> No I have not considered this trade-off carefully.  It's true that
> floats aren't used very often, but it would make floats enormously more
> expensive both in CPU and memory resources.  Currently, they cost
> 8 bytes of allocated heap, and currently our hash-tables cost about
> 6 words per entry (i.e. 48 bytes per entry on 64bit systems).
> Clearly we could cut that down for hash-consing uses (where key==value),
> but it's still pretty bad.

How about if we do to floats what fixnum+bignum would do to integers?

That is, on a 64-bit host, if the low-order 4 bits of the significand are all 
zero, we encode the floating-point number directly in the Lisp_Object (with a 
tag) so that the Lisp_Object need not be dereferenced and uses zero bytes of 
floating-point heap. Otherwise, we put the floating-point value into a special 
hash-consed table that should use about 16 bytes per entry (one for the double, 
one for the pointer to it). I just checked 'make compile-always' and under this 
proposal a full run would create 95694 Lisp floats of which 80048 (84%) would 
use zero bytes and 15646 (16%) would use 16 bytes of floating-point heap. This 
would mean a total floating-point heap usage of 15646*16 = 250 kB, as opposed to 
current master which uses 95694*8 == 766 kB, i.e., we'd shrink floating-point 
heap usage by about a factor of 3.

The tradeoff would be different on a 32-bit platform, since in the above 
discussion we would replace '4 bits' with '36 bits', '15620 (16%)' with '16083 
(17%)', and '16 bytes per entry' with '12 bytes per entry'. But it'd be an 
even-bigger space win, as the total heap usage of 16083*12 = 193 kB of heap 
would be even smaller than the 64-bit total, i.e., we'd shrink floating-point 
heap usage by about a factor of 4.

We could of course use even less storage simply by lowering the precision of 
Emacs Lisp floating-point, although that'd be a user-visible change, would be a 
fairly drastic thing to do on 32-bit platforms, and wouldn't really address the 
bignum eql issue. (Though it might be interesting to try anyway....)

Re: Making 'eq' == 'eql' in bignum branch

[Stefan Monnier]

> That is, on a 64-bit host, if the low-order 4 bits of the significand are
> all zero, we encode the floating-point number directly in the Lisp_Object
> (with a tag) so that the Lisp_Object need not be dereferenced and uses zero
> bytes of floating-point heap.

Sounds like an interesting optimization, yes.

> into a special hash-consed table that should use about 16 bytes per entry
> (one for the double, one for the pointer to it).

I assume by "one for the pointer to it" you meant "one for the pointer
to the next member of the same bucket".

> I just checked 'make
> compile-always' and under this proposal a full run would create 95694 Lisp
> floats of which 80048 (84%) would use zero bytes and 15646 (16%) would use
> 16 bytes of floating-point heap.

I'm not sure this is very relevant: for "make compile-always" the
resource cost of floats is irrelevant anyway (as evidenced by the fact
that only 95694 of them are created).

Similarly for bignums: if we only use them to slightly extend the reach
of integers past the 2^29 or 2^63 limit, then it's OK to hash-cons them,
but if/when someone comes around and uses them for computing with large
numbers he might be disappointed at the resulting performance.
Tho maybe it's not that bad: contrarily to operations of floats (which
are basically super-fast on nowadays CPUs) given the non-linear
algorithmic cost of many bignum operations, adding hash-consing to them
wouldn't slow them down that much


        Stefan

Re: Making 'eq' == 'eql' in bignum branch

[Eli Zaretskii]

> From: Paul Eggert <eggert@cs.ucla.edu>
> Date: Sat, 28 Jul 2018 20:34:32 -0700
> 
> How about if we do to floats what fixnum+bignum would do to integers?
> 
> That is, on a 64-bit host, if the low-order 4 bits of the significand are all 
> zero, we encode the floating-point number directly in the Lisp_Object (with a 
> tag) so that the Lisp_Object need not be dereferenced and uses zero bytes of 
> floating-point heap. Otherwise, we put the floating-point value into a special 
> hash-consed table that should use about 16 bytes per entry (one for the double, 
> one for the pointer to it).

How likely is it to see a floating-point number with 4 low-order bits
of mantissa zero?

Re: Making 'eq' == 'eql' in bignum branch

[Stefan Monnier]

> How likely is it to see a floating-point number with 4 low-order bits
> of mantissa zero?

For floating point values read from Elisp source code, the probability
is not bad at all.  OTOH for code performing any non-trivial
calculation, of course, the probably should hover around 1/2^4, so
pretty low.


        Stefan

Re: Making 'eq' == 'eql' in bignum branch

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Date: Mon, 30 Jul 2018 13:07:35 -0400
> 
> > How likely is it to see a floating-point number with 4 low-order bits
> > of mantissa zero?
> 
> For floating point values read from Elisp source code, the probability
> is not bad at all.  OTOH for code performing any non-trivial
> calculation, of course, the probably should hover around 1/2^4, so
> pretty low.

That's what I thought.  So given this, aren't we optimizing a rare
situation, and OTOH making the code more complex and perhaps less
efficient for the majority of use cases?

Re: Making 'eq' == 'eql' in bignum branch

[Stefan Monnier]

> That's what I thought.  So given this, aren't we optimizing a rare
> situation, and OTOH making the code more complex and perhaps less
> efficient for the majority of use cases?

I think we would be, yes (but I don't know if Paul really intended to
implement this suggestion).


        Stefan

Re: Making 'eq' == 'eql' in bignum branch

[Paul Eggert]

Eli Zaretskii wrote:
> aren't we optimizing a rare situation,

Not at all. In practice, floating point computations favor values like 0.0 and 
100.0 where the bottom 3 bits of the representation are zero. True, computations 
involving random floating-point numbers will get such values only 1/8 of the 
time; but Emacs computations are not typically random.

Any changes in this area should be aimed at typical Emacs usage. If we can get a 
performance win when Emacs compiles itself, that'd be a good sign, though of 
course not conclusive.

Re: Making 'eq' == 'eql' in bignum branch

[Eli Zaretskii]

> Cc: emacs-devel@gnu.org
> From: Paul Eggert <eggert@cs.ucla.edu>
> Date: Mon, 30 Jul 2018 14:58:15 -0700
> 
> Not at all. In practice, floating point computations favor values like 0.0 and 
> 100.0 where the bottom 3 bits of the representation are zero. True, computations 
> involving random floating-point numbers will get such values only 1/8 of the 
> time; but Emacs computations are not typically random.
> 
> Any changes in this area should be aimed at typical Emacs usage. If we can get a 
> performance win when Emacs compiles itself, that'd be a good sign, though of 
> course not conclusive.

Emacs compiling itself is not a typical use of floats in Emacs, IMO.
It's a very specialized use case, which is very rare in normal usage
of Emacs.  I typically build a production release of Emacs just once,
and then use it for many moons.

If we want to claim that typical usage of floats in Emacs favors
round values, we need to back that up by some data.

Re: Making 'eq' == 'eql' in bignum branch

[Paul Eggert]

Eli Zaretskii wrote:
> Emacs compiling itself is not a typical use of floats in Emacs, IMO.

What would be a typical use, then? It'd be hard to evaluate alternatives without 
any benchmarks at all, and in the meantime 'make compile-always', despite its 
flaws, seems to be the best one we have. (It's certainly something I'd like to 
go faster....)

Re: Making 'eq' == 'eql' in bignum branch

[Eli Zaretskii]

> Cc: monnier@iro.umontreal.ca, emacs-devel@gnu.org
> From: Paul Eggert <eggert@cs.ucla.edu>
> Date: Tue, 31 Jul 2018 00:10:35 -0700
> 
> Eli Zaretskii wrote:
> > Emacs compiling itself is not a typical use of floats in Emacs, IMO.
> 
> What would be a typical use, then? It'd be hard to evaluate alternatives without 
> any benchmarks at all

I take it we agree that if someone writes a Lisp program which does FP
calculations, their chance of having a lot of such values would be
quite low, yes?  So we should put such use cases aside.

One way of estimating what happens in the majority of uses is to find
Emacs APIs other than numerical calculations that can return floats,
and try to analyze what kind of floats will be the result.  Like
file-system-info and file-attributes, for example.

> and in the meantime 'make compile-always', despite its flaws, seems
> to be the best one we have. (It's certainly something I'd like to go
> faster....)

How faster, in absolute numbers and in percents, will your build
finish with the proposed changes?

Re: Making 'eq' == 'eql' in bignum branch

[Paul Eggert]

On 07/31/2018 09:12 AM, Eli Zaretskii wrote:
> I take it we agree that if someone writes a Lisp program which does FP
> calculations, their chance of having a lot of such values would be
> quite low, yes?  So we should put such use cases aside.

Nobody should want to write such a program since Emacs is not intended 
for and does not perform well with heavy-duty FP calculations. So I 
agree that we shouldn't worry about such programs; we should focus on 
"real-world Emacs", so to speak.

> One way of estimating what happens in the majority of uses is to find
> Emacs APIs other than numerical calculations that can return floats,
> and try to analyze what kind of floats will be the result.  Like
> file-system-info and file-attributes, for example.

These should be fine on a 64-bit platform. file-system-info invariably 
yields floating-point numbers with the bottom three bits zero, due to 
how modern file systems are built (their sizes are multiples of powers 
of two). file-attributes returns an exact value unless your file is 
larger than 2**61 bytes (2 EiB), and once we get to the rare files 
larger than that we should be using bignums anyway. (The same could be 
said for file-system-info, of course.)

> How faster, in absolute numbers and in percents, will your build
> finish with the proposed changes?

I won't know unless I take the time to code it up.

Re: Making 'eq' == 'eql' in bignum branch

[Eli Zaretskii]

> Cc: monnier@iro.umontreal.ca, emacs-devel@gnu.org
> From: Paul Eggert <eggert@cs.ucla.edu>
> Date: Tue, 31 Jul 2018 15:44:24 -0700
> 
> > One way of estimating what happens in the majority of uses is to find
> > Emacs APIs other than numerical calculations that can return floats,
> > and try to analyze what kind of floats will be the result.  Like
> > file-system-info and file-attributes, for example.
> 
> These should be fine on a 64-bit platform. file-system-info invariably 
> yields floating-point numbers with the bottom three bits zero, due to 
> how modern file systems are built (their sizes are multiples of powers 
> of two). file-attributes returns an exact value unless your file is 
> larger than 2**61 bytes (2 EiB), and once we get to the rare files 
> larger than that we should be using bignums anyway. (The same could be 
> said for file-system-info, of course.)

OK, but those were just examples, I didn't do an exhaustive search for
such APIs.  Are there any others?

Then there are FP calculations in Lisp, e.g. in line-move-partial.
This gets executed many times in any Emacs, and I presume the values
there can be any FP number, so it's more akin to the use case of FP
programs we agreed not to consider.  Searching for 'float', 'round',
and 'floor' through the lisp/ directory brings more than 2000 hits,
quite a few in Calc, but many unrelated to Calc.  For example,
window.el seems to have many hits, as well as CC Mode, Org, and
calendar-related functions.  Also image.el, Gnus, and timer.el.

> > How faster, in absolute numbers and in percents, will your build
> > finish with the proposed changes?
> 
> I won't know unless I take the time to code it up.

OK, but do you have an estimate?  If not, how do you know it will be
faster?

Re: Making 'eq' == 'eql' in bignum branch

[Paul Eggert]

Eli Zaretskii wrote:

> OK, but those were just examples, I didn't do an exhaustive search for
> such APIs.  Are there any others?

Sure, lots, but most of them are rarely used and so are not a big deal as far as 
efficiency goes.

> Ohow do you know it will be faster?

I don't, I just have a suspicion. The only way to find out will be to try it.

Re: Making 'eq' == 'eql' in bignum branch

[Eli Zaretskii]

> Cc: monnier@iro.umontreal.ca, emacs-devel@gnu.org
> From: Paul Eggert <eggert@cs.ucla.edu>
> Date: Tue, 31 Jul 2018 23:43:18 -0700
> 
> Eli Zaretskii wrote:
> 
> > OK, but those were just examples, I didn't do an exhaustive search for
> > such APIs.  Are there any others?
> 
> Sure, lots, but most of them are rarely used and so are not a big deal as far as 
> efficiency goes.

Well, the FP calculations in Lisp are certainly in places that get
used a lot: the line-move stuff and window.el stuff.

Re: Making 'eq' == 'eql' in bignum branch

[Paul Eggert]

[-- Attachment #1: Type: text/plain, Size: 779 bytes --]

Eli Zaretskii wrote:
> Well, the FP calculations in Lisp are certainly in places that get
> used a lot: the line-move stuff and window.el stuff.

OK, I constructed a little benchmark based on some of the window.el stuff, and 
timed it with immediate floats instead of the longstanding way of implementing 
floats as pointers into the heap, and on this benchmark Emacs was 82% faster 
with immediate floats, i.e., the CPU time on my usual 64-bit platform for (bench 
1000000) shrank from 10.776 now to 5.914 seconds with immediate floats. So it 
looks promising. This was the only benchmark I tried.

Patch to Emacs and benchmark source code attached. The benchmark was 
byte-compiled. The Emacs patch is just a hack; it's not intended to be a real 
implementation.

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: flo.diff --]
[-- Type: text/x-patch; name="flo.diff", Size: 8268 bytes --]

diff --git a/configure.ac b/configure.ac
index b6918671e4..74200dd75b 100644
--- a/configure.ac
+++ b/configure.ac
@@ -1515,6 +1515,12 @@ AC_DEFUN
   CPPFLAGS="$C_SWITCH_SYSTEM $C_SWITCH_MACHINE $CPPFLAGS"
 fi
 
+AC_CHECK_SIZEOF([int])
+AC_CHECK_SIZEOF([long int])
+AC_CHECK_SIZEOF([long long int])
+AC_CHECK_SIZEOF([float])
+AC_CHECK_SIZEOF([double])
+
 # Suppress obsolescent Autoconf test for size_t; Emacs assumes C99 or better.
 AC_DEFUN([AC_TYPE_SIZE_T])
 # Likewise for obsolescent test for uid_t, gid_t; Emacs assumes them.
diff --git a/src/alloc.c b/src/alloc.c
index ad716f543c..763df583ab 100644
--- a/src/alloc.c
+++ b/src/alloc.c
@@ -2643,12 +2643,14 @@ static int float_block_index = FLOAT_BLOCK_SIZE;
 
 static struct Lisp_Float *float_free_list;
 
+#if FLOAT_REPR == FLOAT_TAGGED
+
 /* Return a new float object with value FLOAT_VALUE.  */
 
 Lisp_Object
-make_float (double float_value)
+make_tagged_float (double float_value)
 {
-  register Lisp_Object val;
+  Lisp_Object val;
 
   MALLOC_BLOCK_INPUT;
 
@@ -2685,6 +2687,7 @@ make_float (double float_value)
   return val;
 }
 
+#endif
 
 \f
 /***********************************************************************
@@ -5505,6 +5508,7 @@ static Lisp_Object
 purecopy (Lisp_Object obj)
 {
   if (INTEGERP (obj)
+      || (FLOAT_REPR != FLOAT_TAGGED && FLOATP (obj))
       || (! SYMBOLP (obj) && PURE_P (XPNTR (obj)))
       || SUBRP (obj))
     return obj;    /* Already pure.  */
@@ -6600,8 +6604,11 @@ mark_object (Lisp_Object arg)
       }
 
     case Lisp_Float:
-      CHECK_ALLOCATED_AND_LIVE (live_float_p);
-      FLOAT_MARK (XFLOAT (obj));
+      if (FLOAT_REPR == FLOAT_TAGGED)
+	{
+	  CHECK_ALLOCATED_AND_LIVE (live_float_p);
+	  FLOAT_MARK (XFLOAT (obj));
+	}
       break;
 
     case_Lisp_Int:
@@ -6673,7 +6680,7 @@ survives_gc_p (Lisp_Object obj)
       break;
 
     case Lisp_Float:
-      survives_p = FLOAT_MARKED_P (XFLOAT (obj));
+      survives_p = FLOAT_REPR != FLOAT_TAGGED || FLOAT_MARKED_P (XFLOAT (obj));
       break;
 
     default:
diff --git a/src/lisp.h b/src/lisp.h
index bdece817bd..b4ee5656c7 100644
--- a/src/lisp.h
+++ b/src/lisp.h
@@ -81,18 +81,21 @@ DEFINE_GDB_SYMBOL_END (GCTYPEBITS)
 typedef int EMACS_INT;
 typedef unsigned int EMACS_UINT;
 enum { EMACS_INT_WIDTH = INT_WIDTH, EMACS_UINT_WIDTH = UINT_WIDTH };
+#  define SIZEOF_EMACS_INT SIZEOF_INT
 #  define EMACS_INT_MAX INT_MAX
 #  define pI ""
 # elif INTPTR_MAX <= LONG_MAX && !defined WIDE_EMACS_INT
 typedef long int EMACS_INT;
 typedef unsigned long EMACS_UINT;
 enum { EMACS_INT_WIDTH = LONG_WIDTH, EMACS_UINT_WIDTH = ULONG_WIDTH };
+#  define SIZEOF_EMACS_INT SIZEOF_LONG_INT
 #  define EMACS_INT_MAX LONG_MAX
 #  define pI "l"
 # elif INTPTR_MAX <= LLONG_MAX
 typedef long long int EMACS_INT;
 typedef unsigned long long int EMACS_UINT;
 enum { EMACS_INT_WIDTH = LLONG_WIDTH, EMACS_UINT_WIDTH = ULLONG_WIDTH };
+#  define SIZEOF_EMACS_INT SIZEOF_LONG_LONG_INT
 #  define EMACS_INT_MAX LLONG_MAX
 /* MinGW supports %lld only if __USE_MINGW_ANSI_STDIO is non-zero,
    which is arranged by config.h, and (for mingw.org) if GCC is 6.0 or
@@ -618,6 +621,9 @@ enum CHECK_LISP_OBJECT_TYPE { CHECK_LISP_OBJECT_TYPE = false };
 INLINE void set_sub_char_table_contents (Lisp_Object, ptrdiff_t,
 					      Lisp_Object);
 
+/* Defined in alloc.c.  */
+extern Lisp_Object make_tagged_float (double);
+
 /* Defined in chartab.c.  */
 extern Lisp_Object char_table_ref (Lisp_Object, int);
 extern void char_table_set (Lisp_Object, int, Lisp_Object);
@@ -2640,6 +2646,34 @@ XBUFFER_OBJFWD (union Lisp_Fwd *a)
 }
 \f
 /* Lisp floating point type.  */
+
+/* Most hosts nowadays use IEEE floating point, so they use IEC 60559
+   representations, have infinities and NaNs, and do not trap on
+   exceptions.  Define IEEE_FLOATING_POINT to 1 if this host is one of the
+   typical ones.  The C11 macro __STDC_IEC_559__ is close to what is
+   wanted here, but is not quite right because Emacs does not require
+   all the features of C11 Annex F (and does not require C11 at all,
+   for that matter).  */
+
+#define IEEE_FLOATING_POINT (FLT_RADIX == 2 && FLT_MANT_DIG == 24 \
+			     && FLT_MIN_EXP == -125 && FLT_MAX_EXP == 128)
+
+/* Whether Lisp floats are represented directly in Lisp_Object values,
+   as opposed to tagged pointers to storage.  */
+#define FLOAT_TAGGED 0
+#define FLOAT_FLOAT 1
+#define FLOAT_DOUBLE 2
+#define FLOAT_REPR (! (USE_LSB_TAG && IEEE_FLOATING_POINT) ? FLOAT_TAGGED \
+		    : SIZEOF_EMACS_INT == SIZEOF_FLOAT ? FLOAT_FLOAT	\
+		    : SIZEOF_EMACS_INT == SIZEOF_DOUBLE ? FLOAT_DOUBLE	\
+		    : FLOAT_TAGGED)
+#if FLOAT_REPR == FLOAT_FLOAT
+typedef float emacs_float;
+#elif FLOAT_REPR == FLOAT_DOUBLE
+typedef double emacs_float;
+#endif
+
+/* If Lisp floats are tagged pointers, they point to this.  */
 struct Lisp_Float
   {
     union
@@ -2655,9 +2689,21 @@ INLINE bool
   return lisp_h_FLOATP (x);
 }
 
+INLINE Lisp_Object
+make_float (double d)
+{
+#if FLOAT_REPR == FLOAT_TAGGED
+  return make_tagged_float (d);
+#else
+  return XIL (((union { emacs_float f; EMACS_INT i; }) {d} . i & VALMASK)
+	      + Lisp_Float);
+#endif
+}
+
 INLINE struct Lisp_Float *
 XFLOAT (Lisp_Object a)
 {
+  eassume (FLOAT_REPR == FLOAT_TAGGED);
   eassert (FLOATP (a));
   return XUNTAG (a, Lisp_Float, struct Lisp_Float);
 }
@@ -2665,19 +2711,18 @@ XFLOAT (Lisp_Object a)
 INLINE double
 XFLOAT_DATA (Lisp_Object f)
 {
-  return XFLOAT (f)->u.data;
-}
-
-/* Most hosts nowadays use IEEE floating point, so they use IEC 60559
-   representations, have infinities and NaNs, and do not trap on
-   exceptions.  Define IEEE_FLOATING_POINT to 1 if this host is one of the
-   typical ones.  The C11 macro __STDC_IEC_559__ is close to what is
-   wanted here, but is not quite right because Emacs does not require
-   all the features of C11 Annex F (and does not require C11 at all,
-   for that matter).  */
+  union { EMACS_INT i; float f; double d; } u = { .i = XLI (f) - Lisp_Float };
 
-#define IEEE_FLOATING_POINT (FLT_RADIX == 2 && FLT_MANT_DIG == 24 \
-			     && FLT_MIN_EXP == -125 && FLT_MAX_EXP == 128)
+  switch (FLOAT_REPR)
+    {
+    case FLOAT_FLOAT:
+      return u.f;
+    case FLOAT_DOUBLE:
+      return u.d;
+    default:
+      return XFLOAT (f)->u.data;
+    }
+}
 
 /* A character, declared with the following typedef, is a member
    of some character set associated with the current buffer.  */
@@ -3696,7 +3741,6 @@ extern struct Lisp_Vector *allocate_pseudovector (int, int, int,
 				   VECSIZE (type), tag))
 
 extern bool gc_in_progress;
-extern Lisp_Object make_float (double);
 extern void display_malloc_warning (void);
 extern ptrdiff_t inhibit_garbage_collection (void);
 extern Lisp_Object build_overlay (Lisp_Object, Lisp_Object, Lisp_Object);
diff --git a/src/print.c b/src/print.c
index 34c7fa12b6..c10bbc5ac2 100644
--- a/src/print.c
+++ b/src/print.c
@@ -39,6 +39,7 @@ along with GNU Emacs.  If not, see <https://www.gnu.org/licenses/>.  */
 #include <float.h>
 #include <ftoastr.h>
 #include <math.h>
+#include <stdlib.h>
 
 #if IEEE_FLOATING_POINT
 # include <ieee754.h>
@@ -1029,7 +1030,26 @@ float_to_string (char *buf, double data)
     {
       /* Generate the fewest number of digits that represent the
 	 floating point value without losing information.  */
-      len = dtoastr (buf, FLOAT_TO_STRING_BUFSIZE - 2, 0, 0, data);
+      if (FLOAT_REPR == FLOAT_TAGGED)
+	len = dtoastr (buf, FLOAT_TO_STRING_BUFSIZE - 2, 0, 0, data);
+      else
+	for (int prec = ((fabs (data)
+			  < (FLOAT_REPR == FLOAT_FLOAT
+			     ? (double) FLT_MIN
+			     : DBL_MIN))
+			 ? 1
+			 : ((FLOAT_REPR == FLOAT_FLOAT ? FLT_DIG : DBL_DIG)
+			    - 1));
+	       ; prec++)
+	  {
+	    len = snprintf (buf, FLOAT_TO_STRING_BUFSIZE - 2,
+			    "%.*g", prec, data);
+	    if (isnan (data))
+	      break;
+	    if (XFLOAT_DATA (make_float (strtod (buf, NULL))) == data)
+	      break;
+	  }
+
       /* The decimal point must be printed, or the byte compiler can
 	 get confused (Bug#8033). */
       width = 1;

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #3: bench-float.el --]
[-- Type: text/x-emacs-lisp; name="bench-float.el", Size: 1661 bytes --]

;;; -*- lexical-binding:t -*-

;; Inspired by window--resize-child-windows-normal

(defvar this-delta 10)
(defvar window-height 50)
(defvar window-normal-size 0.5)
(defvar my-subs '(0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8))
(defvar my-tmp nil)

; (defun bench-nth (n l)
;   (while (and l (not (zerop n)))
;     (setq l (cdr l))
;     (setq n (1- n)))
;   (car l))

(defun bench-with (n)
  (let ((start (float-time (get-internal-run-time)))
	(i 0))
    (while (< i n)
      (let* ((delta-normal
              (if (= (- this-delta) window-height)
                  (- window-normal-size)
                (/ (float this-delta)
                   window-height)))
             (parent-normal 0.0))
        (dolist (sub my-subs)
          (setq parent-normal
                (+ parent-normal window-normal-size sub)))
        (dolist (sub my-subs)
	  (let ((old-normal (+ sub window-normal-size)))
            (setq my-tmp
	          (min 1.0 ; Don't get larger than 1.
		       (max (- old-normal
			       (* (/ old-normal parent-normal)
			          delta-normal))
		            ;; Don't drop below 0.
		            0.0)))))
        (let ((sum 0))
          (dolist (sub my-subs)
            (setq sum (+ sum window-normal-size sub))
            (setq my-tmp (min 1.0 (max (- 1.0 sum) 0.0))))))
      (setq i (1+ i)))
    (- (float-time (get-internal-run-time)) start)))

(defun bench-without (n)
  (let ((start (float-time (get-internal-run-time)))
	(i 0))
    (while (< i n)
      (setq i (1+ i)))
    (- (float-time (get-internal-run-time)) start)))

(defun bench (n)
  (- (bench-with n)
     (bench-without n)))

Re: Making 'eq' == 'eql' in bignum branch

[Eli Zaretskii]

> Cc: monnier@iro.umontreal.ca, emacs-devel@gnu.org
> From: Paul Eggert <eggert@cs.ucla.edu>
> Date: Fri, 3 Aug 2018 00:40:16 -0700
> 
> Eli Zaretskii wrote:
> > Well, the FP calculations in Lisp are certainly in places that get
> > used a lot: the line-move stuff and window.el stuff.
> 
> OK, I constructed a little benchmark based on some of the window.el stuff, and 
> timed it with immediate floats instead of the longstanding way of implementing 
> floats as pointers into the heap, and on this benchmark Emacs was 82% faster 
> with immediate floats, i.e., the CPU time on my usual 64-bit platform for (bench 
> 1000000) shrank from 10.776 now to 5.914 seconds with immediate floats. So it 
> looks promising. This was the only benchmark I tried.

That does sound promising, thanks.

Re: Making 'eq' == 'eql' in bignum branch

[Pip Cet]

That sounds promising! However, I wonder whether NaN-boxing isn't the
right thing to do once we have bignums (on platforms that support it,
by limiting pointers to 48 bits like x86-64 does). I'm doing that in
my experimental code, and the only real disadvantage is Qnil is no
longer an all-zeroes pattern. I believe that's true for 61-bit float
"immediates" as well, though, because they need the all-zero tag bits.

(My current code is limited to 32-bit integers, which causes some
other problems, but hopefully those will go away once bignum support
is merged in.)

I also wonder how many of the non-immediate floats are infinities, or
other numbers that have all low-order mantissa bits set. We could
catch that by using two tags for floats, one for the all-zero case and
one for the all-ones case.

Note that NaN-boxing doesn't necessarily mean we treat integers as
floating-point values; integers get NaN-boxed, too, so floatp remains
meaningful for those places in our codebase that treat their fixnum
arguments differently from floats.

Re: Making 'eq' == 'eql' in bignum branch

[Stefan Monnier]

> That sounds promising! However, I wonder whether NaN-boxing isn't the
> right thing to do once we have bignums (on platforms that support it,
> by limiting pointers to 48 bits like x86-64 does). I'm doing that in

NaN boxing implies 64bit boxes, which on 32bit platforms imposes
a non-negligible overhead.  So, I think it'd end up costing
significantly more than the tentative patch with which Paul and I have
been playing.


        Stefan

Re: Making 'eq' == 'eql' in bignum branch

[Pip Cet]

On Fri, Aug 10, 2018 at 3:15 PM Stefan Monnier <monnier@iro.umontreal.ca> wrote:
> > That sounds promising! However, I wonder whether NaN-boxing isn't the
> > right thing to do once we have bignums (on platforms that support it,
> > by limiting pointers to 48 bits like x86-64 does). I'm doing that in
>
> NaN boxing implies 64bit boxes, which on 32bit platforms imposes
> a non-negligible overhead.

Sorry, I don't understand. I don't see how any of this applies to
32-bit platforms with 32-bit EMACS_INTs. On those, we cannot have
float immediates or NaN boxing.

> So, I think it'd end up costing
> significantly more than the tentative patch with which Paul and I have
> been playing.

I think we only have a benchmark for 61-bit floats, which would mean
giving up IEEE compliance and surprising the occasional user who wants
it. That's a very high price to pay for saving a few GCs.

Re: Making 'eq' == 'eql' in bignum branch

[Stefan Monnier]

>> NaN boxing implies 64bit boxes, which on 32bit platforms imposes
>> a non-negligible overhead.
> Sorry, I don't understand. I don't see how any of this applies to
> 32-bit platforms with 32-bit EMACS_INTs. On those, we cannot have
> float immediates or NaN boxing.

Indeed.  But we want eq == eql on all platforms, including 32bit ones.

>> So, I think it'd end up costing
>> significantly more than the tentative patch with which Paul and I have
>> been playing.
> I think we only have a benchmark for 61-bit floats, which would mean

His idea was to use "immediate floats" when possible (e.g. with 4 zero
bits) and use hash-consed boxed floats for the rest, hence preserving
full 64bit float precision.

IOW the point was to reduce the proportion of floats we need to
hash-cons.


        Stefan

Re: Making 'eq' == 'eql' in bignum branch

[John Yates]

On Fri, Aug 10, 2018 at 11:44 AM Pip Cet <pipcet@gmail.com> wrote:
> giving up IEEE compliance and surprising the occasional user who wants it.

I have not been paying much attention to this thread so perhaps I am
preaching to the choir here...

In my day job one of my multi-year efforts has been to bring Mathwork's
MATLAB optimization and generated code into ever greater compliance
with the IEEE 754 standard.  As such I have much exposure to both the
specifics of the standard and the (often erroneous) expectations of users.

The standard talks about values, not objects, and relationships between
values.  NaN is a value just as much as 2 or pi.

More importantly the standard says that two NaN values (even if fetched
from the same container within a single expression) _always_ compare
not equal.

Were I that "occasional user who wants" IEEE compliance in my elisp
code I would expect comparisons of NaNs (whether via either eq or eql)
to return nil.  My sense is that to achieve that effect one must _always_
compare floating point objects, even if they reside at the same address.

/john

Re: Making 'eq' == 'eql' in bignum branch

[Eli Zaretskii]

> From: John Yates <john@yates-sheets.org>
> Date: Fri, 10 Aug 2018 12:52:53 -0400
> Cc: Eli Zaretskii <eliz@gnu.org>, Paul Eggert <eggert@cs.ucla.edu>,
> 	Stefan Monnier <monnier@iro.umontreal.ca>,
> 	Emacs developers <emacs-devel@gnu.org>
> 
> More importantly the standard says that two NaN values (even if fetched
> from the same container within a single expression) _always_ compare
> not equal.
> 
> Were I that "occasional user who wants" IEEE compliance in my elisp
> code I would expect comparisons of NaNs (whether via either eq or eql)
> to return nil.  My sense is that to achieve that effect one must _always_
> compare floating point objects, even if they reside at the same address.

We decided to follow IEEE for numerical comparisons ('='), but not for
object comparisons.  I'm using IEEE fp calculations all the time on my
daytime job, and it still makes no sense to me to compare NaN
_objects_ not equal.  IEEE only talks about comparing numerical
values.

Re: Making 'eq' == 'eql' in bignum branch

[Paul Eggert]

Pip Cet wrote:
> That sounds promising! However, I wonder whether NaN-boxing isn't the
> right thing to do once we have bignums (on platforms that support it,
> by limiting pointers to 48 bits like x86-64 does).
I worry that NaN-boxing is less portable. Won't it have problems, for example, 
on a system that uses ASLR and where randomized virtual addresses exceed the 
2**52 limit?

I suppose, though, that if all the Javascript guys are doing NaN boxing then we 
can live in their shadow.
> the only real disadvantage is Qnil is no
> longer an all-zeroes pattern. I believe that's true for 61-bit float
> "immediates" as well, though, because they need the all-zero tag bits.

Actually, Qnil is still all-zeroes with my patch. XFLOAT removes the Lisp_Float 
tag bits before treating the resulting Lisp_Object value as a double. I expect 
this costs an instruction with XFLOAT, and that it's worth it to keep NILP fast.

We can play a similar trick with NaN boxing, so that Qnil is all-zeros there 
too. For example, we could complement all the bits in a Lisp_Object value before 
treating it as a double. So I don't view this as being a major advantage for the 
approach I suggested.

> I also wonder how many of the non-immediate floats are infinities, or
> other numbers that have all low-order mantissa bits set.

Infinities have the significand bits all zero, no? I'm skeptical that this is a 
profitable way to explore.

> I don't see how any of this applies to
> 32-bit platforms with 32-bit EMACS_INTs. On those, we cannot have
> float immediates or NaN boxing.

We can if we're willing to limit ourselves to 'float' precision for Emacs floats 
and at most 2**22-or-so distinct non-numeric objects. That might be doable. 
However I would rather not spend a lot of time worrying about tuning 32-bit 
EMACS_INT platforms.
> 61-bit floats would mean
> giving up IEEE compliance and surprising the occasional user who wants
> it. That's a very high price to pay for saving a few GCs.

RMS had a different opinion; he didn't think full IEEE compliance was worth all 
that much trouble <https://debbugs.gnu.org/32405#47>. Although I was originally 
striving for IEEE compliance, I'm becoming more inclined to try things along 
those lines as it should indeed be faster and I doubt whether Emacs users care 
all that much about full IEEE.

Re: Making 'eq' == 'eql' in bignum branch

[Elias Mårtenson]

[-- Attachment #1: Type: text/plain, Size: 956 bytes --]

On Wed, 1 Aug 2018, 06:45 Paul Eggert, <eggert@cs.ucla.edu> wrote:

>
> These should be fine on a 64-bit platform. file-system-info invariably
> yields floating-point numbers with the bottom three bits zero, due to
> how modern file systems are built (their sizes are multiples of powers
> of two). file-attributes returns an exact value unless your file is
> larger than 2**61 bytes (2 EiB), and once we get to the rare files
> larger than that we should be using bignums anyway. (The same could be
> said for file-system-info, of course.)
>

Sorry for butting in, but in reading this discussion I'm wondering why you
don't use the high order bits of the exponent instead of the low order bits
of the mantissa?

Sure, you'd need some bit fiddling (rotation and masking) to make it into a
proper floating point number, but now you'll find yourself in a situation
where almost 100% of floating point numbers will fit in a word with tags.

Regards,
Elias

>

[-- Attachment #2: Type: text/html, Size: 1581 bytes --]

Re: Making 'eq' == 'eql' in bignum branch

[Paul Eggert]

Elias Mårtenson wrote:
> I'm wondering why you
> don't use the high order bits of the exponent instead of the low order bits
> of the mantissa?

Because my benchmark was quick and dirty, and it was easier to mess with the low 
order bits that the code is already messing with. I have been considering using 
high order bits. (It's also possible to use some of each.)

RE: Making 'eq' == 'eql' in bignum branch

[Drew Adams]

> If we want to claim that typical usage of floats in Emacs favors
> round values, we need to back that up by some data.

+1. I almost wrote something similar. A guess might be that many uses of floats in Emacs come from computations with floats, not from literal numerals. E.g., measurements/calculations involving window pixel sizes and such.

What indications are there that many uses of floats involve round numbers? I guess conversion of integers would be the main source. But do we have any quantitative info regarding relative frequency?

Re: Making 'eq' == 'eql' in bignum branch

[Eli Zaretskii]

> Date: Tue, 31 Jul 2018 08:08:20 -0700 (PDT)
> From: Drew Adams <drew.adams@oracle.com>
> Cc: monnier@iro.umontreal.ca, emacs-devel@gnu.org
> 
> What indications are there that many uses of floats involve round numbers?

I think that's because most float values we have in our sources are
round (in their binary representation).  Values like 1.0 and 0.5, for
example (but not 0.1).

Re: Making 'eq' == 'eql' in bignum branch

[Paul Eggert]

On 07/31/2018 08:08 AM, Drew Adams wrote:
> What indications are there that many uses of floats involve round numbers? I guess conversion of integers would be the main source. But do we have any quantitative info regarding relative frequency?

In the 'make compile-always' benchmark where Emacs compiles all its own 
.el files, 85% of the floating-point objects that Emacs creates have the 
low-order three bits zero. Of course this percentage is 
benchmark-specific; still, for this benchmark most of Emacs's floating 
point numbers come from things like hash-table parameters and suchlike; 
that is, even though they may not be integers they are "round numbers" 
in the sense that you mention.

Re: Making 'eq' == 'eql' in bignum branch

[Stefan Monnier]

Regarding `eq` for bignums, there seems to be some reluctance to
just defining `eq` to be an alias of the current `eql` and I think
I agree because my current approach is more costly now with the new
bignum code: `eql` used to just do

    FLOATP (x) ? costly_eql (x, y) : EQ (x, y)

where FLOATP would just check the 3 low bits of `x`.  But now we have to do

    (FLOATP (x) || BIGNUMP (x)) ? costly_eql (x, y) : EQ (x, y)

where BIGNUMP not only checks the low order bits of `x` but also looks at
the pseudovector_header of `x` if x is a pseudovector (to see if it's
a bignum), so the cost in CPU time is larger and the cost in code size is
even larger yet.

So I think hash-consing bignums makes sense.  It will bring us back the
equivalence between `eq` and `eql` for integers (hence bringing us
closer to making `eq == eql` which I think is a very worthy goal), and
I think its cost should not be too high in practice: the time to lookup
up a GMP number in a hash table basically should never be longer than
the time it took to construct this number in the first place, so that's
a worst-case slowdown factor of 2 which I think is perfectly acceptable.


        Stefan

Re: Making 'eq' == 'eql' in bignum branch

[Paul Eggert]

Stefan Monnier wrote:
> the time to lookup
> up a GMP number in a hash table basically should never be longer than
> the time it took to construct this number in the first place

Doesn't a similar argument apply to the cost of construcing an Lisp float 
(including memory allocation)? If we hashed floats as well as bignums, eq==eql 
would be trivial.

Re: Making 'eq' == 'eql' in bignum branch

[Stefan Monnier]

>> the time to lookup up a GMP number in a hash table basically should
>> never be longer than the time it took to construct this number in the
>> first place
> Doesn't a similar argument apply to the cost of construcing an Lisp float
> (including memory allocation)? If we hashed floats as well as bignums,
> eq==eql would be trivial.

Float operations are fundamentally much faster than a hash-lookup.
The fact that we heap-allocate them makes the relative cost of
hash-consing lower, but I think it's still likely to increase run-time
by a factor higher than 2x.


        Stefan

Re: Making 'eq' == 'eql' in bignum branch

[Pip Cet]

On Mon, Aug 20, 2018 at 12:05 AM Stefan Monnier
<monnier@iro.umontreal.ca> wrote:
> I think its cost should not be too high in practice: the time to lookup
> up a GMP number in a hash table basically should never be longer than
> the time it took to construct this number in the first place, so that's
> a worst-case slowdown factor of 2 which I think is perfectly acceptable.

Can you explain why you think that's the case?

I think it's currently true, but only because make_number always
creates a copy of its argument (at which point we might as well hash
it), and it sounded like Paul was going to fix that soon.

It sounds to me like a good implementation of this would require GMP
support, keeping a hash of each mpz_t in the memory allocated for it.
Perhaps we should make a wishlist of GMP features, which would
currently include three items:

 - make long-running integer operations interruptible (so C-g works)
 - don't abort() when overflowing the 16 GB limit (and/or remove that
limit entirely)
 - hash numbers as you create them, at least for the fast single-pass
operations (addition, left shift, negation).

Do we need to worry about algorithmic complexity attacks? Or about
integers created accidentally that collide in the hash?

I think 64-bit integers, at least, ought to be fast and small. Maybe
we could use our free tag value for that special case.

Re: Making 'eq' == 'eql' in bignum branch

[Paul Eggert]

Pip Cet wrote:

> I think it's currently true, but only because make_number always
> creates a copy of its argument (at which point we might as well hash
> it), and it sounded like Paul was going to fix that soon.

It's on my list of things to look at, though I can't guarantee I'll fix it (much 
less "soon").

> It sounds to me like a good implementation of this would require GMP
> support, keeping a hash of each mpz_t in the memory allocated for it.
> Perhaps we should make a wishlist of GMP features, which would
> currently include three items:
> 
>   - make long-running integer operations interruptible (so C-g works)

We can approximate this by checking for C-g in the GMP-specific memory allocator 
hooks. Perhaps all we need is some sort of guarantee that unbounded computation 
won't occur without some memory allocation action.

>   - don't abort() when overflowing the 16 GB limit (and/or remove that
> limit entirely)

Yes, that reeally needs to be fixed. I think I have a handle on avoiding that 
glitch in the meantime (still polishing it).

>   - hash numbers as you create them, at least for the fast single-pass
> operations (addition, left shift, negation).

Could well be a win, yes. Not sure how much of a win it would be for typical 
use, though. Usually bignums are pretty small.

> Do we need to worry about algorithmic complexity attacks? Or about
> integers created accidentally that collide in the hash?

Both, I'd think.
> I think 64-bit integers, at least, ought to be fast and small. Maybe
> we could use our free tag value for that special case.

Hold on, let's not spend that valuable tag so casually! Perhaps bignums or 
floats are a better use for a tag than 64-bit ints.

One other thing: we can pry another tag free. Now that we have bignums we no 
longer need to give fixnums two tags.

Re: Making 'eq' == 'eql' in bignum branch

[Pip Cet]

On Mon, Aug 20, 2018 at 7:06 AM Paul Eggert <eggert@cs.ucla.edu> wrote:
> Hold on, let's not spend that valuable tag so casually! Perhaps bignums or
> floats are a better use for a tag than 64-bit ints.

Considering it's only those 64-bit ints that don't fit into 61 (or 62,
or 30, or 29) bits already, I'm no longer convinced this is a good
idea at all. At the very least, it would make our interpreter code
harder to read.

> One other thing: we can pry another tag free. Now that we have bignums we no
> longer need to give fixnums two tags.

I think too many places might still assume that most-positive-fixnum
is precisely either one of its current values (OTOH, I'm running with
most-positive-fixnum = 0x7fffffff and things appear to work).

I think references to most-positive-fixnum (and implicit ones, like
using Frandom without an argument) should be eliminated. Fixnum range
is now an implementation detail.

Re: Making 'eq' == 'eql' in bignum branch

[Paul Eggert]

Pip Cet wrote:
> I think too many places might still assume that most-positive-fixnum
> is precisely either one of its current values (OTOH, I'm running with
> most-positive-fixnum = 0x7fffffff and things appear to work).

Older Emacs used different values, and we survived OK when we changed them. 
Plus, xemacs's values differ from both old and current Emacs, and people port 
scripts among these implementations. So I think we'd be all right if we changed 
these values again.

> I think references to most-positive-fixnum (and implicit ones, like
> using Frandom without an argument) should be eliminated. Fixnum range
> is now an implementation detail.

I tend to agree. Also, we should remove the fixnump and bignump functions; these 
functions certainly don't need to be in C code and I don't see why they even 
need to be present at all. We shouldn't be giving users the impression that the 
distinction between fixnums and bignums is important.

Re: Making 'eq' == 'eql' in bignum branch

[Andy Moreton]

On Mon 20 Aug 2018, Paul Eggert wrote:

> Pip Cet wrote:
>> I think references to most-positive-fixnum (and implicit ones, like
>> using Frandom without an argument) should be eliminated. Fixnum range
>> is now an implementation detail.

There will always be a performance diffrence between fixnum and bignum
values, and it may be useful for performance tuning to have a simple way
to discover where that boundary lies.

> I tend to agree. Also, we should remove the fixnump and bignump functions;
> these functions certainly don't need to be in C code and I don't see why they
> even need to be present at all. We shouldn't be giving users the impression
> that the distinction between fixnums and bignums is important.

These are used in the tests to ensure that implementation is correct,
and that values in fixnum range are always represented as fixnums, not
bignums. How do you propose to test that without these predicates ?

    AndyM

Re: Making 'eq' == 'eql' in bignum branch

[Paul Eggert]

Andy Moreton wrote:

> There will always be a performance diffrence between fixnum and bignum
> values, and it may be useful for performance tuning to have a simple way
> to discover where that boundary lies.

Performance nerds can use 'most-positive-fixnum' and 'most-negative-fixnum' for 
that sort of thing. These constants have been there for some time and are not 
going away. My objection is to 'bignump' and 'fixnump', which are no more 
necessary as primitives than 'negativep' would be.

> These are used in the tests to ensure that implementation is correct,
> and that values in fixnum range are always represented as fixnums, not
> bignums. How do you propose to test that without these predicates ?

Tests can use 'most-positive-fixnum' and 'most-negative-fixnum', constants that 
tests need to use anyway in order to generate values like (1+ 
most-positive-fixnum). So they can survive quite well without 'bignump' and 
'fixnump' as primitives.

Re: Making 'eq' == 'eql' in bignum branch

[Andy Moreton]

On Mon 20 Aug 2018, Paul Eggert wrote:

> Andy Moreton wrote:
>
>> There will always be a performance diffrence between fixnum and bignum
>> values, and it may be useful for performance tuning to have a simple way
>> to discover where that boundary lies.
>
> Performance nerds can use 'most-positive-fixnum' and 'most-negative-fixnum'
> for that sort of thing. These constants have been there for some time and are
> not going away. My objection is to 'bignump' and 'fixnump', which are no more
> necessary as primitives than 'negativep' would be.

Pip Cet was advocating removal of most-positive-fixnum, and that is what
I was objecting to as being too severe. We do not need negativep as we
already have other prmitives for testing that property.

>> These are used in the tests to ensure that implementation is correct,
>> and that values in fixnum range are always represented as fixnums, not
>> bignums. How do you propose to test that without these predicates ?
>
> Tests can use 'most-positive-fixnum' and 'most-negative-fixnum', constants
> that tests need to use anyway in order to generate values like (1+
> most-positive-fixnum). So they can survive quite well without 'bignump' and
> 'fixnump' as primitives.

You have avoided the question.

The current codebase assumes that lisp bignum objects only exist for
values outside fixnum range. Without fixnump how can tests check that
values within fixnum range actually have a fixnum representation ? How
can test check that no bignum object is created for a fixnum value ?

    AndyM

Re: Making 'eq' == 'eql' in bignum branch

[Paul Eggert]

Andy Moreton wrote:
> The current codebase assumes that lisp bignum objects only exist for
> values outside fixnum range. Without fixnump how can tests check that
> values within fixnum range actually have a fixnum representation

Tests cannot possibly check that, because 'fixnump' itself might simply return t 
for any integer in fixnum range, even if the integer is internally represented 
as a bignum.

Besides, the current codebase is not actually assuming that Lisp bignum objects 
exist only for values outside fixnum range (the current codebase is not assuming 
that bignums exist at all, for that matter). What the current codebase is 
assuming is alluded to in the "Subject:" line of this email, i.e., it is 
assuming that eq is equivalent to eql on fixnums. And this assumption can be 
tested directly, by doing some computations and checking that eq works as 
expected when results are in fixnum range. The test code doesn't need fixnump or 
bignump to do this sort of test.

Re: Making 'eq' == 'eql' in bignum branch

[Andy Moreton]

On Mon 20 Aug 2018, Paul Eggert wrote:

> Andy Moreton wrote:
>> The current codebase assumes that lisp bignum objects only exist for
>> values outside fixnum range. Without fixnump how can tests check that
>> values within fixnum range actually have a fixnum representation
>
> Tests cannot possibly check that, because 'fixnump' itself might simply return
> t for any integer in fixnum range, even if the integer is internally
> represented as a bignum.

Only true if the tests can make no assumptions about the implementation.
As the tests and the codebase are developed together that need not be
the case.

> Besides, the current codebase is not actually assuming that Lisp bignum
> objects exist only for values outside fixnum range (the current codebase is
> not assuming that bignums exist at all, for that matter).

The C code most certainly does assume this. The lisp code does not care
about representation (other than tripping over various bignum bugs that
need fixing).

    AndyM

Re: Making 'eq' == 'eql' in bignum branch

[Pip Cet]

On Mon, Aug 20, 2018 at 4:43 PM Andy Moreton <andrewjmoreton@gmail.com> wrote:
> On Mon 20 Aug 2018, Paul Eggert wrote:
>
> > Andy Moreton wrote:
> >
> >> There will always be a performance diffrence between fixnum and bignum
> >> values, and it may be useful for performance tuning to have a simple way
> >> to discover where that boundary lies.
> >
> > Performance nerds can use 'most-positive-fixnum' and 'most-negative-fixnum'
> > for that sort of thing. These constants have been there for some time and are
> > not going away. My objection is to 'bignump' and 'fixnump', which are no more
> > necessary as primitives than 'negativep' would be.
>
> Pip Cet was advocating removal of most-positive-fixnum, and that is what
> I was objecting to as being too severe. We do not need negativep as we
> already have other prmitives for testing that property.

I'm sorry for the sloppy wording. `most-positive-fixnum' should stay:
it's an important implementation detail. All the references to it in
lisp/, though, look to me like they should be removed.

The references in test/ should stay, for example, and I think a good
case can be made that the fixnum range should be included in Emacs bug
reports, which would be a legitimate use in the lisp/ directory.

> >> These are used in the tests to ensure that implementation is correct,
> >> and that values in fixnum range are always represented as fixnums, not
> >> bignums. How do you propose to test that without these predicates ?
> >
> > Tests can use 'most-positive-fixnum' and 'most-negative-fixnum', constants
> > that tests need to use anyway in order to generate values like (1+
> > most-positive-fixnum). So they can survive quite well without 'bignump' and
> > 'fixnump' as primitives.
>
> You have avoided the question.
>
> The current codebase assumes that lisp bignum objects only exist for
> values outside fixnum range. Without fixnump how can tests check that
> values within fixnum range actually have a fixnum representation ? How
> can test check that no bignum object is created for a fixnum value ?

I think keeping a Lisp function around for the sole purpose of testing
C code from Lisp is wrong. The right thing to do in this case is to
test the C code from C; I believe GCC does so extensively, and it
would be easy enough to add a test function that eassert()s that
FIXNUMP(Fsub1(Fadd1(MOST_POSITIVE_FIXNUM))), for example.

Again, I think important implementation details, like what a good
range for fast integer arithmetic is ([most-negative-fixnum,
most-positive-fixnum]) and how floats are encoded and whether IEEE
rules are being followed (IEEE_FLOATING_POINT), should be exposed to
Lisp. (For example, I'm playing around with big rational numbers, and
one possible implementation is for them to replace floats, but that
almost naturally requires a ratio-to-nearby-simple-ratio function to
avoid building up huge representations; whether to use such a function
and what it does would be an implementation detail, but one that code
might want to optimize for...)

Does bignump serve a purpose other than testing C code? My impression
is that it doesn't in the current implementation, and the symbol
should be left unbound, but considered reserved. While there is
hypothetical code that wants to know whether a number is currently
encoded as a bignum (I'm thinking of benchmarking code that's run in a
future Emacs with contrarily bignum-encoded fixnum-range integers), I
think it's exotic enough that it can just test (if (fboundp 'fixnump)
(fixnump x) (<= most-negative-fixnum x most-positive-fixnum)). Leaving
bignump in will lead to people thinking it's synonymous to "is this
number outside the fixnum range", and such code will then break if
it's suddenly possible for (bignump 0) to be false.

Perhaps a good analogy is that (bignump x) is on a level with "what C
address is x stored at": it's a question only very curious Lisp
programs would ask, and the answer is subject to change and dangerous
to rely on.

Re: Making 'eq' == 'eql' in bignum branch

[Andy Moreton]

On Mon 20 Aug 2018, Pip Cet wrote:

> On Mon, Aug 20, 2018 at 4:43 PM Andy Moreton <andrewjmoreton@gmail.com> wrote:
>> On Mon 20 Aug 2018, Paul Eggert wrote:
>>
>> > Andy Moreton wrote:
>> >
>> >> There will always be a performance diffrence between fixnum and bignum
>> >> values, and it may be useful for performance tuning to have a simple way
>> >> to discover where that boundary lies.
>> >
>> > Performance nerds can use 'most-positive-fixnum' and 'most-negative-fixnum'
>> > for that sort of thing. These constants have been there for some time and are
>> > not going away. My objection is to 'bignump' and 'fixnump', which are no more
>> > necessary as primitives than 'negativep' would be.
>>
>> Pip Cet was advocating removal of most-positive-fixnum, and that is what
>> I was objecting to as being too severe. We do not need negativep as we
>> already have other prmitives for testing that property.
>
> I'm sorry for the sloppy wording. `most-positive-fixnum' should stay:
> it's an important implementation detail. All the references to it in
> lisp/, though, look to me like they should be removed.

Agreed, Apologies for misunderstanding your original message.

>
> I think keeping a Lisp function around for the sole purpose of testing
> C code from Lisp is wrong. The right thing to do in this case is to
> test the C code from C; I believe GCC does so extensively, and it
> would be easy enough to add a test function that eassert()s that
> FIXNUMP(Fsub1(Fadd1(MOST_POSITIVE_FIXNUM))), for example.

A good point.

> Again, I think important implementation details, like what a good
> range for fast integer arithmetic is ([most-negative-fixnum,
> most-positive-fixnum]) and how floats are encoded and whether IEEE
> rules are being followed (IEEE_FLOATING_POINT), should be exposed to
> Lisp. (For example, I'm playing around with big rational numbers, and
> one possible implementation is for them to replace floats, but that
> almost naturally requires a ratio-to-nearby-simple-ratio function to
> avoid building up huge representations; whether to use such a function
> and what it does would be an implementation detail, but one that code
> might want to optimize for...)

Perhaps these implementation details should also be added to bug reports ?

    AndyM

Re: Making 'eq' == 'eql' in bignum branch

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

To reject fixnump and bignump seems to me like pushing our luck.
It would be wiser to define them and urge people to try to avoid them.

-- 
Dr Richard Stallman
President, Free Software Foundation (https://gnu.org, https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: Making 'eq' == 'eql' in bignum branch

[Eli Zaretskii]

> From: Paul Eggert <eggert@cs.ucla.edu>
> Date: Mon, 20 Aug 2018 09:03:17 -0700
> 
> My objection is to 'bignump' and 'fixnump', which are no more
> necessary as primitives than 'negativep' would be.

We do have natnump, though, so bignump is not as outlandish as it
might sound.

Re: Making 'eq' == 'eql' in bignum branch

[Paul Eggert]

Eli Zaretskii wrote:
>> My objection is to 'bignump' and 'fixnump', which are no more
>> necessary as primitives than 'negativep' would be.
> We do have natnump, though, so bignump is not as outlandish as it
> might sound.

You're right that bignump is not outlandish. However, it's not at all necessary, 
and we should try to keep the core interpreter as simple and clean as possible. 
There is a longstanding tradition in mathematics to talk about "natural numbers" 
and so there is a genuine need in applications for natnump that comes outside of 
Emacs. bignump is nowhere near to natnump in importance. Emacs apps don't need 
bignump, and even our test cases' use of it is unnecessary.

Re: Making 'eq' == 'eql' in bignum branch

[Eli Zaretskii]

> From: Paul Eggert <eggert@cs.ucla.edu>
> Date: Mon, 20 Aug 2018 00:27:39 -0700
> Cc: Stefan Monnier <monnier@iro.umontreal.ca>, emacs-devel@gnu.org
> 
> I tend to agree. Also, we should remove the fixnump and bignump functions; these 
> functions certainly don't need to be in C code and I don't see why they even 
> need to be present at all. We shouldn't be giving users the impression that the 
> distinction between fixnums and bignums is important.

I don't think we should go as far as removing these functions, as they
can still be useful in some situations.  Let's not rush.

Re: Making 'eq' == 'eql' in bignum branch

[Stefan Monnier]

> I don't think we should go as far as removing these functions, as they
> can still be useful in some situations.  Let's not rush.

Agreed.  How 'bout we first try to actually make use of bignums?
E.g. changing Calc to use them instead of its own implementation of bignums?


        Stefan

Re: Making 'eq' == 'eql' in bignum branch

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Date: Mon, 20 Aug 2018 11:26:14 -0400
> 
> > I don't think we should go as far as removing these functions, as they
> > can still be useful in some situations.  Let's not rush.
> 
> Agreed.  How 'bout we first try to actually make use of bignums?
> E.g. changing Calc to use them instead of its own implementation of bignums?

Yes, collecting actual experience with bignums is a good way of
figuring out what is important and what's less so.

Re: Making 'eq' == 'eql' in bignum branch

[Robert Pluim]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> I don't think we should go as far as removing these functions, as they
>> can still be useful in some situations.  Let's not rush.
>
> Agreed.  How 'bout we first try to actually make use of bignums?
> E.g. changing Calc to use them instead of its own implementation of bignums?

That probably wouldn't be too hard. Any suggestions for what kind of
tests you'd want of the result?

Robert

Re: Making 'eq' == 'eql' in bignum branch

[Robert Pluim]

Robert Pluim <rpluim@gmail.com> writes:

> Stefan Monnier <monnier@iro.umontreal.ca> writes:
>
>>> I don't think we should go as far as removing these functions, as they
>>> can still be useful in some situations.  Let's not rush.
>>
>> Agreed.  How 'bout we first try to actually make use of bignums?
>> E.g. changing Calc to use them instead of its own implementation of bignums?
>
> That probably wouldn't be too hard. Any suggestions for what kind of
> tests you'd want of the result?

First rough cut attached. It works for basic arithmetic stuff, but
still has bugs, eg when using trigonometric functions. I assume itʼs
faster, but havenʼt measured it.

diff --git i/lisp/calc/calc-aent.el w/lisp/calc/calc-aent.el
index 93aacac8cb..4e58ce8ec2 100644
--- i/lisp/calc/calc-aent.el
+++ w/lisp/calc/calc-aent.el
@@ -82,7 +82,7 @@ calc-do-quick-calc
 			       " ")
 		shortbuf buf)
 	  (if (and (= (length alg-exp) 1)
-		   (memq (car-safe (car alg-exp)) '(nil bigpos bigneg))
+;		   (memq (car-safe (car alg-exp)) '(nil bigpos bigneg))
 		   (< (length buf) 20)
 		   (= calc-number-radix 10))
 	      (setq buf (concat buf "  ("
diff --git i/lisp/calc/calc-alg.el w/lisp/calc/calc-alg.el
index 7a448d20ec..2440155ae5 100644
--- i/lisp/calc/calc-alg.el
+++ w/lisp/calc/calc-alg.el
@@ -258,9 +258,9 @@ math-beforep
 	       (and (eq comp 0)
 		    (not (equal a b))
 		    (> (length (memq (car-safe a)
-				     '(bigneg nil bigpos frac float)))
+				     '(nil frac float)))
 		       (length (memq (car-safe b)
-				     '(bigneg nil bigpos frac float))))))))
+				     '(nil frac float))))))))
 	((equal b '(neg (var inf var-inf))) nil)
 	((equal a '(neg (var inf var-inf))) t)
 	((equal a '(var inf var-inf)) nil)
diff --git i/lisp/calc/calc-bin.el w/lisp/calc/calc-bin.el
index c05a71a2d7..253e632dee 100644
--- i/lisp/calc/calc-bin.el
+++ w/lisp/calc/calc-bin.el
@@ -273,14 +273,6 @@ calcFunc-and
 					     (math-binary-arg b w)))
 		      w))))
 
-(defun math-binary-arg (a w)
-  (if (not (Math-integerp a))
-      (setq a (math-trunc a)))
-  (if (Math-integer-negp a)
-      (math-not-bignum (cdr (math-bignum-test (math-sub -1 a)))
-		       (math-abs (if w (math-trunc w) calc-word-size)))
-    (cdr (Math-bignum-test a))))
-
 (defun math-binary-modulo-args (f a b w)
   (let (mod)
     (if (eq (car-safe a) 'mod)
@@ -310,14 +302,6 @@ math-binary-modulo-args
 		       (funcall f a w))
 		     mod))))
 
-(defun math-and-bignum (a b)   ; [l l l]
-  (and a b
-       (let ((qa (math-div-bignum-digit a math-bignum-digit-power-of-two))
-	     (qb (math-div-bignum-digit b math-bignum-digit-power-of-two)))
-	 (math-mul-bignum-digit (math-and-bignum (math-norm-bignum (car qa))
-						  (math-norm-bignum (car qb)))
-				 math-bignum-digit-power-of-two
-				 (logand (cdr qa) (cdr qb))))))
 
 (defun calcFunc-or (a b &optional w)   ; [I I I] [Public]
   (cond ((Math-messy-integerp w)
@@ -337,14 +321,6 @@ calcFunc-or
 					    (math-binary-arg b w)))
 		      w))))
 
-(defun math-or-bignum (a b)   ; [l l l]
-  (and (or a b)
-       (let ((qa (math-div-bignum-digit a math-bignum-digit-power-of-two))
-	     (qb (math-div-bignum-digit b math-bignum-digit-power-of-two)))
-	 (math-mul-bignum-digit (math-or-bignum (math-norm-bignum (car qa))
-						 (math-norm-bignum (car qb)))
-				 math-bignum-digit-power-of-two
-				 (logior (cdr qa) (cdr qb))))))
 
 (defun calcFunc-xor (a b &optional w)   ; [I I I] [Public]
   (cond ((Math-messy-integerp w)
@@ -364,14 +340,6 @@ calcFunc-xor
 					     (math-binary-arg b w)))
 		      w))))
 
-(defun math-xor-bignum (a b)   ; [l l l]
-  (and (or a b)
-       (let ((qa (math-div-bignum-digit a math-bignum-digit-power-of-two))
-	     (qb (math-div-bignum-digit b math-bignum-digit-power-of-two)))
-	 (math-mul-bignum-digit (math-xor-bignum (math-norm-bignum (car qa))
-						  (math-norm-bignum (car qb)))
-				 math-bignum-digit-power-of-two
-				 (logxor (cdr qa) (cdr qb))))))
 
 (defun calcFunc-diff (a b &optional w)   ; [I I I] [Public]
   (cond ((Math-messy-integerp w)
@@ -391,14 +359,6 @@ calcFunc-diff
 					      (math-binary-arg b w)))
 		      w))))
 
-(defun math-diff-bignum (a b)   ; [l l l]
-  (and a
-       (let ((qa (math-div-bignum-digit a math-bignum-digit-power-of-two))
-	     (qb (math-div-bignum-digit b math-bignum-digit-power-of-two)))
-	 (math-mul-bignum-digit (math-diff-bignum (math-norm-bignum (car qa))
-						   (math-norm-bignum (car qb)))
-				 math-bignum-digit-power-of-two
-				 (logand (cdr qa) (lognot (cdr qb)))))))
 
 (defun calcFunc-not (a &optional w)   ; [I I] [Public]
   (cond ((Math-messy-integerp w)
@@ -416,16 +376,6 @@ calcFunc-not
 		  (math-not-bignum (math-binary-arg a w)
 				   w))))))
 
-(defun math-not-bignum (a w)   ; [l l]
-  (let ((q (math-div-bignum-digit a math-bignum-digit-power-of-two)))
-    (if (<= w math-bignum-logb-digit-size)
-	(list (logand (lognot (cdr q))
-		      (1- (lsh 1 w))))
-      (math-mul-bignum-digit (math-not-bignum (math-norm-bignum (car q))
-					       (- w math-bignum-logb-digit-size))
-			      math-bignum-digit-power-of-two
-			      (logxor (cdr q)
-                                      (1- math-bignum-digit-power-of-two))))))
 
 (defun calcFunc-lsh (a &optional n w)   ; [I I] [Public]
   (setq a (math-trunc a)
diff --git i/lisp/calc/calc-ext.el w/lisp/calc/calc-ext.el
index 5feff23f72..5922ce0acf 100644
--- i/lisp/calc/calc-ext.el
+++ w/lisp/calc/calc-ext.el
@@ -2116,45 +2116,40 @@ math-expand-formulas
 
 ;;; True if A is an odd integer.  [P R R] [Public]
 (defun math-oddp (a)
-  (if (consp a)
-      (and (memq (car a) '(bigpos bigneg))
-	   (= (% (nth 1 a) 2) 1))
-    (/= (% a 2) 0)))
+    (/= (% a 2) 0))
 
 ;;; True if A is a small or big integer.  [P x] [Public]
 (defun math-integerp (a)
-  (or (integerp a)
-      (memq (car-safe a) '(bigpos bigneg))))
+  (integerp a))
 
 ;;; True if A is (numerically) a non-negative integer.  [P N] [Public]
 (defun math-natnump (a)
-  (or (natnump a)
-      (eq (car-safe a) 'bigpos)))
+  (natnump a))
 
 ;;; True if A is a rational (or integer).  [P x] [Public]
 (defun math-ratp (a)
   (or (integerp a)
-      (memq (car-safe a) '(bigpos bigneg frac))))
+      (eq (car-safe a) 'frac)))
 
 ;;; True if A is a real (or rational).  [P x] [Public]
 (defun math-realp (a)
   (or (integerp a)
-      (memq (car-safe a) '(bigpos bigneg frac float))))
+      (memq (car-safe a) '(frac float))))
 
 ;;; True if A is a real or HMS form.  [P x] [Public]
 (defun math-anglep (a)
   (or (integerp a)
-      (memq (car-safe a) '(bigpos bigneg frac float hms))))
+      (memq (car-safe a) '(frac float hms))))
 
 ;;; True if A is a number of any kind.  [P x] [Public]
 (defun math-numberp (a)
   (or (integerp a)
-      (memq (car-safe a) '(bigpos bigneg frac float cplx polar))))
+      (memq (car-safe a) '(frac float cplx polar))))
 
 ;;; True if A is a complex number or angle.  [P x] [Public]
 (defun math-scalarp (a)
   (or (integerp a)
-      (memq (car-safe a) '(bigpos bigneg frac float cplx polar hms))))
+      (memq (car-safe a) '(frac float cplx polar hms))))
 
 ;;; True if A is a vector.  [P x] [Public]
 (defun math-vectorp (a)
@@ -2163,13 +2158,13 @@ math-vectorp
 ;;; True if A is any vector or scalar data object.  [P x]
 (defun math-objvecp (a)    ;  [Public]
   (or (integerp a)
-      (memq (car-safe a) '(bigpos bigneg frac float cplx polar
+      (memq (car-safe a) '(frac float cplx polar
 				  hms date sdev intv mod vec incomplete))))
 
 ;;; True if A is an object not composed of sub-formulas .  [P x] [Public]
 (defun math-primp (a)
   (or (integerp a)
-      (memq (car-safe a) '(bigpos bigneg frac float cplx polar
+      (memq (car-safe a) '(frac float cplx polar
 				  hms date mod var))))
 
 ;;; True if A is numerically (but not literally) an integer.  [P x] [Public]
@@ -2186,7 +2181,6 @@ math-num-integerp
 ;;; True if A is (numerically) a non-negative integer.  [P N] [Public]
 (defun math-num-natnump (a)
   (or (natnump a)
-      (eq (car-safe a) 'bigpos)
       (and (eq (car-safe a) 'float)
 	   (Math-natnump (nth 1 a))
 	   (>= (nth 2 a) 0))))
@@ -2276,7 +2270,7 @@ math-ident-row-p
 ;;; True if A is any scalar data object.  [P x]
 (defun math-objectp (a)    ;  [Public]
   (or (integerp a)
-      (memq (car-safe a) '(bigpos bigneg frac float cplx
+      (memq (car-safe a) '(frac float cplx
 				  polar hms date sdev intv mod))))
 
 ;;; Verify that A is an integer and return A in integer form.  [I N; - x]
@@ -2348,13 +2342,7 @@ math-contains-sdev-p
 
 ;;; Coerce integer A to be a small integer.  [S I]
 (defun math-fixnum (a)
-  (if (consp a)
-      (if (cdr a)
-	  (if (eq (car a) 'bigneg)
-	      (- (math-fixnum-big (cdr a)))
-	    (math-fixnum-big (cdr a)))
-	0)
-    a))
+    a)
 
 (defun math-fixnum-big (a)
   (if (cdr a)
@@ -2469,12 +2457,6 @@ math-norm-bignum
 	   (setcdr last nil)
 	   a))))
 
-(defun math-bignum-test (a)   ; [B N; B s; b b]
-  (if (consp a)
-      a
-    (math-bignum a)))
-
-
 ;;; Return 0 for zero, -1 for negative, 1 for positive.  [S n] [Public]
 (defun calcFunc-sign (a &optional x)
   (let ((signs (math-possible-signs a)))
diff --git i/lisp/calc/calc-macs.el w/lisp/calc/calc-macs.el
index 0a1f552840..8c68e30e6b 100644
--- i/lisp/calc/calc-macs.el
+++ w/lisp/calc/calc-macs.el
@@ -29,9 +29,6 @@
 (declare-function math-looks-negp "calc-misc" (a))
 (declare-function math-posp "calc-misc" (a))
 (declare-function math-compare "calc-ext" (a b))
-(declare-function math-bignum "calc" (a))
-(declare-function math-compare-bignum "calc-ext" (a b))
-
 
 (defmacro calc-wrapper (&rest body)
   `(calc-do (function (lambda ()
@@ -70,29 +67,22 @@ calc-with-trail-buffer
 ;;; Faster in-line version zerop, normalized values only.
 (defsubst Math-zerop (a)		; [P N]
   (if (consp a)
-      (and (not (memq (car a) '(bigpos bigneg)))
-	   (if (eq (car a) 'float)
-	       (eq (nth 1 a) 0)
-	     (math-zerop a)))
+      (if (eq (car a) 'float)
+	  (eq (nth 1 a) 0)
+	(math-zerop a))
     (eq a 0)))
 
 (defsubst Math-integer-negp (a)
-  (if (consp a)
-      (eq (car a) 'bigneg)
-    (< a 0)))
+    (< a 0))
 
 (defsubst Math-integer-posp (a)
-  (if (consp a)
-      (eq (car a) 'bigpos)
-    (> a 0)))
+    (> a 0))
 
 (defsubst Math-negp (a)
   (if (consp a)
-      (or (eq (car a) 'bigneg)
-	  (and (not (eq (car a) 'bigpos))
-	       (if (memq (car a) '(frac float))
-		   (Math-integer-negp (nth 1 a))
-		 (math-negp a))))
+      (if (memq (car a) '(frac float))
+	  (Math-integer-negp (nth 1 a))
+	(math-negp a))
     (< a 0)))
 
 (defsubst Math-looks-negp (a)		; [P x] [Public]
@@ -104,41 +94,38 @@ Math-looks-negp
 
 (defsubst Math-posp (a)
   (if (consp a)
-      (or (eq (car a) 'bigpos)
-	  (and (not (eq (car a) 'bigneg))
-	       (if (memq (car a) '(frac float))
-		   (Math-integer-posp (nth 1 a))
-		 (math-posp a))))
+      (if (memq (car a) '(frac float))
+	  (Math-integer-posp (nth 1 a))
+	(math-posp a))
     (> a 0)))
 
 (defsubst Math-integerp (a)
-  (or (not (consp a))
-      (memq (car a) '(bigpos bigneg))))
+  (not (consp a)))
 
 (defsubst Math-natnump (a)
   (if (consp a)
-      (eq (car a) 'bigpos)
+      nil
     (>= a 0)))
 
 (defsubst Math-ratp (a)
   (or (not (consp a))
-      (memq (car a) '(bigpos bigneg frac))))
+      (eq (car a) 'frac)))
 
 (defsubst Math-realp (a)
   (or (not (consp a))
-      (memq (car a) '(bigpos bigneg frac float))))
+      (memq (car a) '(frac float))))
 
 (defsubst Math-anglep (a)
   (or (not (consp a))
-      (memq (car a) '(bigpos bigneg frac float hms))))
+      (memq (car a) '(frac float hms))))
 
 (defsubst Math-numberp (a)
   (or (not (consp a))
-      (memq (car a) '(bigpos bigneg frac float cplx polar))))
+      (memq (car a) '(frac float cplx polar))))
 
 (defsubst Math-scalarp (a)
   (or (not (consp a))
-      (memq (car a) '(bigpos bigneg frac float cplx polar hms))))
+      (memq (car a) '(frac float cplx polar hms))))
 
 (defsubst Math-vectorp (a)
   (and (consp a) (eq (car a) 'vec)))
@@ -151,21 +138,17 @@ Math-messy-integerp
 (defsubst Math-objectp (a)		;  [Public]
   (or (not (consp a))
       (memq (car a)
-	    '(bigpos bigneg frac float cplx polar hms date sdev intv mod))))
+	    '(frac float cplx polar hms date sdev intv mod))))
 
 (defsubst Math-objvecp (a)		;  [Public]
   (or (not (consp a))
       (memq (car a)
-	    '(bigpos bigneg frac float cplx polar hms date
+	    '(frac float cplx polar hms date
 		     sdev intv mod vec))))
 
 ;;; Compute the negative of A.  [O O; o o] [Public]
 (defsubst Math-integer-neg (a)
-  (if (consp a)
-      (if (eq (car a) 'bigpos)
-	  (cons 'bigneg (cdr a))
-	(cons 'bigpos (cdr a)))
-    (- a)))
+    (- a))
 
 (defsubst Math-equal (a b)
   (= (math-compare a b) 0))
@@ -175,19 +158,14 @@ Math-lessp
 
 (defsubst Math-primp (a)
   (or (not (consp a))
-      (memq (car a) '(bigpos bigneg frac float cplx polar
+      (memq (car a) '(frac float cplx polar
 			     hms date mod var))))
 
 (defsubst Math-num-integerp (a)
   (or (not (consp a))
-      (memq (car a) '(bigpos bigneg))
       (and (eq (car a) 'float)
 	   (>= (nth 2 a) 0))))
 
-(defsubst Math-bignum-test (a)		; [B N; B s; b b]
-  (if (consp a)
-      a
-    (math-bignum a)))
 
 (defsubst Math-equal-int (a b)
   (or (eq a b)
diff --git i/lisp/calc/calc-misc.el w/lisp/calc/calc-misc.el
index 29e8510413..eebfe3834b 100644
--- i/lisp/calc/calc-misc.el
+++ w/lisp/calc/calc-misc.el
@@ -658,10 +658,7 @@ math-concat
 ;;;###autoload
 (defun math-zerop (a)
   (if (consp a)
-      (cond ((memq (car a) '(bigpos bigneg))
-	     (while (eq (car (setq a (cdr a))) 0))
-	     (null a))
-	    ((memq (car a) '(frac float polar mod))
+      (cond ((memq (car a) '(frac float polar mod))
 	     (math-zerop (nth 1 a)))
 	    ((eq (car a) 'cplx)
 	     (and (math-zerop (nth 1 a)) (math-zerop (nth 2 a))))
@@ -677,9 +674,7 @@ math-zerop
 ;;;###autoload
 (defun math-negp (a)
   (if (consp a)
-      (cond ((eq (car a) 'bigpos) nil)
-	    ((eq (car a) 'bigneg) (cdr a))
-	    ((memq (car a) '(float frac))
+      (cond ((memq (car a) '(float frac))
 	     (Math-integer-negp (nth 1 a)))
 	    ((eq (car a) 'hms)
 	     (if (math-zerop (nth 1 a))
@@ -712,9 +707,7 @@ math-looks-negp
 ;;;###autoload
 (defun math-posp (a)
   (if (consp a)
-      (cond ((eq (car a) 'bigpos) (cdr a))
-	    ((eq (car a) 'bigneg) nil)
-	    ((memq (car a) '(float frac))
+      (cond ((memq (car a) '(float frac))
 	     (Math-integer-posp (nth 1 a)))
 	    ((eq (car a) 'hms)
 	     (if (math-zerop (nth 1 a))
@@ -742,20 +735,13 @@ 'math-fixnatnump
 ;; True if A is an even integer.  [P R R] [Public]
 ;;;###autoload
 (defun math-evenp (a)
-  (if (consp a)
-      (and (memq (car a) '(bigpos bigneg))
-	   (= (% (nth 1 a) 2) 0))
-    (= (% a 2) 0)))
+    (= (% a 2) 0))
 
 ;; Compute A / 2, for small or big integer A.  [I i]
 ;; If A is negative, type of truncation is undefined.
 ;;;###autoload
 (defun math-div2 (a)
-  (if (consp a)
-      (if (cdr a)
-	  (math-normalize (cons (car a) (math-div2-bignum (cdr a))))
-	0)
-    (/ a 2)))
+    (/ a 2))
 
 ;;;###autoload
 (defun math-div2-bignum (a)   ; [l l]
diff --git i/lisp/calc/calc.el w/lisp/calc/calc.el
index 4bebd5f47b..cdf4580dde 100644
--- i/lisp/calc/calc.el
+++ w/lisp/calc/calc.el
@@ -2627,42 +2627,7 @@ math-normalize
   (setq math-normalize-error nil)
   (cond
    ((not (consp math-normalize-a))
-    (if (integerp math-normalize-a)
-	(if (or (>= math-normalize-a math-small-integer-size)
-                (<= math-normalize-a (- math-small-integer-size)))
-	    (math-bignum math-normalize-a)
-	  math-normalize-a)
-      math-normalize-a))
-   ((eq (car math-normalize-a) 'bigpos)
-    (if (eq (nth (1- (length math-normalize-a)) math-normalize-a) 0)
-	(let* ((last (setq math-normalize-a
-                           (copy-sequence math-normalize-a))) (digs math-normalize-a))
-	  (while (setq digs (cdr digs))
-	    (or (eq (car digs) 0) (setq last digs)))
-	  (setcdr last nil)))
-    (if (cdr (cdr (cdr math-normalize-a)))
-	math-normalize-a
-      (cond
-       ((cdr (cdr math-normalize-a)) (+ (nth 1 math-normalize-a)
-                                        (* (nth 2 math-normalize-a)
-                                           math-bignum-digit-size)))
-       ((cdr math-normalize-a) (nth 1 math-normalize-a))
-       (t 0))))
-   ((eq (car math-normalize-a) 'bigneg)
-    (if (eq (nth (1- (length math-normalize-a)) math-normalize-a) 0)
-	(let* ((last (setq math-normalize-a (copy-sequence math-normalize-a)))
-               (digs math-normalize-a))
-	  (while (setq digs (cdr digs))
-	    (or (eq (car digs) 0) (setq last digs)))
-	  (setcdr last nil)))
-    (if (cdr (cdr (cdr math-normalize-a)))
-	math-normalize-a
-      (cond
-       ((cdr (cdr math-normalize-a)) (- (+ (nth 1 math-normalize-a)
-                                           (* (nth 2 math-normalize-a)
-                                              math-bignum-digit-size))))
-       ((cdr math-normalize-a) (- (nth 1 math-normalize-a)))
-       (t 0))))
+      math-normalize-a)
    ((eq (car math-normalize-a) 'float)
     (math-make-float (math-normalize (nth 1 math-normalize-a))
                      (nth 2 math-normalize-a)))
@@ -2774,30 +2739,6 @@ math-check-complete
 	((consp a) a)
 	(t (error "Invalid data object encountered"))))
 
-
-
-;; Coerce integer A to be a bignum.  [B S]
-(defun math-bignum (a)
-  (cond
-   ((>= a 0)
-    (cons 'bigpos (math-bignum-big a)))
-   ((= a most-negative-fixnum)
-    ;; Note: cannot get the negation directly because
-    ;; (- most-negative-fixnum) is most-negative-fixnum.
-    ;;
-    ;; most-negative-fixnum := -most-positive-fixnum - 1
-    (math-sub (cons 'bigneg (math-bignum-big most-positive-fixnum))
-	      1))
-   (t
-    (cons 'bigneg (math-bignum-big (- a))))))
-
-(defun math-bignum-big (a)   ; [L s]
-  (if (= a 0)
-      nil
-    (cons (% a math-bignum-digit-size)
-          (math-bignum-big (/ a math-bignum-digit-size)))))
-
-
 ;; Build a normalized floating-point number.  [F I S]
 (defun math-make-float (mant exp)
   (if (eq mant 0)
@@ -2847,8 +2788,6 @@ math-float
 
 (defun math-neg (a)
   (cond ((not (consp a)) (- a))
-	((eq (car a) 'bigpos) (cons 'bigneg (cdr a)))
-	((eq (car a) 'bigneg) (cons 'bigpos (cdr a)))
 	((memq (car a) '(frac float))
 	 (list (car a) (Math-integer-neg (nth 1 a)) (nth 2 a)))
 	((memq (car a) '(cplx vec hms date calcFunc-idn))
@@ -2881,76 +2820,23 @@ math-scale-int
 (defun math-scale-left (a n)   ; [I I S]
   (if (= n 0)
       a
-    (if (consp a)
-	(cons (car a) (math-scale-left-bignum (cdr a) n))
-      (if (>= n math-bignum-digit-length)
-	  (if (or (>= a math-bignum-digit-size)
-                  (<= a (- math-bignum-digit-size)))
-	      (math-scale-left (math-bignum a) n)
-	    (math-scale-left (* a math-bignum-digit-size)
-                             (- n math-bignum-digit-length)))
-        (let ((sz (expt 10 (- (* 2 math-bignum-digit-length) n))))
-          (if (or (>= a sz) (<= a (- sz)))
-              (math-scale-left (math-bignum a) n)
-            (* a (expt 10 n))))))))
-
-(defun math-scale-left-bignum (a n)
-  (if (>= n math-bignum-digit-length)
-      (while (>= (setq a (cons 0 a)
-		       n (- n math-bignum-digit-length))
-                 math-bignum-digit-length)))
-  (if (> n 0)
-      (math-mul-bignum-digit a (expt 10 n) 0)
-    a))
+    (* a (expt 10 n))))
 
 (defun math-scale-right (a n)   ; [i i S]
   (if (= n 0)
       a
-    (if (consp a)
-	(cons (car a) (math-scale-right-bignum (cdr a) n))
-      (if (<= a 0)
-	  (if (= a 0)
-	      0
-	    (- (math-scale-right (- a) n)))
-	(if (>= n math-bignum-digit-length)
-	    (while (and (> (setq a (/ a math-bignum-digit-size)) 0)
-			(>= (setq n (- n math-bignum-digit-length))
-                            math-bignum-digit-length))))
-	(if (> n 0)
-            (/ a (expt 10 n))
-          a)))))
-
-(defun math-scale-right-bignum (a n)   ; [L L S; l l S]
-  (if (>= n math-bignum-digit-length)
-      (setq a (nthcdr (/ n math-bignum-digit-length) a)
-	    n (% n math-bignum-digit-length)))
-  (if (> n 0)
-      (cdr (math-mul-bignum-digit a (expt 10 (- math-bignum-digit-length n)) 0))
-    a))
+    (if (<= a 0)
+	(if (= a 0)
+	    0
+	  (- (math-scale-right (- a) n)))
+      (if (> n 0)
+          (/ a (expt 10 n))
+        a))))
 
 ;;; Multiply (with rounding) the integer A by 10^N.   [I i S]
 (defun math-scale-rounding (a n)
   (cond ((>= n 0)
 	 (math-scale-left a n))
-	((consp a)
-	 (math-normalize
-	  (cons (car a)
-		(let ((val (if (< n (- math-bignum-digit-length))
-			       (math-scale-right-bignum
-                                (cdr a)
-                                (- (- math-bignum-digit-length) n))
-			     (if (< n 0)
-				 (math-mul-bignum-digit
-                                  (cdr a)
-                                  (expt 10 (+ math-bignum-digit-length n)) 0)
-                               (cdr a)))))  ; n = -math-bignum-digit-length
-		  (if (and val (>= (car val) (/ math-bignum-digit-size 2)))
-		      (if (cdr val)
-			  (if (eq (car (cdr val)) (1- math-bignum-digit-size))
-			      (math-add-bignum (cdr val) '(1))
-			    (cons (1+ (car (cdr val))) (cdr (cdr val))))
-			'(1))
-		    (cdr val))))))
 	(t
 	 (if (< a 0)
 	     (- (math-scale-rounding (- a) n))
@@ -2963,36 +2849,13 @@ math-scale-rounding
 (defun math-add (a b)
   (or
    (and (not (or (consp a) (consp b)))
-	(progn
-	  (setq a (+ a b))
-	  (if (or (<= a (- math-small-integer-size)) (>= a math-small-integer-size))
-	      (math-bignum a)
-	    a)))
+	    (+ a b))
    (and (Math-zerop a) (not (eq (car-safe a) 'mod))
 	(if (and (math-floatp a) (Math-ratp b)) (math-float b) b))
    (and (Math-zerop b) (not (eq (car-safe b) 'mod))
 	(if (and (math-floatp b) (Math-ratp a)) (math-float a) a))
    (and (Math-objvecp a) (Math-objvecp b)
 	(or
-	 (and (Math-integerp a) (Math-integerp b)
-	      (progn
-		(or (consp a) (setq a (math-bignum a)))
-		(or (consp b) (setq b (math-bignum b)))
-		(if (eq (car a) 'bigneg)
-		    (if (eq (car b) 'bigneg)
-			(cons 'bigneg (math-add-bignum (cdr a) (cdr b)))
-		      (math-normalize
-		       (let ((diff (math-sub-bignum (cdr b) (cdr a))))
-			 (if (eq diff 'neg)
-			     (cons 'bigneg (math-sub-bignum (cdr a) (cdr b)))
-			   (cons 'bigpos diff)))))
-		  (if (eq (car b) 'bigneg)
-		      (math-normalize
-		       (let ((diff (math-sub-bignum (cdr a) (cdr b))))
-			 (if (eq diff 'neg)
-			     (cons 'bigneg (math-sub-bignum (cdr b) (cdr a)))
-			   (cons 'bigpos diff))))
-		    (cons 'bigpos (math-add-bignum (cdr a) (cdr b)))))))
 	 (and (Math-ratp a) (Math-ratp b)
 	      (require 'calc-ext)
 	      (calc-add-fractions a b))
@@ -3008,78 +2871,6 @@ math-add
    (and (require 'calc-ext)
 	(math-add-symb-fancy a b))))
 
-(defun math-add-bignum (a b)   ; [L L L; l l l]
-  (if a
-      (if b
-	  (let* ((a (copy-sequence a)) (aa a) (carry nil) sum)
-	    (while (and aa b)
-	      (if carry
-		  (if (< (setq sum (+ (car aa) (car b)))
-                         (1- math-bignum-digit-size))
-		      (progn
-			(setcar aa (1+ sum))
-			(setq carry nil))
-		    (setcar aa (- sum (1- math-bignum-digit-size))))
-		(if (< (setq sum (+ (car aa) (car b))) math-bignum-digit-size)
-		    (setcar aa sum)
-		  (setcar aa (- sum math-bignum-digit-size))
-		  (setq carry t)))
-	      (setq aa (cdr aa)
-		    b (cdr b)))
-	    (if carry
-		(if b
-		    (nconc a (math-add-bignum b '(1)))
-		  (while (eq (car aa) (1- math-bignum-digit-size))
-		    (setcar aa 0)
-		    (setq aa (cdr aa)))
-		  (if aa
-		      (progn
-			(setcar aa (1+ (car aa)))
-			a)
-		    (nconc a '(1))))
-	      (if b
-		  (nconc a b)
-		a)))
-	a)
-    b))
-
-(defun math-sub-bignum (a b)   ; [l l l]
-  (if b
-      (if a
-	  (let* ((a (copy-sequence a)) (aa a) (borrow nil) diff)
-	    (while (and aa b)
-	      (if borrow
-		  (if (>= (setq diff (- (car aa) (car b))) 1)
-		      (progn
-			(setcar aa (1- diff))
-			(setq borrow nil))
-		    (setcar aa (+ diff (1- math-bignum-digit-size))))
-		(if (>= (setq diff (- (car aa) (car b))) 0)
-		    (setcar aa diff)
-		  (setcar aa (+ diff math-bignum-digit-size))
-		  (setq borrow t)))
-	      (setq aa (cdr aa)
-		    b (cdr b)))
-	    (if borrow
-		(progn
-		  (while (eq (car aa) 0)
-		    (setcar aa (1- math-bignum-digit-size))
-		    (setq aa (cdr aa)))
-		  (if aa
-		      (progn
-			(setcar aa (1- (car aa)))
-			a)
-		    'neg))
-	      (while (eq (car b) 0)
-		(setq b (cdr b)))
-	      (if b
-		  'neg
-		a)))
-	(while (eq (car b) 0)
-	  (setq b (cdr b)))
-	(and b
-	     'neg))
-    a))
 
 (defun math-add-float (a b)   ; [F F F]
   (let ((ediff (- (nth 2 a) (nth 2 b))))
@@ -3100,12 +2891,7 @@ math-add-float
 
 ;;; Compute the difference of A and B.  [O O O] [Public]
 (defun math-sub (a b)
-  (if (or (consp a) (consp b))
-      (math-add a (math-neg b))
-    (setq a (- a b))
-    (if (or (<= a (- math-small-integer-size)) (>= a math-small-integer-size))
-	(math-bignum a)
-      a)))
+  (- a b))
 
 (defun math-sub-float (a b)   ; [F F F]
   (let ((ediff (- (nth 2 a) (nth 2 b))))
@@ -3130,8 +2916,6 @@ math-sub-float
 (defun math-mul (a b)
   (or
    (and (not (consp a)) (not (consp b))
-	(< a math-bignum-digit-size) (> a (- math-bignum-digit-size))
-        (< b math-bignum-digit-size) (> b (- math-bignum-digit-size))
 	(* a b))
    (and (Math-zerop a) (not (eq (car-safe b) 'mod))
 	(if (Math-scalarp b)
@@ -3145,17 +2929,6 @@ math-mul
 	  (math-mul-zero b a)))
    (and (Math-objvecp a) (Math-objvecp b)
 	(or
-	 (and (Math-integerp a) (Math-integerp b)
-	      (progn
-		(or (consp a) (setq a (math-bignum a)))
-		(or (consp b) (setq b (math-bignum b)))
-		(math-normalize
-		 (cons (if (eq (car a) (car b)) 'bigpos 'bigneg)
-		       (if (cdr (cdr a))
-			   (if (cdr (cdr b))
-			       (math-mul-bignum (cdr a) (cdr b))
-			     (math-mul-bignum-digit (cdr a) (nth 1 b) 0))
-			 (math-mul-bignum-digit (cdr b) (nth 1 a) 0))))))
 	 (and (Math-ratp a) (Math-ratp b)
 	      (require 'calc-ext)
 	      (calc-mul-fractions a b))
@@ -3184,51 +2957,6 @@ math-infinitep
 	   '(var uinf var-uinf)
 	 a)))
 
-;;; Multiply digit lists A and B.  [L L L; l l l]
-(defun math-mul-bignum (a b)
-  (and a b
-       (let* ((sum (if (<= (car b) 1)
-		       (if (= (car b) 0)
-			   (list 0)
-			 (copy-sequence a))
-		     (math-mul-bignum-digit a (car b) 0)))
-	      (sump sum) c d aa ss prod)
-	 (while (setq b (cdr b))
-	   (setq ss (setq sump (or (cdr sump) (setcdr sump (list 0))))
-		 d (car b)
-		 c 0
-		 aa a)
-	   (while (progn
-		    (setcar ss (% (setq prod (+ (+ (car ss) (* (car aa) d))
-						c))
-                                  math-bignum-digit-size))
-		    (setq aa (cdr aa)))
-	     (setq c (/ prod math-bignum-digit-size)
-		   ss (or (cdr ss) (setcdr ss (list 0)))))
-	   (if (>= prod math-bignum-digit-size)
-	       (if (cdr ss)
-		   (setcar (cdr ss) (+ (/ prod math-bignum-digit-size) (car (cdr ss))))
-		 (setcdr ss (list (/ prod math-bignum-digit-size))))))
-	 sum)))
-
-;;; Multiply digit list A by digit D.  [L L D D; l l D D]
-(defun math-mul-bignum-digit (a d c)
-  (if a
-      (if (<= d 1)
-	  (and (= d 1) a)
-	(let* ((a (copy-sequence a)) (aa a) prod)
-	  (while (progn
-		   (setcar aa
-                           (% (setq prod (+ (* (car aa) d) c))
-                              math-bignum-digit-size))
-		   (cdr aa))
-	    (setq aa (cdr aa)
-		  c (/ prod math-bignum-digit-size)))
-	  (if (>= prod math-bignum-digit-size)
-	      (setcdr aa (list (/ prod math-bignum-digit-size))))
-	  a))
-    (and (> c 0)
-	 (list c))))
 
 
 ;;; Compute the integer (quotient . remainder) of A and B, which may be
@@ -3237,93 +2965,12 @@ math-mul-bignum-digit
 (defun math-idivmod (a b)
   (if (eq b 0)
       (math-reject-arg a "*Division by zero"))
-  (if (or (consp a) (consp b))
-      (if (and (natnump b) (< b math-bignum-digit-size))
-	  (let ((res (math-div-bignum-digit (cdr a) b)))
-	    (cons
-	     (math-normalize (cons (car a) (car res)))
-	     (cdr res)))
-	(or (consp a) (setq a (math-bignum a)))
-	(or (consp b) (setq b (math-bignum b)))
-	(let ((res (math-div-bignum (cdr a) (cdr b))))
-	  (cons
-	   (math-normalize (cons (if (eq (car a) (car b)) 'bigpos 'bigneg)
-				 (car res)))
-	   (math-normalize (cons (car a) (cdr res))))))
-    (cons (/ a b) (% a b))))
+  (cons (/ a b) (% a b)))
 
 (defun math-quotient (a b)   ; [I I I] [Public]
-  (if (and (not (consp a)) (not (consp b)))
-      (if (= b 0)
-	  (math-reject-arg a "*Division by zero")
-	(/ a b))
-    (if (and (natnump b) (< b math-bignum-digit-size))
-	(if (= b 0)
-	    (math-reject-arg a "*Division by zero")
-	  (math-normalize (cons (car a)
-				(car (math-div-bignum-digit (cdr a) b)))))
-      (or (consp a) (setq a (math-bignum a)))
-      (or (consp b) (setq b (math-bignum b)))
-      (let* ((alen (1- (length a)))
-	     (blen (1- (length b)))
-	     (d (/ math-bignum-digit-size (1+ (nth (1- blen) (cdr b)))))
-	     (res (math-div-bignum-big (math-mul-bignum-digit (cdr a) d 0)
-				       (math-mul-bignum-digit (cdr b) d 0)
-				       alen blen)))
-	(math-normalize (cons (if (eq (car a) (car b)) 'bigpos 'bigneg)
-			      (car res)))))))
-
-
-;;; Divide a bignum digit list by another.  [l.l l L]
-;;; The following division algorithm is borrowed from Knuth vol. II, sec. 4.3.1
-(defun math-div-bignum (a b)
-  (if (cdr b)
-      (let* ((alen (length a))
-	     (blen (length b))
-	     (d (/ math-bignum-digit-size (1+ (nth (1- blen) b))))
-	     (res (math-div-bignum-big (math-mul-bignum-digit a d 0)
-				       (math-mul-bignum-digit b d 0)
-				       alen blen)))
-	(if (= d 1)
-	    res
-	  (cons (car res)
-		(car (math-div-bignum-digit (cdr res) d)))))
-    (let ((res (math-div-bignum-digit a (car b))))
-      (cons (car res) (list (cdr res))))))
-
-;;; Divide a bignum digit list by a digit.  [l.D l D]
-(defun math-div-bignum-digit (a b)
-  (if a
-      (let* ((res (math-div-bignum-digit (cdr a) b))
-	     (num (+ (* (cdr res) math-bignum-digit-size) (car a))))
-	(cons
-	 (cons (/ num b) (car res))
-	 (% num b)))
-    '(nil . 0)))
-
-(defun math-div-bignum-big (a b alen blen)   ; [l.l l L]
-  (if (< alen blen)
-      (cons nil a)
-    (let* ((res (math-div-bignum-big (cdr a) b (1- alen) blen))
-	   (num (cons (car a) (cdr res)))
-	   (res2 (math-div-bignum-part num b blen)))
-      (cons
-       (cons (car res2) (car res))
-       (cdr res2)))))
-
-(defun math-div-bignum-part (a b blen)   ; a < b*math-bignum-digit-size  [D.l l L]
-  (let* ((num (+ (* (or (nth blen a) 0) math-bignum-digit-size)
-                 (or (nth (1- blen) a) 0)))
-	 (den (nth (1- blen) b))
-	 (guess (min (/ num den) (1- math-bignum-digit-size))))
-    (math-div-bignum-try a b (math-mul-bignum-digit b guess 0) guess)))
-
-(defun math-div-bignum-try (a b c guess)   ; [D.l l l D]
-  (let ((rem (math-sub-bignum a c)))
-    (if (eq rem 'neg)
-	(math-div-bignum-try a b (math-sub-bignum c b) (1- guess))
-      (cons guess rem))))
-
+  (if (= b 0)
+      (math-reject-arg a "*Division by zero")
+    (/ a b)))
 
 ;;; Compute the quotient of A and B.  [O O N] [Public]
 (defun math-div (a b)
@@ -3548,11 +3195,11 @@ math-format-number
 			      (math-format-binary a)
 			    (math-format-radix a))))
 	      (math-format-radix a))))
-      (math-format-number (math-bignum a))))
+      (math-format-bignum a)))
    ((stringp a) a)
    ((not (consp a)) (prin1-to-string a))
-   ((eq (car a) 'bigpos) (math-format-bignum (cdr a)))
-   ((eq (car a) 'bigneg) (concat "-" (math-format-bignum (cdr a))))
+   ((eq (car a) 'bigpos) (error "bignum found"))
+   ((eq (car a) 'bigneg) (error "bignum found"))
    ((and (eq (car a) 'float) (= calc-number-radix 10))
     (if (Math-integer-negp (nth 1 a))
 	(concat "-" (math-format-number (math-neg a)))
@@ -3642,21 +3289,7 @@ math-format-bignum
     (math-format-bignum-fancy a)))
 
 (defun math-format-bignum-decimal (a)   ; [X L]
-  (if a
-      (let ((s ""))
-	(while (cdr (cdr a))
-	  (setq s (concat
-                   (format
-                    (concat "%0"
-                            (number-to-string (* 2 math-bignum-digit-length))
-                            "d")
-                    (+ (* (nth 1 a) math-bignum-digit-size) (car a))) s)
-		a (cdr (cdr a))))
-	(concat (int-to-string
-                 (+ (* (or (nth 1 a) 0) math-bignum-digit-size) (car a))) s))
-    "0"))
-
-
+  (number-to-string a))
 
 ;;; Parse a simple number in string form.   [N X] [Public]
 (defun math-read-number (s &optional decimal)
@@ -3673,9 +3306,7 @@ math-read-number
                   (eq (aref digs 0) ?0)
                   (null decimal))
              (math-read-number (concat "8#" digs))
-           (if (<= (length digs) (* 2 math-bignum-digit-length))
-               (string-to-number digs)
-             (cons 'bigpos (math-read-bignum digs))))))
+           (string-to-number digs))))
 
       ;; Clean up the string if necessary
       ((string-match "\\`\\(.*\\)[ \t\n]+\\([^\001]*\\)\\'" s)
@@ -3730,14 +3361,10 @@ math-read-number-simple
      ((string-match "^[0-9]+$" s)
       (if (string-match "^\\(0+\\)" s)
           (setq s (substring s (match-end 0))))
-      (if (<= (length s) (* 2 math-bignum-digit-length))
-          (string-to-number s)
-        (cons 'bigpos (math-read-bignum s))))
+          (string-to-number s))
      ;; Minus sign
      ((string-match "^-[0-9]+$" s)
-      (if (<= (length s) (1+ (* 2 math-bignum-digit-length)))
-          (string-to-number s)
-        (cons 'bigneg (math-read-bignum (substring s 1)))))
+      (string-to-number s))
      ;; Decimal point
      ((string-match "^\\(-?[0-9]*\\)\\.\\([0-9]*\\)$" s)
       (let ((int (math-match-substring s 1))
diff --git i/test/lisp/calc/calc-tests.el w/test/lisp/calc/calc-tests.el
index fbd5f0e3a1..6f17ed3691 100644
--- i/test/lisp/calc/calc-tests.el
+++ w/test/lisp/calc/calc-tests.el
@@ -62,12 +62,6 @@ calc-tests-simple
 	(calc-top-n 1))
     (calc-pop 0)))
 
-(ert-deftest test-math-bignum ()
-  ;; bug#17556
-  (let ((n (math-bignum most-negative-fixnum)))
-    (should (math-negp n))
-    (should (cl-notany #'cl-minusp (cdr n)))))
-
 (ert-deftest test-calc-remove-units ()
   (should (calc-tests-equal (calc-tests-simple #'calc-remove-units "-1 m") -1)))
 

Re: Making 'eq' == 'eql' in bignum branch

[Paul Eggert]

Robert Pluim wrote:
> First rough cut attached. It works for basic arithmetic stuff, but
> still has bugs, eg when using trigonometric functions. I assume itʼs
> faster, but havenʼt measured it.

This looks promising; thanks for looking into it.

Re: Making 'eq' == 'eql' in bignum branch

[Lars Ingebrigtsen]

Slightly off-topic, but I thought it was amusing:

I was reading a blog post on C over/underruns, and I got to this bit:

        Note that there is no value than can hold 2147483648, so if you
	negate (int)0x80000000, you get (int)0x80000000 again. That is
	something to look out for, because it means abs() returns a
	negative value when fed -2147483648.

I didn't know that, and indeed in Emacs until a few weeks ago:

(abs most-negative-fixnum) => -2305843009213693952

Now, of course, with bignum support, we get a more correct result:

(abs most-negative-fixnum) => 2305843009213693952
(bignump (abs most-negative-fixnum)) => t

So now we don't have to fix the documentation of `abs':

---
abs is a built-in function in ‘C source code’.

(abs ARG)

Return the absolute value of ARG.
---


-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: Making 'eq' == 'eql' in bignum branch

[Herring, Davis]

> I was reading a blog post on C over/underruns, and I got to this bit:
> 
>         Note that there is no value than can hold 2147483648, so if you
>         negate (int)0x80000000, you get (int)0x80000000 again. That is
>         something to look out for, because it means abs() returns a
>         negative value when fed -2147483648.
> 
> I didn't know that, [...]

You still don't, because C doesn't work that way.  Even writing (int)0x80000000 (on a system with suitably-sized ints) is implementation-defined <https://en.cppreference.com/w/c/language/conversion#Integer_conversions>.  Applying - to it (on almost any system) is overflow and hence undefined behavior <https://en.cppreference.com/w/c/language/operator_arithmetic#Unary_arithmetic>.

Of course, you have observed the "correctness" of this statement, but that's because it has well-defined (if surprising) behavior in (at least) x86 assembly <https://stackoverflow.com/questions/26440463/does-neg-instruction-in-assembly-language-sets-the-overflow-flag> and, formally, that _happened_ to be the manifestation of the undefined behavior (in the C implementation of Emacs).

On the gripping hand, when the compiler can't prove anything about an integer argument, it is reasonable for us to assume that this will be the behavior; this is far from the only place in Emacs where what is formally undefined behavior is expected to "do the obvious thing".  But it's wrong for "a blog post on C" to say that is unconditionally the behavior.

Davis

Re: Making 'eq' == 'eql' in bignum branch

[Lars Ingebrigtsen]

"Herring, Davis" <herring@lanl.gov> writes:

> But it's wrong for "a blog post on C" to say that is unconditionally
> the behavior.

It doesn't.  There are caveats galore, but including them all
(ones-complement machines and other obscure things, other word sizes
etc) didn't seem very interesting to me in an Emacs bignum context.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: Making 'eq' == 'eql' in bignum branch

[Paul Eggert]

Herring, Davis wrote:
> this is far from the only place in Emacs where what is formally undefined behavior is expected to "do the obvious thing"

Emacs does not rely on undefined behavior for integer arithmetic. Even in Emacs 
26, (abs most-negative-fixnum) yields most-negative-fixnum without doing 
anything undefined at the C level, because (- most-negative-fixnum) fits within 
machine limits and the tagging operation does not overflow.

Although there may be a few places left in Emacs that rely on undefined integer 
behavior, these are now considered bugs and occasionally I try to stamp them out 
by building with gcc -fsanitize=undefined. For example, in Emacs 26 (* 
most-positive-fixnum most-positive-fixnum) returns 1 (the overflowed value) 
without overflowing at the C level.

Emacs *does* rely on undefined behavior when it tags pointers, but that's a 
different kettle of fish.

Re: Making 'eq' == 'eql' in bignum branch

[Stefan Monnier]

>>> I don't think we should go as far as removing these functions, as they
>>> can still be useful in some situations.  Let's not rush.
>> Agreed.  How 'bout we first try to actually make use of bignums?
>> E.g. changing Calc to use them instead of its own implementation of bignums?
> That probably wouldn't be too hard.

IIUC, such changes have already been applied to the Calc that's
distributed with SXEmacs (where Calc's former maintainer moved), so it's
probably worth looking at that code (and associated doc).

> Any suggestions for what kind of tests you'd want of the result?

Fast and exhaustive tests, of course ;-)


        Stefan

Re: Making 'eq' == 'eql' in bignum branch

[Robert Pluim]

Stefan Monnier <monnier@IRO.UMontreal.CA> writes:

>>>> I don't think we should go as far as removing these functions, as they
>>>> can still be useful in some situations.  Let's not rush.
>>> Agreed.  How 'bout we first try to actually make use of bignums?
>>> E.g. changing Calc to use them instead of its own implementation of bignums?
>> That probably wouldn't be too hard.
>
> IIUC, such changes have already been applied to the Calc that's
> distributed with SXEmacs (where Calc's former maintainer moved), so it's
> probably worth looking at that code (and associated doc).

I didnʼt know about that. Would there be any issue with merging such
changes back to emacs? I assume itʼs all GPL, but there might be
copyright assignment issues.

>> Any suggestions for what kind of tests you'd want of the result?
>
> Fast and exhaustive tests, of course ;-)

It passes the test-suite, which has 5 tests :-)

In terms of speed:

(require 'calc-units)
(dotimes (i 10)
  (benchmark 1
    '(dotimes (i 10000)
       (calcFunc-mul 230584300921369395100000000
                     10000000000000000000000000000000000000000
                     234287928374298347289374298347234782937428934723489729374823
                     7492837429837))))

master:

Elapsed time: 0.687519s (0.114858s in 15 GCs)
Elapsed time: 0.683179s (0.122283s in 14 GCs)
Elapsed time: 0.688424s (0.125096s in 15 GCs)
Elapsed time: 0.683192s (0.117345s in 14 GCs)
Elapsed time: 0.691941s (0.127961s in 15 GCs)
Elapsed time: 0.692849s (0.126826s in 15 GCs)
Elapsed time: 0.679750s (0.116932s in 14 GCs)
Elapsed time: 0.682929s (0.122974s in 15 GCs)
Elapsed time: 0.689449s (0.125723s in 15 GCs)
Elapsed time: 0.707762s (0.114604s in 14 GCs)

master using native bignums:

Elapsed time: 0.074445s (0.028334s in 4 GCs)
Elapsed time: 0.057261s (0.022541s in 3 GCs)
Elapsed time: 0.063773s (0.030844s in 4 GCs)
Elapsed time: 0.056425s (0.023070s in 3 GCs)
Elapsed time: 0.065788s (0.031533s in 4 GCs)
Elapsed time: 0.065265s (0.031403s in 4 GCs)
Elapsed time: 0.056984s (0.022835s in 3 GCs)
Elapsed time: 0.064079s (0.030197s in 4 GCs)
Elapsed time: 0.057158s (0.023442s in 3 GCs)
Elapsed time: 0.071579s (0.035323s in 4 GCs)

so using native bignums appears to be ~10x faster that calc's bignums,
and create less garbage.

Re: Making 'eq' == 'eql' in bignum branch

[Eli Zaretskii]

> From: Robert Pluim <rpluim@gmail.com>
> Date: Thu, 23 Aug 2018 10:13:38 +0200
> Cc: emacs-devel@gnu.org
> 
> > IIUC, such changes have already been applied to the Calc that's
> > distributed with SXEmacs (where Calc's former maintainer moved), so it's
> > probably worth looking at that code (and associated doc).
> 
> I didnʼt know about that. Would there be any issue with merging such
> changes back to emacs? I assume itʼs all GPL, but there might be
> copyright assignment issues.

I presume the changes are beyond the dozen or so lines which we are
allowed to accept without copyright assignment, in which case this
will require legal paperwork as a prerequisite.

> so using native bignums appears to be ~10x faster that calc's bignums,
> and create less garbage.

Sounds great, thanks for working on this.

Re: Making 'eq' == 'eql' in bignum branch

[Robert Pluim]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Robert Pluim <rpluim@gmail.com>
>> Date: Thu, 23 Aug 2018 10:13:38 +0200
>> Cc: emacs-devel@gnu.org
>> 
>> > IIUC, such changes have already been applied to the Calc that's
>> > distributed with SXEmacs (where Calc's former maintainer moved), so it's
>> > probably worth looking at that code (and associated doc).
>> 
>> I didnʼt know about that. Would there be any issue with merging such
>> changes back to emacs? I assume itʼs all GPL, but there might be
>> copyright assignment issues.
>
> I presume the changes are beyond the dozen or so lines which we are
> allowed to accept without copyright assignment, in which case this
> will require legal paperwork as a prerequisite.

My current patch is ~1200 lines, and incomplete, so yes. Although
given that I can't locate those calc changes on the SXEmacs website,
the point is kind of moot anyway.

Robert

Re: Making 'eq' == 'eql' in bignum branch

[Stefan Monnier]

> I didnʼt know about that. Would there be any issue with merging such
> changes back to emacs? I assume itʼs all GPL, but there might be
> copyright assignment issues.

He signed the paperwork (back when he was maintaining Emacs's Calc), so
there shouldn't be any issue in this respect.
It'd be good to tell him about this change in Emacs, tho.  He may be
convinced to come back to Emacs now ;-)

>> Fast and exhaustive tests, of course ;-)
> It passes the test-suite, which has 5 tests :-)

All the tests?  That sounds like "exhaustive" to me!


        Stefan

Re: Making 'eq' == 'eql' in bignum branch

[Robert Pluim]

Robert Pluim <rpluim@gmail.com> writes:

> Stefan Monnier <monnier@iro.umontreal.ca> writes:
>
>>> I didnʼt know about that. Would there be any issue with merging such
>>> changes back to emacs? I assume itʼs all GPL, but there might be
>>> copyright assignment issues.
>>
>> He signed the paperwork (back when he was maintaining Emacs's Calc), so
>> there shouldn't be any issue in this respect.
>> It'd be good to tell him about this change in Emacs, tho.  He may be
>> convinced to come back to Emacs now ;-)
>>

Well, the calc in the SXEmacs git repo still uses its own bignum
implementation, and their email archives are offline, so Iʼm no
wiser. Could you point me in the direction of the author (privately if
you think thatʼs necessary)?

>>>> Fast and exhaustive tests, of course ;-)
>>> It passes the test-suite, which has 5 tests :-)
>>
>> All the tests?  That sounds like "exhaustive" to me!

None of them directly test bignums though. Iʼll have to write some
more.

Robert

Re: Making 'eq' == 'eql' in bignum branch

[Stefan Monnier]

> Well, the calc in the SXEmacs git repo still uses its own bignum
> implementation, and their email archives are offline, so Iʼm no
> wiser. Could you point me in the direction of the author (privately if
> you think thatʼs necessary)?

Hmm... I remember seeing the discussion on the sxemacs-devel
mailing-list (via Gmane).  His name is Jay Belanger.

IIRC he didn't add just support for GMP numbers but also for GNU mpfr
(which we don't have).


        Stefan

Re: Making 'eq' == 'eql' in bignum branch

[Paul Eggert]

Eli Zaretskii wrote:
> I don't think we should go as far as removing these functions, as they
> can still be useful in some situations.

'still'? They were just introduced a week ago. No user code is using them. Why 
push them onto users? It's a step in the wrong direction.

Re: Making 'eq' == 'eql' in bignum branch

[Eli Zaretskii]

> Cc: pipcet@gmail.com, monnier@iro.umontreal.ca, emacs-devel@gnu.org
> From: Paul Eggert <eggert@cs.ucla.edu>
> Date: Mon, 20 Aug 2018 09:01:23 -0700
> 
> It's a step in the wrong direction.

I don't think we know that yet.  We should collect hands-on experience
first.

Re: Making 'eq' == 'eql' in bignum branch

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > Also, we should remove the fixnump and bignump functions; these
  > functions certainly don't need to be in C code and I don't see why
  > they even need to be present at all. We shouldn't be giving users
  > the impression that the distinction between fixnums and bignums is
  > important.

Since it does affect behavior in some cases, we should have those
functions.  We can also ask people to write to emacs-devel to report
any time that they find it necessary for code to call them.  We might
find out about reasons that didn't occur to us, and then maybe then
we could make them unnecessary.

-- 
Dr Richard Stallman
President, Free Software Foundation (https://gnu.org, https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: Making 'eq' == 'eql' in bignum branch

[Eli Zaretskii]

> From: Pip Cet <pipcet@gmail.com>
> Date: Mon, 20 Aug 2018 07:19:39 +0000
> Cc: Stefan Monnier <monnier@iro.umontreal.ca>, emacs-devel@gnu.org
> 
> I think references to most-positive-fixnum (and implicit ones, like
> using Frandom without an argument) should be eliminated. Fixnum range
> is now an implementation detail.

That is too radical, IMO.  Having bignums _allows_ us to write
programs that are oblivious to most-positive-fixnum's value, but it
doesn't _require_ it.  It is perfectly legitimate to want to write a
program that is sensitive to that value, e.g., because using bignums
makes some program significantly slow, and so you'd like to ask the
users whether they want to proceed regardless.  Or maybe even refuse
to do something (e.g., buffers and strings generally cannot be larger
than that).

We have other system dependent variables and constants (e.g.,
shell-file-name), and for a good reason.  Sometimes you need those so
you could actually implement a portable program.

So I don't think we want to argue for a total elimination of
most-positive-fixnum from our code.

Re: Making 'eq' == 'eql' in bignum branch

[Lars Ingebrigtsen]

Eli Zaretskii <eliz@gnu.org> writes:

> So I don't think we want to argue for a total elimination of
> most-positive-fixnum from our code.

I agree, but its usage in Emacsen with larger numbers present can be
problematic.

I'm pretty sure that I've written code that's essentially along the
lines of

(when list-of-numbers
  (let ((i most-positive-fixnum))
    (dolist (a list-of-numbers)
      (setq i (min i a)))
    i))

to get the smallest number in a set.  (Not exactly like that, of course,
but using `most-positive-fixnum' as a guard value, knowing that none of
numbers in list-of-numbers can be larger than that guard value.)

So perhaps a warning might be warranted?

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: Making 'eq' == 'eql' in bignum branch

[Eli Zaretskii]

> From: Lars Ingebrigtsen <larsi@gnus.org>
> Cc: Pip Cet <pipcet@gmail.com>,  eggert@cs.ucla.edu,  monnier@iro.umontreal.ca,  emacs-devel@gnu.org
> Date: Mon, 20 Aug 2018 17:26:18 +0200
> 
> So perhaps a warning might be warranted?

Too annoying, IMO.  Instead, we should keep this in mind while
reviewing patches.

Re: Making 'eq' == 'eql' in bignum branch

[Stefan Monnier]

>> So perhaps a warning might be warranted?
> Too annoying, IMO.

Agreed.  I think for now a "grep" is more appropriate,


        Stefan

Re: Making 'eq' == 'eql' in bignum branch

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > I'm pretty sure that I've written code that's essentially along the
  > lines of

  > (when list-of-numbers
  >   (let ((i most-positive-fixnum))
  >     (dolist (a list-of-numbers)
  >       (setq i (min i a)))
  >     i))

  > to get the smallest number in a set.

This seems to be a real problem.

I wonder -- do most of these instances involve binding
a variable to most-positive-fixnum?  If so, that could be a
way to warn about cases that are likely to be wrong, while
skipping other cases.

How many problem cases is this likely to miss?

-- 
Dr Richard Stallman
President, Free Software Foundation (https://gnu.org, https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: Making 'eq' == 'eql' in bignum branch

[Paul Eggert]

>    > I'm pretty sure that I've written code that's essentially along the
>    > lines of
> 
>    > (when list-of-numbers
>    >   (let ((i most-positive-fixnum))
>    >     (dolist (a list-of-numbers)
>    >       (setq i (min i a)))
>    >     i))
> 
>    > to get the smallest number in a set.
> 
> This seems to be a real problem.

It is a real problem, but not a serious one.

A few days ago I audited Emacs master for all uses of most-positive-fixnum and 
most-negative-fixnum, and in commit a4a3c92e9de59bd0251f36326375cce898919edc I 
fixed five instances of problems like the one mentioned above. After doing this 
exercise, my impression is that although this problem is real it is not serious. 
The unfixed code was already broken on Emacs 26, as integers that exceeded 
most-positive-fixnum wrapped around silently, breaking the min calculations in a 
different way. With bignums in place the unfixed code was still broken, but 
bignums did not introduce a bug that wasn't there already.

Also, many of these calculations involve things like column counts or window 
positions, for which it's extremely unlikely that users will go beyond fixnum 
limits, and which can help to explain why the dubious code in question has 
remained unfixed in Emacs for so long. Although we should fix such limitations, 
misbehavior is so unlikely here that it's not high priority.

In case you're curious, here are the remaining dubious instances of 
most-negative-fixnum and most-positive-fixnum in the Emacs master source, and 
what they mean in terms of correctness. This should give you a deeper feel for 
thie issue.

* Several modules assumes that buffer sizes fits into fixnums. This assumption 
is correct for Emacs master, though it will become dubious if we change Emacs to 
support buffer sizes greater than most-positive-fixnum. The affected modules are 
emacs-lisp/syntax.el, org/org-list.el, progmodes/cc-engine.el, progmodes/js.el, 
and simple.el.

* net/tramp-sh.el contains some assumptions that file sizes and other attributes 
that don't fit into fixnums are converted to floats by the underlying drivers. 
This assumption won't be true of list-attributes, and presumably shouldn't be 
true of tramp's other drivers (though I don't know the details here; I'm 
handwaving a bit).

* gnus/gnus-registry.el and registry.el assume entry counts fit in fixnums.

* net/eww.el assumes readability scores are never less than most-negative-fixnum.

* org/org-element.el assumes that element keys are fixnums.

* subr.el assumes that undo counts fit in fixnum.

* xterm.el assumes that Emacs will never run for more than most-positive-fixnum 
seconds.

Some of these assumptions are quite safe, others are a bit more dubious, none 
seem to represent serious bugs.

Tramp and fixnum (was: Making 'eq' == 'eql' in bignum branch)

[Michael Albinus]

Paul Eggert <eggert@cs.ucla.edu> writes:

Hi Paul,

> * net/tramp-sh.el contains some assumptions that file sizes and other
> attributes that don't fit into fixnums are converted to floats by the
> underlying drivers. This assumption won't be true of list-attributes,
> and presumably shouldn't be true of tramp's other drivers (though I
> don't know the details here; I'm handwaving a bit).

Could you pls explain in more detail what you believe is wrong? A
pointer to a suspicious code snippet would help.

What do you mean with `list-attributes'? Maybe `file-attributes'?

Best regards, Michael.

Re: Tramp and fixnum (was: Making 'eq' == 'eql' in bignum branch)

[Paul Eggert]

Michael Albinus wrote:

>> * net/tramp-sh.el contains some assumptions that file sizes and other
>> attributes that don't fit into fixnums are converted to floats by the
>> underlying drivers. This assumption won't be true of list-attributes,
>> and presumably shouldn't be true of tramp's other drivers (though I
>> don't know the details here; I'm handwaving a bit).
> 
> Could you pls explain in more detail what you believe is wrong? A
> pointer to a suspicious code snippet would help.
> 
> What do you mean with `list-attributes'? Maybe `file-attributes'?

Yes, sorry, I meant 'file-attributes'. The fishy-looking code is in 
tramp-convert-file-attributes. Here's one snippet:

     (when (and (floatp (nth 2 attr))
                (<= (nth 2 attr) most-positive-fixnum))
       (setcar (nthcdr 2 attr) (round (nth 2 attr))))

Although this snippet shouldn't hurt, it should be unnecessary once 
file-attributes is fixed to not return floats. And in the meantime, the "(<= 
(nth 2 attr) most-positive-fixnum)" is unnecessary since file-attributes never 
returns a float less than or equal to most-positive-fixnum. There are three 
snippets like this.

The main offender is the inode converter in that function, which splits integers 
into three parts, whereas it should simply keep the integers as-is since they 
don't overflow any more.

Re: Tramp and fixnum (was: Making 'eq' == 'eql' in bignum branch)

[Eli Zaretskii]

> From: Paul Eggert <eggert@cs.ucla.edu>
> Date: Sun, 26 Aug 2018 08:34:25 -0700
> Cc: emacs-devel@gnu.org
> 
> > What do you mean with `list-attributes'? Maybe `file-attributes'?
> 
> Yes, sorry, I meant 'file-attributes'. The fishy-looking code is in 
> tramp-convert-file-attributes. Here's one snippet:
> 
>      (when (and (floatp (nth 2 attr))
>                 (<= (nth 2 attr) most-positive-fixnum))
>        (setcar (nthcdr 2 attr) (round (nth 2 attr))))
> 
> Although this snippet shouldn't hurt, it should be unnecessary once 
> file-attributes is fixed to not return floats.

Fixed in what version of Emacs?  Tramp wants to support more than just
the latest master version.

> The main offender is the inode converter in that function, which splits integers 
> into three parts, whereas it should simply keep the integers as-is since they 
> don't overflow any more.

They do in older Emacs versions, AFAIR.

Re: Tramp and fixnum

[Michael Albinus]

Eli Zaretskii <eliz@gnu.org> writes:

>> Yes, sorry, I meant 'file-attributes'. The fishy-looking code is in 
>> tramp-convert-file-attributes. Here's one snippet:
>> 
>>      (when (and (floatp (nth 2 attr))
>>                 (<= (nth 2 attr) most-positive-fixnum))
>>        (setcar (nthcdr 2 attr) (round (nth 2 attr))))
>> 
>> Although this snippet shouldn't hurt, it should be unnecessary once 
>> file-attributes is fixed to not return floats.
>
> Fixed in what version of Emacs?  Tramp wants to support more than just
> the latest master version.

Yes, back to Emacs 24.

>> The main offender is the inode converter in that function, which
>> splits integers
>> into three parts, whereas it should simply keep the integers as-is since they 
>> don't overflow any more.
>
> They do in older Emacs versions, AFAIR.

Yes. See Bug#25854, which triggered this code. It was reported towards
Emacs 26.0.50.

Best regards, Michael.

Re: Tramp and fixnum (was: Making 'eq' == 'eql' in bignum branch)

[Paul Eggert]

Eli Zaretskii wrote:
>> From: Paul Eggert <eggert@cs.ucla.edu>
>>       (when (and (floatp (nth 2 attr))
>>                  (<= (nth 2 attr) most-positive-fixnum))
>>         (setcar (nthcdr 2 attr) (round (nth 2 attr))))
>>
>> Although this snippet shouldn't hurt, it should be unnecessary once
>> file-attributes is fixed to not return floats.
> 
> Fixed in what version of Emacs?  Tramp wants to support more than just
> the latest master version.

Good point, I keep forgetting. In that case, that snippet won't hurt.

I just now installed patches into Emacs to fix file-attributes as mentioned 
above, along with a bunch of other functions dealing with system types like 
ino_t and pid_t.

Re: Tramp and fixnum

[Michael Albinus]

Paul Eggert <eggert@cs.ucla.edu> writes:

Hi Paul,

> And in the meantime, the "(<= (nth 2 attr) most-positive-fixnum)" is
> unnecessary since file-attributes never returns a float less than or
> equal to most-positive-fixnum.

It still does for older Emacsen. But maybe I'll write compatibility
functions like the other ones in tramp-compat.el. Using floats in
file-attributes shall be reduced to Emacs versions they still need.

Next days, likely.

Best regards, Michael.

Re: Tramp and fixnum

[Paul Eggert]

Michael Albinus wrote:
> It still does for older Emacsen. But maybe I'll write compatibility
> functions like the other ones in tramp-compat.el. Using floats in
> file-attributes shall be reduced to Emacs versions they still need.

Please give the latest master a try, since its file-attributes now generates 
bignums instead of floats when the file attribute integers don't fix into fixnums.

Re: Tramp and fixnum

[Michael Albinus]

Paul Eggert <eggert@cs.ucla.edu> writes:

Hi Paul,

>> It still does for older Emacsen. But maybe I'll write compatibility
>> functions like the other ones in tramp-compat.el. Using floats in
>> file-attributes shall be reduced to Emacs versions they still need.
>
> Please give the latest master a try, since its file-attributes now
> generates bignums instead of floats when the file attribute integers
> don't fix into fixnums.

Will do next days.

Best regards, Michael.

Re: Tramp and fixnum

[Michael Albinus]

Michael Albinus <michael.albinus@gmx.de> writes:

Hi Paul,

>>> It still does for older Emacsen. But maybe I'll write compatibility
>>> functions like the other ones in tramp-compat.el. Using floats in
>>> file-attributes shall be reduced to Emacs versions they still need.
>>
>> Please give the latest master a try, since its file-attributes now
>> generates bignums instead of floats when the file attribute integers
>> don't fix into fixnums.
>
> Will do next days.

This was more simple as expected. Tramp supports Emacsen 24+, and all of
them convert integer to floats automatically when needed. So I could get
rid of the "e0" hack.

And for the inode number conversion, I check whether function `bignump'
is available. If yes, I don't convert.

See commit adcf904b3a.

Best regards, Michael.

Re: Tramp and fixnum

[Paul Eggert]

Michael Albinus wrote:

> And for the inode number conversion, I check whether function `bignump'
> is available. If yes, I don't convert.

I must be slow this morning, but I don't see why that test needs to know whether 
bignums are supported. That is, for this change:

-    ;; Convert inode.
-    (unless (listp (nth 10 attr))
+    ;; Convert inode.  Big numbers have been added to Emacs 27.
+    (unless (or (fboundp 'bignump) (listp (nth 10 attr)))

How about changing to the following instead?

      (let ((inode (nth 10 attr)))
         (unless (or (integerp inode) (listp inode))
            ...

That is, if the inode number is already an integer, leave it alone. This should 
work regardless of whether bignums are supported.

Re: Tramp and fixnum

[Michael Albinus]

Paul Eggert <eggert@cs.ucla.edu> writes:

Hi Paul,

> +    ;; Convert inode.  Big numbers have been added to Emacs 27.
> +    (unless (or (fboundp 'bignump) (listp (nth 10 attr)))
>
> How about changing to the following instead?
>
>      (let ((inode (nth 10 attr)))
>         (unless (or (integerp inode) (listp inode))
>            ...
>
> That is, if the inode number is already an integer, leave it
> alone. This should work regardless of whether bignums are supported.

Indeed, that's better. Finally, it just needs

    (when (floatp (nth 10 attr))

Pushed to the repositories.

Best regards, Michael.

Re: Making 'eq' == 'eql' in bignum branch

[Tom Tromey]

>>>>> "Paul" == Paul Eggert <eggert@cs.ucla.edu> writes:

Paul> * Several modules assumes that buffer sizes fits into fixnums. This
Paul> assumption is correct for Emacs master, though it will become dubious
Paul> if we change Emacs to support buffer sizes greater than
Paul> most-positive-fixnum. The affected modules are emacs-lisp/syntax.el,
Paul> org/org-list.el, progmodes/cc-engine.el, progmodes/js.el, and
Paul> simple.el.

The js.el uses fall into 2 categories.

One, assuming the maximum paren depth is most-positive-fixnum:

(defconst js--initial-pitem
  (make-js--pitem
   :paren-depth most-negative-fixnum
   :type 'toplevel))

This seems reasonably safe.  You'd need a truly absurd .js file to break
this.


Two, as an argument to forward-comment: (forward-comment most-positive-fixnum).
This also seems safe, as the argument is a comment count, not a buffer position.
That said these could perhaps be changed to (forward-comment (point-max)).

Tom

Re: Making 'eq' == 'eql' in bignum branch

[Stefan Monnier]

> Two, as an argument to forward-comment: (forward-comment
> most-positive-fixnum).  This also seems safe, as the argument is
> a comment count, not a buffer position.  That said these could perhaps
> be changed to (forward-comment (point-max)).

Yes, point-max is better.  Same seems to apply to the cases in
syntax.el.


        Stefan

Re: Making 'eq' == 'eql' in bignum branch

[Alan Mackenzie]

Hello, Paul.

On Sat, Aug 25, 2018 at 16:27:20 -0700, Paul Eggert wrote:

[ .... ]

> * Several modules assumes that buffer sizes fits into fixnums. This
> assumption is correct for Emacs master, though it will become dubious
> if we change Emacs to support buffer sizes greater than
> most-positive-fixnum. The affected modules are emacs-lisp/syntax.el,
> org/org-list.el, progmodes/cc-engine.el, progmodes/js.el, and
> simple.el.

I'm trying to imagine a buffer whose size won't fit into a fixnum, on a
64-bit machine.  most-positive-fixnum is close to 2^61.  Imagine
scrolling through a 2^61 byte buffer.

Let's imagine it has an average line length of 64 bytes, in a window 64
lines deep, and auto-repeat is at 16 screens per second.  That's 2^(6+6+4)
characters per second scroll rate, i.e. 2^16.

So the number of seconds it would take would be 2^(61 - 16) = 2^45.

2^45 seconds = 2^45 / 86400 / 365.25 years, this being a little over 1.1
million years.  Holding the PageDown key down for this length of time
would get a little tedious.

Surely, at least for 64-bit systems, expecting buffer sizes ever to
exceed fixnums, is somewhat unrealistic.  Is anybody seriously going to
be editing a 540 Mbyte buffer on a 32-bit system nowadays?

Surely the assumption of buffer sizes being fixnums is realistic and
harmless.

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: Making 'eq' == 'eql' in bignum branch

[Stefan Monnier]

> I'm trying to imagine a buffer whose size won't fit into a fixnum, on a
> 64-bit machine.

Unlikely (at least in the foreseeable future), indeed.
But for 32bit builds, this was deemed important enough to
introduce --with-wide-int.


        Stefan

Re: Making 'eq' == 'eql' in bignum branch

[Eli Zaretskii]

> Date: Sun, 26 Aug 2018 20:19:47 +0000
> Cc: rms@gnu.org, Lars Ingebrigtsen <larsi@gnus.org>, eliz@gnu.org,
>   monnier@iro.umontreal.ca, pipcet@gmail.com, emacs-devel@gnu.org
> From: Alan Mackenzie <acm@muc.de>
> 
> Is anybody seriously going to be editing a 540 Mbyte buffer on a
> 32-bit system nowadays?

I do.  But such buffers are not larger than most-positive-fixnum,
because I use a build --with-wide-int, where an Emacs integer is a
64-bit type.

The actual problem is to exceed SIZE_T_MAX, i.e. have a buffer that
cannot be referenced by a single 'char *' pointer.

Re: Making 'eq' == 'eql' in bignum branch

[Stefan Monnier]

> I do.  But such buffers are not larger than most-positive-fixnum,
> because I use a build --with-wide-int, where an Emacs integer is a
> 64-bit type.

BTW, anyone interested in adding support for buffers larger than
most-positive-fixnum (using bignums) so that it can also be
done --without-wide-int?


        Stefan

Re: Making 'eq' == 'eql' in bignum branch

[Paul Eggert]

Stefan Monnier wrote:
> BTW, anyone interested in adding support for buffers larger than
> most-positive-fixnum (using bignums) so that it can also be
> done --without-wide-int?

It's on my list of things to do, yes. I have an abnormally fond liking for 
obsolescent hardware.

Re: Making 'eq' == 'eql' in bignum branch

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: Alan Mackenzie <acm@muc.de>,  eggert@cs.ucla.edu,  rms@gnu.org,  larsi@gnus.org,  pipcet@gmail.com,  emacs-devel@gnu.org
> Date: Mon, 27 Aug 2018 00:45:33 -0400
> 
> > I do.  But such buffers are not larger than most-positive-fixnum,
> > because I use a build --with-wide-int, where an Emacs integer is a
> > 64-bit type.
> 
> BTW, anyone interested in adding support for buffers larger than
> most-positive-fixnum (using bignums) so that it can also be
> done --without-wide-int?

Wouldn't that be much slower than --with-wide-int?  Instead of a
couple of machine instructions we will have a function call.

Re: Making 'eq' == 'eql' in bignum branch

[Stefan Monnier]

>> BTW, anyone interested in adding support for buffers larger than
>> most-positive-fixnum (using bignums) so that it can also be
>> done --without-wide-int?
> Wouldn't that be much slower than --with-wide-int?

I don't want to pay for --with-wide-int in my builds because I almost
never look at such large files.  But I'd be happy to be able to look at
such large files every once in a while.

Whether it'd be significantly slower depends a lot on whether we keep
most buffer positions as C integers or as Lisp integers and how often we
convert from one to the other.  I don't have a clear enough idea about
that to venture a guess.


        Stefan

Re: Making 'eq' == 'eql' in bignum branch

[Eli Zaretskii]

> From: Stefan Monnier <monnier@IRO.UMontreal.CA>
> Cc: acm@muc.de, eggert@cs.ucla.edu, rms@gnu.org, larsi@gnus.org,
>         pipcet@gmail.com, emacs-devel@gnu.org
> Date: Mon, 27 Aug 2018 11:18:21 -0400
> 
> >> BTW, anyone interested in adding support for buffers larger than
> >> most-positive-fixnum (using bignums) so that it can also be
> >> done --without-wide-int?
> > Wouldn't that be much slower than --with-wide-int?
> 
> I don't want to pay for --with-wide-int in my builds because I almost
> never look at such large files.

How can you not pay for that?  If you are saying that operating on
buffers and strings will only use GMP when we exceed the C limits, it
would mean an addition of a lot of tests where we currently just do
simple arithmetics on buffer/string positions.  Or am I missing
something?

Re: Making 'eq' == 'eql' in bignum branch

[Stefan Monnier]

> How can you not pay for that?

There might be a cost to supporting bignum buffer positions (even when
only fixnum positions are being used), indeed.

Like the case where bignums are used, I don't have a clear enough idea
of whether it'll end up too costly or not.


        Stefan

Re: Making 'eq' == 'eql' in bignum branch

[Paul Eggert]

Stefan Monnier wrote:
> Like the case where bignums are used, I don't have a clear enough idea
> of whether it'll end up too costly or not.

Yes, the only way to find out whether the cost will be acceptable is to try it.

My intuition is that it'll be OK, in the sense that large buffers are already so 
slow for other other reasons that the cost of testing for bignums will be not 
noticed by users.

Re: Making 'eq' == 'eql' in bignum branch

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

I'm glad this wasn't hard.

Should we add something in the Emacs Lisp manual to give advice
about cases like this?

How about if we make min and max accept, and ignore, nil as argument?

-- 
Dr Richard Stallman
President, Free Software Foundation (https://gnu.org, https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: Making 'eq' == 'eql' in bignum branch

[Clément Pit-Claudel]

On 2018-08-26 16:45, Richard Stallman wrote:
> How about if we make min and max accept, and ignore, nil as argument?

I usually use 1.0e+INF and -1.0e+INF when I need a neutral element for min and max respectively, rather than most-positive-fixnum.  Is there anything wrong with that?  If not, I think we don't even need to change min and max at all.

Clément.

Re: Making 'eq' == 'eql' in bignum branch

[Paul Eggert]

Clément Pit-Claudel wrote:
> I usually use 1.0e+INF and -1.0e+INF when I need a neutral element for min and max respectively, rather than most-positive-fixnum.  Is there anything wrong with that?

That should work, yes.

Re: Making 'eq' == 'eql' in bignum branch

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > I usually use 1.0e+INF and -1.0e+INF when I need a neutral element
  > for min and max respectively, rather than most-positive-fixnum.
  > Is there anything wrong with that?

Only that it is a magic sequence that most programmers won't remember.
We could put it in the doc strings and manual.

But perhaps nil is better.

					If not, I think we don't even
  > need to change min and max at all.

Why NOT make nil work here?  It would be useful in a number of ways,
I think.  For instance, you could run mapcar with a function
that would produce a number or nil, and nil would mean "ignore this one",
then apply max or min.


-- 
Dr Richard Stallman
President, Free Software Foundation (https://gnu.org, https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: Making 'eq' == 'eql' in bignum branch

[Paul Eggert]

Richard Stallman wrote:

> For instance, you could run mapcar with a function
> that would produce a number or nil, and nil would mean "ignore this one",
> then apply max or min.

Infinities serve the same function, so I'm not seeing the benefit of adding nil 
as another special case.

Re: Making 'eq' == 'eql' in bignum branch

[Tom Tromey]

>>>>> "RMS" == Richard Stallman <rms@gnu.org> writes:

RMS> [[[ To any NSA and FBI agents reading my email: please consider    ]]]
RMS> [[[ whether defending the US Constitution against all enemies,     ]]]
RMS> [[[ foreign or domestic, requires you to follow Snowden's example. ]]]

>> I usually use 1.0e+INF and -1.0e+INF when I need a neutral element
>> for min and max respectively, rather than most-positive-fixnum.
>> Is there anything wrong with that?

RMS> Only that it is a magic sequence that most programmers won't remember.
RMS> We could put it in the doc strings and manual.

How about adding (defconst Infinity 1.0e+INF) to subr.el?
That would be easier to remember.

Tom

Re: Making 'eq' == 'eql' in bignum branch

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > How about adding (defconst Infinity 1.0e+INF) to subr.el?

(max) is easier to remember.  Also, the capital initial makes the name
strange.

However, I think that making max and min allow nil
would be a good change for other reasons.

-- 
Dr Richard Stallman
President, Free Software Foundation (https://gnu.org, https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: Making 'eq' == 'eql' in bignum branch

[Clément Pit-Claudel]

On 2018-08-28 17:30, Richard Stallman wrote:
> (max) is easier to remember.

Really, even though (max) will be *minus* infinity?

Re: Making 'eq' == 'eql' in bignum branch

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > > (max) is easier to remember.

  > Really, even though (max) will be *minus* infinity?

Maybe you are right.
-- 
Dr Richard Stallman
President, Free Software Foundation (https://gnu.org, https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: Making 'eq' == 'eql' in bignum branch

[Clément Pit-Claudel]

On 2018-08-27 18:50, Richard Stallman wrote:
> Only that it is a magic sequence that most programmers won't remember.
> We could put it in the doc strings and manual.

Python spells it float("+inf"), and I see it used fairly often there.
I'd expect anyone who can remember most-positive-fixnum to quickly memorize 1.0e+INF :) (when I forget, I just eval (exp 9999999999999) and get 1.0e+INF as a reminder.)

Re: Making 'eq' == 'eql' in bignum branch

[Stefan Monnier]

> I'd expect anyone who can remember most-positive-fixnum to quickly memorize
> 1.0e+INF

TAB completion is pretty useful to fill the holes of my memory w.r.t
most-positive-fixnum, whereas I'm afraid it will be of not help with
1.0e+INF.


        Stefan

Re: Making 'eq' == 'eql' in bignum branch

[Clément Pit-Claudel]

On 2018-08-28 23:53, Stefan Monnier wrote:
>> I'd expect anyone who can remember most-positive-fixnum to quickly memorize
>> 1.0e+INF
> 
> TAB completion is pretty useful to fill the holes of my memory w.r.t
> most-positive-fixnum, whereas I'm afraid it will be of not help with
> 1.0e+INF.

(defvar 1\.0e+INF 1.0e+INF) ? :)

Re: Making 'eq' == 'eql' in bignum branch

[Lars Ingebrigtsen]

Clément Pit-Claudel <cpitclaudel@gmail.com> writes:

> I usually use 1.0e+INF and -1.0e+INF when I need a neutral element for
> min and max respectively, rather than most-positive-fixnum. 

Hey, nice tip.  I hadn't even considered those (well, I didn't know they
had a read syntax -- is that new?).  And I've already started using them
where I would previously have used most-positive-fixnum, like in this
code I just found myself typing:

(cl-sort films '<
         :key (lambda (e)
                (or (getf e :year) 1.0e+INF)))

This sorts a list of films by year, and sorts the ones with unknown
(nil) years to the end.  This would obviously previously have been a
problem in the year 2305843009213693951 CE on legacy 64-bit machines,
but my code is now future proof.

But anyway, as Richard implies, it's a read syntax that's strangely
difficult to remember.  Would it make sense to have a variable like
`positive-infinity' (etc) or `math-positive-infinity'?  Or perhaps we'll
all start remembering it after seeing it sprinkled throughout the source
code after a while.

Hm...  Probably the latter.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: Making 'eq' == 'eql' in bignum branch

[Pip Cet]

On Tue, Aug 28, 2018 at 1:50 AM Lars Ingebrigtsen <larsi@gnus.org> wrote:
> But anyway, as Richard implies, it's a read syntax that's strangely
> difficult to remember.  Would it make sense to have a variable like
> `positive-infinity' (etc) or `math-positive-infinity'?  Or perhaps we'll
> all start remembering it after seeing it sprinkled throughout the source
> code after a while.

How about making (min) = 1.0e+INF and (max) = -1.0e+INF? That would be
easier to type, and it's mathematically correct. We could even
optimize it in the bytecode compiler.

Re: Making 'eq' == 'eql' in bignum branch

[Paul Eggert]

Pip Cet wrote:
> How about making (min) = 1.0e+INF and (max) = -1.0e+INF? That would be
> easier to type, and it's mathematically correct. We could even
> optimize it in the bytecode compiler.

Makes sense to me.

Re: Making 'eq' == 'eql' in bignum branch

[Ken Raeburn]

On Aug 27, 2018, at 22:25, Pip Cet <pipcet@gmail.com> wrote:
> 
> On Tue, Aug 28, 2018 at 1:50 AM Lars Ingebrigtsen <larsi@gnus.org> wrote:
>> But anyway, as Richard implies, it's a read syntax that's strangely
>> difficult to remember.  Would it make sense to have a variable like
>> `positive-infinity' (etc) or `math-positive-infinity'?  Or perhaps we'll
>> all start remembering it after seeing it sprinkled throughout the source
>> code after a while.
> 
> How about making (min) = 1.0e+INF and (max) = -1.0e+INF? That would be
> easier to type, and it's mathematically correct. We could even
> optimize it in the bytecode compiler.

That’s great if you’re applying max or min to a possibly-empty collection of values. But if you’ve got a loop where you’re tracking a minimum/maximum value along with doing other stuff (possibly including computing the values you’re looking for the minimum of), this may not help much.

If “1.0e+INF” is hard to remember, how about defining symbols “+inifinity” and “-inifinity” (and if you want to get fancy with the Unicode, +∞ and -∞) with the appropriate numeric values? I kind of like the nil idea too.

There’s no reason we couldn’t do some or all of the above. Only question is, should “(min nil)” return nil or infinity?

Ken

Re: Making 'eq' == 'eql' in bignum branch

[Helmut Eller]

On Tue, Aug 28 2018, Ken Raeburn wrote:

> If “1.0e+INF” is hard to remember, how about defining symbols
> “+inifinity” and “-inifinity” (and if you want to get fancy with the
> Unicode, +∞ and -∞) with the appropriate numeric values? I kind of
> like the nil idea too.

There is already most-positive-float.  To bad that those "constants"
need to be set with cl-float-limits.

Helmut

Re: Making 'eq' == 'eql' in bignum branch

[Stefan Monnier]

> That’s great if you’re applying max or min to a possibly-empty collection of
> values. But if you’ve got a loop where you’re tracking a minimum/maximum
> value along with doing other stuff (possibly including computing the values
> you’re looking for the minimum of), this may not help much.

It helps because you can use (min) or (max) as your starting value.

> If “1.0e+INF” is hard to remember, how about defining symbols “+inifinity”

Pip's suggestion is to use (max) for that.


        Stefan

Re: Making 'eq' == 'eql' in bignum branch

[Clément Pit-Claudel]

On 2018-08-28 08:15, Stefan Monnier wrote:
>> If “1.0e+INF” is hard to remember, how about defining symbols “+inifinity”
> Pip's suggestion is to use (max) for that.

I think you read Pip's suggestion backward: (max) would return -infinity, not +infinity.  (Which makes sense, since you want max to distribute over list concatenation)

FWIW, I'm not a fan of using either (min) or (max) to represent minus infinity; it looks a bit odd.  Many other languages have an infinity constant already, so that pattern is well established; making (max) an alias for either of the infinities is prone to introducing confusion.

Re: Making 'eq' == 'eql' in bignum branch

[Pip Cet]

On Tue, Aug 28, 2018 at 6:19 PM Clément Pit-Claudel
<cpitclaudel@gmail.com> wrote:
>
> On 2018-08-28 08:15, Stefan Monnier wrote:
> >> If “1.0e+INF” is hard to remember, how about defining symbols “+inifinity”
> > Pip's suggestion is to use (max) for that.
>
> I think you read Pip's suggestion backward: (max) would return -infinity, not +infinity.  (Which makes sense, since you want max to distribute over list concatenation)
>
> FWIW, I'm not a fan of using either (min) or (max) to represent minus infinity; it looks a bit odd.  Many other languages have an infinity constant already, so that pattern is well established; making (max) an alias for either of the infinities is prone to introducing confusion.

It's more natural in code than it sounds at first: you use (min) when
finding a minimum, (max) when finding a maximum. However, if there is
a good name for it, an infinity constant is perfectly fine. Such a
name should make clear that we're talking about a signed infinity, not
one that makes 1/x continuous by adding a point at infinity to the
real line.

Signed infinities are convenient symbols, but they're not numbers; I
recall the horrified reaction of a mathematician when I told her about
IEEE infinities (she thought I was making it up about NaNs).

I think it's all so confusing that it would be best to accept the
slight performance cost of having a function like

(defun minpair (alist)
  "Return an element of ALIST for which (cdr X) is minimal."
  ...)

and using it consistently, in addition to defining (min), (max),
defining a pair of infinity constants, making `min' and `max' ignore
nil arguments, and changing `eq' to work on infinities. And maybe
changing the read and print syntax for infinities.

Re: Making 'eq' == 'eql' in bignum branch

[Herring, Davis]

> Such a name should make clear that we're talking about a signed
> infinity, not one that makes 1/x continuous by adding a point at
> infinity to the real line.

Sure: the extended reals <https://en.wikipedia.org/wiki/Extended_real_number_line>, not the projective reals <https://en.wikipedia.org/wiki/Projectively_extended_real_line>.

> Signed infinities are convenient symbols, but they're not numbers; I
> recall the horrified reaction of a mathematician when I told her about
> IEEE infinities (she thought I was making it up about NaNs).

Of course +/-inf are numbers.  They're not real numbers, but neither are complex numbers, split-complex numbers, dual numbers, p-adic numbers, quaternions, octonions, sedenions, hyperreal numbers, or (please no) surreal numbers.  With all due respect to your mathematician friend, she has no exclusive claim over the definition (such as it is) of "number" (and I'm a bit surprised if she hadn't heard of at least one of the extended real lines I linked).

NaN, on the other hand, is of course a floating-point value that explicitly claims not to be a number.  I'm not sure whether to believe it; it seems very useful that the operations on floating-point "numbers" are closed, but that depends on NaN being one of them...

Davis

Re: Making 'eq' == 'eql' in bignum branch

[Helmut Eller]

On Wed, Aug 29 2018, Herring, Davis wrote:

> Of course +/-inf are numbers.  They're not real numbers, but neither
> are complex numbers, split-complex numbers, dual numbers, p-adic
> numbers, quaternions, octonions, sedenions, hyperreal numbers, or
> (please no) surreal numbers.  With all due respect to your
> mathematician friend, she has no exclusive claim over the definition
> (such as it is) of "number" (and I'm a bit surprised if she hadn't
> heard of at least one of the extended real lines I linked).

Also, it seems quite common to define max/min so that +/-inf are the
neutral elements <https://en.wikipedia.org/wiki/Identity_element>.

In Lisp, many associative operations return the neutral element if
called with zero arguments, e.g. (*) => 1.  So I think it's quite nice
that (max) returns -infinity.

In Scheme, the syntax +inf.0 and -inf.0 is used for infinities.  Emacs
Lisp could use the same as +inf.0 is just an ordinary symbol.  At least
Schemers could remember that.

One could also write (/ 1 0.0) for +infinity; the easiest to remember
IMO.

Helmut

Re: Making 'eq' == 'eql' in bignum branch

[Pip Cet]

On Wed, Aug 29, 2018 at 3:42 AM Herring, Davis <herring@lanl.gov> wrote:
> > Signed infinities are convenient symbols, but they're not numbers; I
> > recall the horrified reaction of a mathematician when I told her about
> > IEEE infinities (she thought I was making it up about NaNs).
>
> Of course +/-inf are numbers.

I'm sorry I made a tangential mathematical statement; while I think
it's correct that the usual definition of "number" excludes infinities
(but not numbers that happen to be infinite), you are perfectly
correct that it's okay to redefine the term as you see fit. To get
this back to Emacs matters, `numberp' is fine as it is.

> They're not real numbers, but neither are complex numbers, split-complex numbers, dual numbers, p-adic numbers, quaternions, octonions, sedenions, hyperreal numbers, or (please no) surreal numbers.  With all due respect to your mathematician friend, she has no exclusive claim over the definition (such as it is) of "number" (and I'm a bit surprised if she hadn't heard of at least one of the extended real lines I linked).

She has; she simply has a different definition of "number" (and no
doubt the three of us have three different definitions of the term).

> NaN, on the other hand, is of course a floating-point value that explicitly claims not to be a number.  I'm not sure whether to believe it; it seems very useful that the operations on floating-point "numbers" are closed, but that depends on NaN being one of them...

Four definitions!

For those not familiar with APL, JavaScript is a popular language that
has Math.min() = Infinity.

Re: Making 'eq' == 'eql' in bignum branch

[Alan Mackenzie]

Hello, Davis.

On Wed, Aug 29, 2018 at 03:42:26 +0000, Herring, Davis wrote:
> > Such a name should make clear that we're talking about a signed
> > infinity, not one that makes 1/x continuous by adding a point at
> > infinity to the real line.

> Sure: the extended reals
> <https://en.wikipedia.org/wiki/Extended_real_number_line>, not the
> projective reals
> <https://en.wikipedia.org/wiki/Projectively_extended_real_line>.

> > Signed infinities are convenient symbols, but they're not numbers; I
> > recall the horrified reaction of a mathematician when I told her
> > about IEEE infinities (she thought I was making it up about NaNs).

> Of course +/-inf are numbers.

I'm speaking not as a mathematician, but as somebody with a degree in
maths.

+/-infinity are not numbers.  They do not satisfy the axioms which
define numbers.  For example they do not satisfy

    (x + y) + z = x + (y + z)

.  They cannot be made to satisfy these axioms.

> They're not real numbers, but neither are complex numbers,
> split-complex numbers, dual numbers, p-adic numbers, quaternions,
> octonions, sedenions, hyperreal numbers, or (please no) surreal
> numbers.

That's a strawman.  The issue being discussed here is numbers, not
arbitrary algebraic structures.

> With all due respect to your mathematician friend, ....

That is offensive in the extreme.

> .... she has no exclusive claim over the definition (such as it is) of
> "number"

That's analagous to saying that climate scientists don't have the
exclusive say-so about climate change.  Ha ha, who needs experts?

Mathematicians are _experts_ on numbers, with a deep understanding of
their definitions, properties, and limitations.  I would have expected,
in this mailing list (as contrasted with lesser forums) to see respect
for expertise, not disparagement.

> (and I'm a bit surprised if she hadn't heard of at least one of the
> extended real lines I linked).

Again, offensive.  "Infinity", with or without a sign, might be classed
as an "extended number", but as noted above, is not a number.

> NaN, on the other hand, is of course a floating-point value that
> explicitly claims not to be a number.  I'm not sure whether to believe
> it; it seems very useful that the operations on floating-point
> "numbers" are closed, but that depends on NaN being one of them...

NaN is not a number, either, for much the same reasons.  If you actually
attempt to do anything numerical with NaN you will run into trouble.

There have been discussions about the Lisp functions max and min being
called without arguments.  Some people have advocated that non-numbers
be returned in such cases.  I request most earnestly that this is not
done.  Callers of these functions have a right to assume that returned
results are mathematically correct and that they can do arithmetic with
them.

Currently, max and min throw errors if called with no arguments.  This
is surely the right thing to do, and we should carry on doing it.

> Davis

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: Making 'eq' == 'eql' in bignum branch

[Clément Pit-Claudel]

Hey Alan,


On 2018-08-31 15:59, Alan Mackenzie wrote:
> I'm speaking not as a mathematician, but as somebody with a degree in
> maths.

I'm speaking as someone who didn't expect a conversation like this one to spark such a strong-worded discussion :)  The whole debate rings odd to me (I agree with https://math.stackexchange.com/questions/36289/is-infinity-a-number: It comes down to the definition of "number," as well as the definition of "infinity")

> +/-infinity are not numbers.  They do not satisfy the axioms which
> define numbers.  For example they do not satisfy
> 
>     (x + y) + z = x + (y + z)
> 
> .  They cannot be made to satisfy these axioms.

I see the point, but… floating point numbers are not associative either, and it doesn't seem to controversial to call them "numbers".

>> They're not real numbers, but neither are complex numbers,
>> split-complex numbers, dual numbers, p-adic numbers, quaternions,
>> octonions, sedenions, hyperreal numbers, or (please no) surreal
>> numbers.
> 
> That's a strawman.  The issue being discussed here is numbers, not
> arbitrary algebraic structures.

I read this as an example of the many meanings that the word "number" can take, depending on the context.  It sounded pretty convincing.

>> .... she has no exclusive claim over the definition (such as it is) of
>> "number"
> 
> That's analagous to saying that climate scientists don't have the
> exclusive say-so about climate change.  Ha ha, who needs experts?

This is a very strongly worded charge.  The original formulation could have been better, but to me it didn't look like a rejection of expertise… rather, I took it as a statement that the word "number" gets assigned different meanings across fields.

>> (and I'm a bit surprised if she hadn't heard of at least one of the
>> extended real lines I linked).
> 
> Again, offensive.  "Infinity", with or without a sign, might be classed
> as an "extended number", but as noted above, is not a number.

I don't understand this part.  Does the word "number" on its own even have a strict definition, beyond something vague like Wikipedia's "mathematical object used to count, measure and also label"?

Cheers,
Clément.

Re: Making 'eq' == 'eql' in bignum branch

[Alan Mackenzie]

Hello, Clément.

On Fri, Aug 31, 2018 at 16:30:46 -0400, Clément Pit-Claudel wrote:
> Hey Alan,


> On 2018-08-31 15:59, Alan Mackenzie wrote:
> > I'm speaking not as a mathematician, but as somebody with a degree in
> > maths.

> I'm speaking as someone who didn't expect a conversation like this one
> to spark such a strong-worded discussion :)  The whole debate rings
> odd to me (I agree with
> https://math.stackexchange.com/questions/36289/is-infinity-a-number:
> It comes down to the definition of "number," as well as the definition
> of "infinity")

"Number" is defined in maths (of course, it's not simple, with many
different types of number), "infinity" is not, except in special
restricted contexts.

> > +/-infinity are not numbers.  They do not satisfy the axioms which
> > define numbers.  For example they do not satisfy
> > 
> >     (x + y) + z = x + (y + z)
> > 
> > .  They cannot be made to satisfy these axioms.

> I see the point, but… floating point numbers are not associative
> either, and it doesn't seem to controversial to call them "numbers".

Addition on real numbers is associative.  It is also, almost, on
floating points, but not quite, because floating point is only a
practical approximation to real numbers.  In practice, it means you've
got to be careful with floating point calculations, and what order you
do them in.

> >> They're not real numbers, but neither are complex numbers,
> >> split-complex numbers, dual numbers, p-adic numbers, quaternions,
> >> octonions, sedenions, hyperreal numbers, or (please no) surreal
> >> numbers.

> > That's a strawman.  The issue being discussed here is numbers, not
> > arbitrary algebraic structures.

> I read this as an example of the many meanings that the word "number"
> can take, depending on the context.  It sounded pretty convincing.

To me, it sounded like a diversion from the concept of numbers we have
in Emacs.

> >> .... she has no exclusive claim over the definition (such as it is)
> >> of "number"

> > That's analagous to saying that climate scientists don't have the
> > exclusive say-so about climate change.  Ha ha, who needs experts?

> This is a very strongly worded charge.  The original formulation could
> have been better, but to me it didn't look like a rejection of
> expertise… rather, I took it as a statement that the word "number"
> gets assigned different meanings across fields.

Yes, it may.  But which meaning should we stick to in Emacs?  I say the
mathematicians' meaning.  It's the most likely to be consistent and the
least likely to cause untoward side-effects.

> >> (and I'm a bit surprised if she hadn't heard of at least one of the
> >> extended real lines I linked).

> > Again, offensive.  "Infinity", with or without a sign, might be
> > classed as an "extended number", but as noted above, is not a
> > number.

> I don't understand this part.  Does the word "number" on its own even
> have a strict definition, beyond something vague like Wikipedia's
> "mathematical object used to count, measure and also label"?

There are many different types (grades, if you like) of number.  In
Emacs, we have computer approximations of Integers and real numbers.
Both of these concepts are rigorously defined in maths, in several
different ways (which can be shown to be equivalent).  Other types of
number include rationals (aka fractions), algebraic numbers (solutions
of polynomial equations with integer coefficients), and complex numbers
(which cannot appear as arguments to min or max, since the relation <
cannot be defined on complex numbers without creating contradictions).

> Cheers,
> Clément.

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: Making 'eq' == 'eql' in bignum branch

[Herring, Davis]

> The original formulation could have been better, but to me
> it didn't look like a rejection of expertise… rather, I took it
> as a statement that the word "number" gets assigned
> different meanings across fields.

You're definitely right that it could have been better: I now realize that I made a poor assumption.  Re-quoting:

> Signed infinities are convenient symbols, but they're not numbers; I
> recall the horrified reaction of a mathematician when I told her about
> IEEE infinities

From context, I took this to mean that the mathematician in question objected to IEEE infinities _because +/-inf "aren't numbers"_, when her opinion could easily have been something rather more nuanced like "the utility of floating-point arithmetic is reduced by the inclusion of values that are not real numbers".  There are legitimate issues with IEEE arithmetic, like the fact that infinities are produced despite the general rule that results are supposed to be rounded to the nearest representable value (which can't ever be infinity).

My apologies for presuming to know the reasoning of someone not even party to the conversation!

Davis

Re: Making 'eq' == 'eql' in bignum branch

[Stefan Monnier]

I don't have any opinion on whether infinities should be called numbers
or not, but regarding:

> There have been discussions about the Lisp functions max and min being
> called without arguments.  Some people have advocated that non-numbers
> be returned in such cases.  I request most earnestly that this is not
> done.  Callers of these functions have a right to assume that returned
> results are mathematically correct and that they can do arithmetic with
> them.

Currently many arithmetic functions, when called with "non-numbers" do
not signal an error, but instead return another such "non-number".
E.g. (+ 1.0e+INF 1) returns 1.0e+INF.
This is considered normal according to the IEEE practice.

So, from that point of view, I think it's fairly normal for min/max to
follow the same practice and return some kind of non-number when called
with another non-number, and by extension when not called with a number.

This non-number result (e.g. 1.0e+INF for `min`) can very be considered
as "returning an error" (just not using Elisp's error handling system,
but using IEEE's NaNs instead).


        Stefan


PS: I prefer to stay away from anything past the Z set of numbers, and
I think anything involving floats will inevitably suck in corner cases,
so any behavior you choose in that case is OK AFAIC ;-)

Re: Making 'eq' == 'eql' in bignum branch

[Herring, Davis]

> +/-infinity are not numbers.  They do not satisfy the axioms which
> define numbers.  For example they do not satisfy
> 
>     (x + y) + z = x + (y + z)

What is the list of such axioms?  If it includes the existence of an additive identity, you lose the natural numbers; if it includes the trichotomy property, you lose the complex numbers; if it includes commutativity of addition, you lose the ordinal numbers.  Of course we can decide that any or all of those "aren't really numbers", but why?

While addition is one of the operations with the broadest applicability (but see the supernatural numbers) and associativity is one of the properties most widely required of a binary operation (but see the octonions), even associativity of addition is sometimes violated (consider the example of infinite series <https://en.wikipedia.org/wiki/Associative_property#Non-associative_operation>).

>> They're not real numbers, but neither are complex numbers,
>> split-complex numbers, dual numbers, p-adic numbers, quaternions,
>> octonions, sedenions, hyperreal numbers, or (please no) surreal
>> numbers.
> 
> That's a strawman.  The issue being discussed here is numbers, not
> arbitrary algebraic structures.

Which of those sets aren't sets of numbers?  No one said anything about vector spaces, Lie groups, Sobolev spaces, tangent bundles, etc.

>> With all due respect to your mathematician friend, ....
> 
> That is offensive in the extreme.

I'm sorry, but I don't understand how.  I _meant_ the "due respect" that I wrote.  Perhaps I should not presume that any two people are friends, but the broad use of that term was meant to be generous, not condescending.

>> .... she has no exclusive claim over the definition (such as it is) of
>> "number"
> 
> That's analagous to saying that climate scientists don't have the
> exclusive say-so about climate change.  Ha ha, who needs experts?

I don't follow the analogy.  Outside of trivial misconceptions (like "weather" vs. "climate"), I don't know of a situation where the _definition_ of "climate change" is unclear.  On the other hand, I don't know of any rigorous definition for "number" (thus my request for axioms above).

> I would have expected,
> in this mailing list (as contrasted with lesser forums) to see respect
> for expertise, not disparagement.

No one is disparaging expertise, certainly not in general.  Experts routinely disagree, without any disrespect, about the more philosophical topics in their field.  Given (what I consider to be) such a weakness of definition, I remain uncertain why a mathematical expert would so flatly reject a widely-used category of "number".

>> (and I'm a bit surprised if she hadn't heard of at least one of the
>> extended real lines I linked).
> 
> Again, offensive.

Again, I'm afraid you'll have to explain.  I stated that I considered it unlikely that a mathematician would be unfamiliar with the extended reals (considering that they are used as a notational device even in Calculus I).  I said so because I don't know what other explanation to offer for such surprise at the IEEE rules, based so closely as they are on the extended reals.

Davis

Re: Making 'eq' == 'eql' in bignum branch

[Alan Mackenzie]

Hello, Davis.

On Sat, Sep 01, 2018 at 01:05:33 +0000, Herring, Davis wrote:
> > +/-infinity are not numbers.  They do not satisfy the axioms which
> > define numbers.  For example they do not satisfy

> >     (x + y) + z = x + (y + z)

> What is the list of such axioms?

Obviously different for different sorts of number.  For the real numbers,
briefly, numbers form a field (loosely speaking, a structure in which one
can do +, -, *, / (except by 0, of course)), they are ordered (loosely
speaking, for any pair of numbers, they are either equal or one is less
than the other, together with axioms for manipulating <), and the
completeness axiom aplies (if a non-empty set of reals is bounded above,
there exists a real number which is a least upper bound).

> If it includes the existence of an additive identity, you lose the
> natural numbers; if it includes the trichotomy property, you lose the
> complex numbers; if it includes commutativity of addition, you lose the
> ordinal numbers.  Of course we can decide that any or all of those
> "aren't really numbers", but why?

Because this is an emacs groups, and we have to use "number" in that
context.  We have integers (of various sorts) and floating point numbers.
I believe SXEmacs has more categories of "number" than Emacs, but I don't
know what (if anything) they get used for.

[ .... ]

> >> They're not real numbers, but neither are complex numbers,
> >> split-complex numbers, dual numbers, p-adic numbers, quaternions,
> >> octonions, sedenions, hyperreal numbers, or (please no) surreal
> >> numbers.

> > That's a strawman.  The issue being discussed here is numbers, not
> > arbitrary algebraic structures.

> Which of those sets aren't sets of numbers?  No one said anything
> about vector spaces, Lie groups, Sobolev spaces, tangent bundles, etc.

See above.

> >> With all due respect to your mathematician friend, ....

> > That is offensive in the extreme.

> I'm sorry, but I don't understand how.  I _meant_ the "due respect"
> that I wrote.  Perhaps I should not presume that any two people are
> friends, but the broad use of that term was meant to be generous, not
> condescending.

"With all due respect, ....., I am ignoring your expert judgment" is how
it came over to me.

> >> .... she has no exclusive claim over the definition (such as it is)
> >> of "number"

> > That's analagous to saying that climate scientists don't have the
> > exclusive say-so about climate change.  Ha ha, who needs experts?

> I don't follow the analogy.  Outside of trivial misconceptions (like
> "weather" vs. "climate"), I don't know of a situation where the
> _definition_ of "climate change" is unclear.

The analogy was about respect and contempt for expertise, not
definitions.

> On the other hand, I don't know of any rigorous definition for "number"
> (thus my request for axioms above).

Note that these two "infinities" are being proposed as members of
(Emacs's) real numbers, so it is reasonable to focus on real numbers.  I
hope my summary of the axioms above has helped.

> > I would have expected, in this mailing list (as contrasted with
> > lesser forums) to see respect for expertise, not disparagement.

> No one is disparaging expertise, certainly not in general.  Experts
> routinely disagree, without any disrespect, about the more
> philosophical topics in their field.  Given (what I consider to be)
> such a weakness of definition, I remain uncertain why a mathematical
> expert would so flatly reject a widely-used category of "number".

I've explained already.  Because extending the real numbers with
"infinity" means they no longer satisfy the field axioms, or the ordering
axioms; they're no longer real numbers.

[ .... ]

> Davis

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: Making 'eq' == 'eql' in bignum branch

[Herring, Davis]

> Because this is an emacs groups, and we have to use "number" in that
> context.

I think we've finally found the root of our miscommunication: I took the discussion to be about numbers in general, from which we might draw useful conclusions for Emacs, rather than to (already) be about Emacs' model for the real numbers.

>> What is the list of such axioms?
> 
> Obviously different for different sorts of number.

Certainly: that was the entire point of my most recent message -- and of my first in this thread.

> For the real numbers, [...]

...And conversely I was never asking for the axioms for real numbers, although it's good to remember those every once in a while (I usually think only of Cauchy completeness, but that allows interlopers like the Levi-Civita field).

> Note that these two "infinities" are being proposed as members of
> (Emacs's) real numbers, so it is reasonable to focus on real numbers.

They're not being proposed -- they're already there (if not in C, then at least on all machines in common use), since IEEE 754 is in fact based on the (affinely) extended reals.  It's begging the question to call Emacs' numbers real and then object to the inclusion of infinities based on that, but you're right that it's reasonable to _focus_ on the cofinite real subset when trying to engineer a usable system we can reason about.

Davis

Re: Making 'eq' == 'eql' in bignum branch

[Clément Pit-Claudel]

Hi again Alan,

Thanks for your thoughtful response!

On 2018-08-31 15:59, Alan Mackenzie wrote:
> There have been discussions about the Lisp functions max and min being
> called without arguments.  Some people have advocated that non-numbers
> be returned in such cases.  I request most earnestly that this is not
> done.  Callers of these functions have a right to assume that returned
> results are mathematically correct and that they can do arithmetic with
> them.
>
> Currently, max and min throw errors if called with no arguments.  This
> is surely the right thing to do, and we should carry on doing it.

Thinking more about your message, I'm not sure I agree with this point, either.  I don't know about the US or Germany, but at least in France it is common to define inf and sup as functions from ℝ into ℝ ∪ {–∞, +∞}, the extended real number line.  With these definitions, inf(∅) is customarily set to +∞, and sup(∅) is set to +∞.

https://en.wikipedia.org/wiki/Empty_set#Extended_real_numbers says:

    Since the empty set has no members, when it is considered as a subset of any ordered set, then every member of that set will be an upper bound and lower bound for the empty set. For example, when considered as a subset of the real numbers, with its usual ordering, represented by the real number line, every real number is both an upper and lower bound for the empty set. When considered as a subset of the extended reals formed by adding two "numbers" or "points" to the real numbers, namely negative infinity, denoted −∞, which is defined to be less than every other extended real number, and positive infinity, denoted +∞, which is defined to be greater than every other extended real number, then:

        sup ∅ = min({−∞, +∞} ∪ R) = −∞

    and

        inf ∅ = max({−∞, +∞} ∪ R) = +∞

    That is, the least upper bound (sup or supremum) of the empty set is negative infinity, while the greatest lower bound (inf or infimum) is positive infinity. By analogy with the above, in the domain of the extended reals, negative infinity is the identity element for the maximum and supremum operators, while positive infinity is the identity element for minimum and infimum.

I could be convinced that returning −∞ and +∞ is wrong based on the fact that inf and sup are not min and max, and that the outputs of min and max should be elements of the sets they operate on; conversely, I'm also receptive to the convenience aspect or min and max returning infinities; but all in all I don't find the "numberness" argument particularly compelling, based on the reasoning outlined above.

Clément.

Re: Making 'eq' == 'eql' in bignum branch

[Elias Mårtenson]

[-- Attachment #1: Type: text/plain, Size: 578 bytes --]

On Wed, 29 Aug 2018 at 02:19, Clément Pit-Claudel <cpitclaudel@gmail.com>
wrote:


> FWIW, I'm not a fan of using either (min) or (max) to represent minus
> infinity; it looks a bit odd.  Many other languages have an infinity
> constant already, so that pattern is well established; making (max) an
> alias for either of the infinities is prone to introducing confusion.
>

For anyone curious, the way to do it in APL is ⌈/⍬ and ⌊/⍬, which literally
means take the max and min values of an empty list, so it's not completely
unheard of.

Regards,
Elias

[-- Attachment #2: Type: text/html, Size: 881 bytes --]

Re: Making 'eq' == 'eql' in bignum branch

[Clément Pit-Claudel]

On 2018-08-28 23:22, Elias Mårtenson wrote:
> On Wed, 29 Aug 2018 at 02:19, Clément Pit-Claudel <cpitclaudel@gmail.com <mailto:cpitclaudel@gmail.com>> wrote:
>  
> 
>     FWIW, I'm not a fan of using either (min) or (max) to represent minus infinity; it looks a bit odd.  Many other languages have an infinity constant already, so that pattern is well established; making (max) an alias for either of the infinities is prone to introducing confusion.
> 
> 
> For anyone curious, the way to do it in APL is ⌈/⍬ and ⌊/⍬, which literally means take the max and min values of an empty list, so it's not completely unheard of.

Let me amend what I wrote above.  I'm not excited about (max) becoming the common spelling for negative infinity, but I'm fine with changing max to accept zero arguments and return -infinity in that case (and, indeed, that's the only value that makes sense to me).

I'd simply prefer to have a more explicit spelling for both infinities, and if 1.0e+INF if too hard to remember, a new read syntax or a new defconst would have my preference over (max) and (min).

Clément.

Re: Making 'eq' == 'eql' in bignum branch

[Stefan Monnier]

> Let me amend what I wrote above.  I'm not excited about (max) becoming the
> common spelling for negative infinity, but I'm fine with changing max to
> accept zero arguments and return -infinity in that case (and, indeed, that's
> the only value that makes sense to me).

Same here, except rather than "I'm fine with" I'd say that I'm in favor
of it (for the same reason that I made `apply` work when called with
a single argument).

> I'd simply prefer to have a more explicit spelling for both infinities, and
> if 1.0e+INF if too hard to remember, a new read syntax or a new defconst
> would have my preference over (max) and (min).

(aol-mode 1)


        Stefan

Re: Making 'eq' == 'eql' in bignum branch

[Stefan Monnier]

>> I'd simply prefer to have a more explicit spelling for both infinities, and
>> if 1.0e+INF if too hard to remember, a new read syntax or a new defconst
>> would have my preference over (max) and (min).

(defconst +∞ 1.0e+INF)
(defconst -∞ -1.0e+INF)


        Stefan

Re: Making 'eq' == 'eql' in bignum branch

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > (defconst +∞ 1.0e+INF)

To start using non-ASCII characters in the names of Emacs Lisp
constructs is a fundamental change in practices.  It is not
something to be done lightly.

-- 
Dr Richard Stallman
President, Free Software Foundation (https://gnu.org, https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: Making 'eq' == 'eql' in bignum branch

[Paul Eggert]

Stefan Monnier wrote:

> Same here, except rather than "I'm fine with" I'd say that I'm in favor
> of it (for the same reason that I made `apply` work when called with
> a single argument).

I looked into implementing something along those lines and ran into a problem: 
the C standard does not require support for floating-point infinity, and a few 
mildly-plausible Emacs targets lack it. The most significant is IBM z/OS when 
compiled in default mode. There is also the Unisys Clearpath (formerly 
Burroughs) and Dorado (formerly Univac) lines. These environments are still 
supported, and help explain why the C standard does not require support for a 
working INFINITY macro in <math.h>, and why Common Lisp does not require 
infinity constants.

That being said, I now see that the Emacs source code has been using <math.h>'s 
INFINITY macro since Emacs 25.1. Although this was just an oversight (and I just 
now fixed it on master), nobody has complained.

With that in mind we could simply be upfront about it and explicitly require 
IEEE floating point, excluding old mainframe environments and some embedded 
platforms. Then we could change 'min' and 'max' to return infinity, and we can 
also simplify the C code a bit by assuming IEEE_FLOATING_POINT. On the other 
hand perhaps it's better to leave sleeping dogs lie, and not jump completely 
into the IEEE bed, and wait for the next C standard to revamp the floating point 
API again (which it looks like it will) before worrying about this.

As for a named constant, I now see a problem with it: it wouldn't work in 
expressions like (let ((gamow '(1 2 3 infinity))) ...). So perhaps we should 
just ask people to remember to use 1e+INF instead, as it doesn't have this 
syntactic glitch.

Re: Making 'eq' == 'eql' in bignum branch

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > How about making (min) = 1.0e+INF and (max) = -1.0e+INF? That would be
  > easier to type, and it's mathematically correct.

I think that is a good idea.  However, I still think it would be
a convenience to make max and min ignore nil as argument.

-- 
Dr Richard Stallman
President, Free Software Foundation (https://gnu.org, https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: Making 'eq' == 'eql' in bignum branch

[Johan Bockgård]

Clément Pit-Claudel <cpitclaudel@gmail.com> writes:

> On 2018-08-26 16:45, Richard Stallman wrote:
>> How about if we make min and max accept, and ignore, nil as argument?
>
> I usually use 1.0e+INF and -1.0e+INF when I need a neutral element for
> min and max respectively, rather than most-positive-fixnum. Is there
> anything wrong with that? If not, I think we don't even need to change
> min and max at all.

It doesn't preserve integerness; (min 1 NEUTRAL-ELEMENT) should be 1,
but (min 1 1e+INF) is 1.0.

Re: Making 'eq' == 'eql' in bignum branch

[Clément Pit-Claudel]

On 2018-08-30 15:13, Johan Bockgård wrote:
> Clément Pit-Claudel <cpitclaudel@gmail.com> writes:
> 
>> On 2018-08-26 16:45, Richard Stallman wrote:
>>> How about if we make min and max accept, and ignore, nil as argument?
>>
>> I usually use 1.0e+INF and -1.0e+INF when I need a neutral element for
>> min and max respectively, rather than most-positive-fixnum. Is there
>> anything wrong with that? If not, I think we don't even need to change
>> min and max at all.
> 
> It doesn't preserve integerness; (min 1 NEUTRAL-ELEMENT) should be 1,
> but (min 1 1e+INF) is 1.0.

What platform are you on? On my machine (x86_64-pc-linux-gnu) it returns 1, not 1.0.

And, AFAICT, the implementation always returns one of its arguments:

static Lisp_Object
minmax_driver (ptrdiff_t nargs, Lisp_Object *args,
	       enum Arith_Comparison comparison)
{
  Lisp_Object accum = args[0];
  CHECK_NUMBER_COERCE_MARKER (accum);
  for (ptrdiff_t argnum = 1; argnum < nargs; argnum++)
    {
      Lisp_Object val = args[argnum];
      CHECK_NUMBER_COERCE_MARKER (val);
      if (!NILP (arithcompare (val, accum, comparison)))
	accum = val;
      else if (FLOATP (val) && isnan (XFLOAT_DATA (val)))
	return val;
    }
  return accum;
}

Clément.

Re: Making 'eq' == 'eql' in bignum branch

[Tom Tromey]

>>>>> "Clément" == Clément Pit-Claudel <cpitclaudel@gmail.com> writes:

>> It doesn't preserve integerness; (min 1 NEUTRAL-ELEMENT) should be 1,
>> but (min 1 1e+INF) is 1.0.

Clément> What platform are you on? On my machine (x86_64-pc-linux-gnu)
Clément> it returns 1, not 1.0.

Clément> And, AFAICT, the implementation always returns one of its arguments:

This changed in recent history, min/max used to do float promotion.

Tom

Re: Making 'eq' == 'eql' in bignum branch

[Clément Pit-Claudel]

On 2018-08-30 17:35, Tom Tromey wrote:
> Clément> What platform are you on? On my machine (x86_64-pc-linux-gnu)
> Clément> it returns 1, not 1.0.
> 
> This changed in recent history, min/max used to do float promotion.

Neat, that explains it :) Thanks!

Re: Making 'eq' == 'eql' in bignum branch

[Stefan Monnier]

>> I think references to most-positive-fixnum (and implicit ones, like
>> using Frandom without an argument) should be eliminated. Fixnum range
>> is now an implementation detail.
> That is too radical, IMO.

I think what Pip is saying is that we should review all *uses* of
most-positive-fixnum, because it's very likely that many/most of them
are now wrong or suboptimal.

That doesn't mean we should remove most-positive-fixnum itself.


        Stefan

Re: Making 'eq' == 'eql' in bignum branch

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Date: Mon, 20 Aug 2018 11:37:01 -0400
> 
> >> I think references to most-positive-fixnum (and implicit ones, like
> >> using Frandom without an argument) should be eliminated. Fixnum range
> >> is now an implementation detail.
> > That is too radical, IMO.
> 
> I think what Pip is saying is that we should review all *uses* of
> most-positive-fixnum, because it's very likely that many/most of them
> are now wrong or suboptimal.
> 
> That doesn't mean we should remove most-positive-fixnum itself.

I didn't think he meant to remove it.  But the goal of eliminating
references to it can only go so far, because there are legitimate use
cases for referencing it.  References that are no longer needed should
be removed, but this is a case by case decision, and sometimes a
judgment call, not an absolute requirement that Pip's wording hinted
at.

Re: Making 'eq' == 'eql' in bignum branch

[Paul Eggert]

Eli Zaretskii wrote:
> I don't think we want to argue for a total elimination of
> most-positive-fixnum from our code.

I agree. Software like 'calc' springs to mind: quite possibly it will want to do 
some arithmetic in fixnums, and will want to know the fixnum range to do that, 
and most-negative-fixnum and most-positive-fixnum are how it can do that.

As I recall, Gerd introduced most-positive-fixnum and most-negative-fixnum to 
Emacs Lisp in 2001, as it was a good idea taken from Common Lisp. However, 
fixnump and bignump are not in Common Lisp, and we don't need them in Emacs Lisp 
either. When we're reinventing the wheel why add unnecessary gizmos?

Re: Making 'eq' == 'eql' in bignum branch

[Helmut Eller]

On Mon, Aug 20 2018, Paul Eggert wrote:

> As I recall, Gerd introduced most-positive-fixnum and
> most-negative-fixnum to Emacs Lisp in 2001, as it was a good idea
> taken from Common Lisp. However, fixnump and bignump are not in Common
> Lisp, and we don't need them in Emacs Lisp either. When we're
> reinventing the wheel why add unnecessary gizmos?

Common Lisp doesn't have so many type predicates because it has TYPEP.
Both FIXNUM and BIGNUM are standard types in Common Lisp.  The BIGNUM
type is actually defined as (AND INTEGER (NOT FIXNUM)).  So if you
remove fixnump then at least make typep (and typecase etc.) work with
'fixnum and 'bignum type specifiers.

Helmut

Re: Making 'eq' == 'eql' in bignum branch

[Stefan Monnier]

> I think it's currently true, but only because make_number always
> creates a copy of its argument (at which point we might as well hash
> it), and it sounded like Paul was going to fix that soon.

Two options:
1- the operation does not create a new number.  Since Elisp's
   side only sees immutable bignum numbers if the number is not new we
   indeed shouldn't create a new number and can reuse the existing
   bignum without any need for hash-consing either: so hash-consing
   doesn't make this case any slower.
2- the operation does create a new number.  In that case the
   creation of the number almost inevitably will have to touch every
   "limb" of the number (it has a complexity of O(N) in the best case),
   so computing the hash shouldn't take noticeably more time.

> It sounds to me like a good implementation of this would require GMP
> support, keeping a hash of each mpz_t in the memory allocated for it.

We can keep the hash in the hash-table.

> Do we need to worry about algorithmic complexity attacks? Or about
> integers created accidentally that collide in the hash?

Emacs already offers so many other attack vectors that I don't think
it's worth worrying about this one at this point.

> I think 64-bit integers, at least, ought to be fast and small.

Indeed, it's for those small bignums that hash-consing will be most
expensive, but I'm not sure it'll be a significant problem.


        Stefan

Re: Using the GNU GMP Library for Bignums in Emacs

[Eli Zaretskii]

> From: Paul Eggert <eggert@cs.ucla.edu>
> Date: Wed, 18 Jul 2018 03:20:40 -0700
> Cc: Tom Tromey <tom@tromey.com>, emacs-devel@gnu.org
> 
> +/* Return true if X and Y are the same floating-point value.
> +   This looks at X's and Y's representation, since (unlike '==')
> +   it returns true if X and Y are the same NaN.  */
> +static bool
> +same_float (Lisp_Object x, Lisp_Object y)
> +{
> +  union double_and_words
> +    xu = { .val = XFLOAT_DATA (x) },
> +    yu = { .val = XFLOAT_DATA (y) };
> +  EMACS_UINT neql = 0;
> +  for (int i = 0; i < WORDS_PER_DOUBLE; i++)
> +    neql |= xu.word[i] ^ yu.word[i];
> +  return !neql;
> +}

Wouldn't calling this upon every float comparison yield a
non-negligible run-time penalty?  Maybe we should use 'isnan' instead?

Re: Using the GNU GMP Library for Bignums in Emacs

[Paul Eggert]

On 07/18/2018 08:55 AM, Eli Zaretskii wrote:
> Wouldn't calling this upon every float comparison yield a
> non-negligible run-time penalty?  Maybe we should use 'isnan' instead?

No, it's actually faster (at least on x86-64), since it uses a single 
64-bit integer comparison rather than up to three 64-bit floating-point 
comparisons. (The for-loop in question is unrolled into a single 
execution of its body, among other things.)

isnan would slow us down here, since the code does not need to differ 
for NaNs versus non-NaNs.

Re: Using the GNU GMP Library for Bignums in Emacs

[Ken Raeburn]

On Jul 18, 2018, at 06:20, Paul Eggert <eggert@cs.ucla.edu> wrote:
> 
> Stefan Monnier wrote:
>> I recently posted a patch which makes EQ behave like `eql` attached.
>> I just tried to see its impact on the ELisp compilation time
>> (i.e. I did `rm **/*.elc; make`).
> 
> I tried to reproduce this on my machine, and ran into some trouble (the code had warnings that caused compilation to fail). Although your patch is evidently intended only as a quick benchmark and not as an actual change, I worry that the benchmark isn't realistic enough, as some usage of EQ in C code will need to change to Feql or equivalent. Also, the C code will need to change how hashing works since XHASH etc. must be consistent with eq.
> 
> Looking into this a bit more, I discovered that eql currently operates incorrectly on NaNs, as it's inconsistent with how hash tables work. I installed the attached patch into master to fix that.

If we’re going to treat NaNs as having distinguishable bit patterns so we can tell whether two are the same or not, should we also have printable/readable forms that distinguish them? If not, how do we create and use distinguishable NaNs?

For that matter, are we guaranteed that operations like (/ 0.0 0.0) will always generate NaNs with the same bit pattern?

I would’ve guessed that it might be preferable to go the other direction, and use one canonical NaN value in Lisp, which would thus always be eq/eql to all other NaN expressions.

Ken

Re: Using the GNU GMP Library for Bignums in Emacs

[Paul Eggert]

Ken Raeburn wrote:
> If we’re going to treat NaNs as having distinguishable bit patterns so we can tell whether two are the same or not, should we also have printable/readable forms that distinguish them? If not, how do we create and use distinguishable NaNs?

Good point. It'd make sense to have printable/readable forms; please see below.

> are we guaranteed that operations like (/ 0.0 0.0) will always generate NaNs with the same bit pattern?

No, and this is an issue even with older Emacs. For example, (format "%s" (/ 0.0 
0.0)) returns different strings on different platforms, because some machines 
return a negative NaN whereas others don't.

> I would’ve guessed that it might be preferable to go the other direction, and use one canonical NaN value in Lisp, which would thus always be eq/eql to all other NaN expressions.

For some time Emacs has been able to distinguish negative from nonnegative NaNs. 
Functions like 'format' and 'copysign' depend on a NaN's sign, and (- x) 
reliably changes the sign of a NaN.

I see three ways to go here.

1. We change eql, memql, sxhash-eql, etc. to treat all NaNs alike, so that there 
is just one NaN value from the Lisp point of view. We also change 'format', 
'copysign' etc. to ignore a NaN's sign, and look through any other use of 
floating-point values to make sure that NaN signs are ignored.

2. We change eql, memql, sxhash-eql, etc. so that only the sign (not the 
significand) of a NaN is looked at, so that there are just two NaN values from 
the Lisp point of view.

3. We alter 'read', 'format' etc. to read and generate NaN significands. For 
example, (format "%s" NaN) could return "0.1e+NaN" if the significand's 
low-order bit was set.

(1) sounds too drastic, as the sign of a NaN can be useful in some cases and 
Emacs has long provided for obtaining the sign of a NaN. Although either (2) or 
(3) would be OK, I'm inclined to go for (3) as I expect it would be a bit 
cleaner and more useful.

Re: Using the GNU GMP Library for Bignums in Emacs

[Paul Eggert]

[-- Attachment #1: Type: text/plain, Size: 1233 bytes --]

On July 22 I wrote:

> I see three ways to go here.
> 
> 1. We change eql, memql, sxhash-eql, etc. to treat all NaNs alike, so that there 
> is just one NaN value from the Lisp point of view. We also change 'format', 
> 'copysign' etc. to ignore a NaN's sign, and look through any other use of 
> floating-point values to make sure that NaN signs are ignored.
> 
> 2. We change eql, memql, sxhash-eql, etc. so that only the sign (not the 
> significand) of a NaN is looked at, so that there are just two NaN values from 
> the Lisp point of view.
> 
> 3. We alter 'read', 'format' etc. to read and generate NaN significands. For 
> example, (format "%s" NaN) could return "0.1e+NaN" if the significand's 
> low-order bit was set.
> 
> (1) sounds too drastic, as the sign of a NaN can be useful in some cases and 
> Emacs has long provided for obtaining the sign of a NaN. Although either (2) or 
> (3) would be OK, I'm inclined to go for (3) as I expect it would be a bit 
> cleaner and more useful.


No further comment on this, and I went with (3) and installed into master the 
attached patch to do something along those lines on platforms that have 
ieee754.h (mostly GNUish platforms, I expect). Other platforms are unaffected.

[-- Attachment #2: 0001-Read-and-print-NaN-significand-if-ieee754.h.patch --]
[-- Type: text/x-patch, Size: 4330 bytes --]

From 6e37d2fd05bea373c472af1c6e80238ace5e1c94 Mon Sep 17 00:00:00 2001
From: Paul Eggert <eggert@cs.ucla.edu>
Date: Wed, 1 Aug 2018 00:49:39 -0700
Subject: [PATCH] Read and print NaN significand if <ieee754.h>

* configure.ac: Check for ieee754.h.
* doc/lispref/numbers.texi (Float Basics): Document
that NaN string representation digits are machine-dependent.
* etc/NEWS: Mention the change.
* src/lread.c, src/print.c [HAVE_IEEE754_H]: Include ieee754.h.
* src/lread.c (string_to_number) [HAVE_IEEE754_H]:
* src/print.c (float_to_string) [HAVE_IEEE754_H]:
Read and print NaN significand.
---
 configure.ac             |  1 +
 doc/lispref/numbers.texi |  2 +-
 etc/NEWS                 |  3 +++
 src/lread.c              | 11 +++++++++++
 src/print.c              | 11 +++++++++++
 5 files changed, 27 insertions(+), 1 deletion(-)

diff --git a/configure.ac b/configure.ac
index b691867..dbdcce7 100644
--- a/configure.ac
+++ b/configure.ac
@@ -1668,6 +1668,7 @@ AC_DEFUN
 
 dnl checks for header files
 AC_CHECK_HEADERS_ONCE(
+  ieee754.h
   linux/fs.h
   malloc.h
   sys/systeminfo.h
diff --git a/doc/lispref/numbers.texi b/doc/lispref/numbers.texi
index 14d5059..a3317c9 100644
--- a/doc/lispref/numbers.texi
+++ b/doc/lispref/numbers.texi
@@ -241,7 +241,7 @@ Float Basics
 NaNs carry a sign and a significand, and non-numeric functions treat
 two NaNs as equal when their
 signs and significands agree.  Significands of NaNs are
-machine-dependent and are not directly visible to Emacs Lisp.
+machine-dependent, as are the digits in their string representation.
 
   When NaNs and signed zeros are involved, non-numeric functions like
 @code{eql}, @code{equal}, @code{sxhash-eql}, @code{sxhash-equal} and
diff --git a/etc/NEWS b/etc/NEWS
index f1ea835..9e7a765 100644
--- a/etc/NEWS
+++ b/etc/NEWS
@@ -880,6 +880,9 @@ Formerly, some of these functions ignored signs and significands of
 NaNs.  Now, all these functions treat NaN signs and significands as
 significant.  For example, (eql 0.0e+NaN -0.0e+NaN) now returns nil
 because the two NaNs have different signs; formerly it returned t.
+Also, on platforms that have <ieee754.h> Emacs now reads and prints
+NaN significands; e.g., if X is a NaN, (format "%s" X) now returns
+"0.0e+NaN", "1.0e+NaN", etc., depending on X's significand.
 
 +++
 ** The function 'make-string' accepts an additional optional argument.
diff --git a/src/lread.c b/src/lread.c
index 50fc6ef..290b0f6 100644
--- a/src/lread.c
+++ b/src/lread.c
@@ -72,6 +72,10 @@ along with GNU Emacs.  If not, see <https://www.gnu.org/licenses/>.  */
 #define file_tell ftell
 #endif
 
+#if HAVE_IEEE754_H
+# include <ieee754.h>
+#endif
+
 /* The objects or placeholders read with the #n=object form.
 
    A hash table maps a number to either a placeholder (while the
@@ -3757,8 +3761,15 @@ string_to_number (char const *string, int base, int flags)
 	    {
 	      state |= E_EXP;
 	      cp += 3;
+#if HAVE_IEEE754_H
+	      union ieee754_double u
+		= { .ieee_nan = { .exponent = -1, .quiet_nan = 1,
+				  .mantissa0 = n >> 31 >> 1, .mantissa1 = n }};
+	      value = u.d;
+#else
 	      /* NAN is a "positive" NaN on all known Emacs hosts.  */
 	      value = NAN;
+#endif
 	    }
 	  else
 	    cp = ecp;
diff --git a/src/print.c b/src/print.c
index da6ec1a..add2160 100644
--- a/src/print.c
+++ b/src/print.c
@@ -40,6 +40,10 @@ along with GNU Emacs.  If not, see <https://www.gnu.org/licenses/>.  */
 #include <ftoastr.h>
 #include <math.h>
 
+#if HAVE_IEEE754_H
+# include <ieee754.h>
+#endif
+
 #ifdef WINDOWSNT
 # include <sys/socket.h> /* for F_DUPFD_CLOEXEC */
 #endif
@@ -1011,6 +1015,12 @@ float_to_string (char *buf, double data)
     }
   if (isnan (data))
     {
+#if HAVE_IEEE754_H
+      union ieee754_double u = { .d = data };
+      uprintmax_t hi = u.ieee_nan.mantissa0;
+      return sprintf (buf, &"-%"pMu".0e+NaN"[!u.ieee_nan.negative],
+		      (hi << 31 << 1) + u.ieee_nan.mantissa1);
+#else
       /* Prepend "-" if the NaN's sign bit is negative.
 	 The sign bit of a double is the bit that is 1 in -0.0.  */
       static char const NaN_string[] = "0.0e+NaN";
@@ -1029,6 +1039,7 @@ float_to_string (char *buf, double data)
 
       strcpy (buf + negative, NaN_string);
       return negative + sizeof NaN_string - 1;
+#endif
     }
 
   if (NILP (Vfloat_output_format)
-- 
2.7.4

Re: Using the GNU GMP Library for Bignums in Emacs

[Eli Zaretskii]

> From: Paul Eggert <eggert@cs.ucla.edu>
> Date: Wed, 1 Aug 2018 00:57:09 -0700
> Cc: Tom Tromey <tom@tromey.com>, Stefan Monnier <monnier@IRO.UMontreal.CA>,
> 	Richard Stallman <rms@gnu.org>, emacs-devel@gnu.org
> 
> No further comment on this, and I went with (3) and installed into master the 
> attached patch to do something along those lines on platforms that have 
> ieee754.h (mostly GNUish platforms, I expect). Other platforms are unaffected.

Thanks, but isn't there a way to test whether a platform supports IEEE
FP representation except by looking for ieee754.h?  AFAICS, that
header just defines a few data structures that any IEEE-compliant
platform should be able to work with.  So limiting this to glibc
platforms is too restrictive, I think.

Re: Using the GNU GMP Library for Bignums in Emacs

[Paul Eggert]

[-- Attachment #1: Type: text/plain, Size: 582 bytes --]

Eli Zaretskii wrote:
> isn't there a way to test whether a platform supports IEEE
> FP representation except by looking for ieee754.h?  AFAICS, that
> header just defines a few data structures that any IEEE-compliant
> platform should be able to work with.

We can try using ieee754.h on non-GNU platforms and see what happens. In theory, 
there could be weird platforms where it doesn't work (as no standard specifies 
IEEE 754 layout); in practice, we'll probably be OK. So I created a Gnulib 
module for ieee754.h, merged it in, and installed the attached into master.

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-Substitute-a-ieee754.h-on-hosts-lacking-it.patch --]
[-- Type: text/x-patch; name="0001-Substitute-a-ieee754.h-on-hosts-lacking-it.patch", Size: 16730 bytes --]

From d216d7d248199aa6c99cd642116717c5b301ae6d Mon Sep 17 00:00:00 2001
From: Paul Eggert <eggert@cs.ucla.edu>
Date: Wed, 1 Aug 2018 18:53:31 -0700
Subject: [PATCH] Substitute a <ieee754.h> on hosts lacking it

* .gitignore: Add lib/ieee754.h.
* admin/merge-gnulib (GNULIB_MODULES): Add ieee754-h.
* configure.ac: Remove ieee754.h check, as Gnulib now does that.
* etc/NEWS: Mention this.
* lib/gnulib.mk.in, m4/gnulib-comp.m4: Regenerate.
* lib/ieee754.in.h, m4/ieee754-h.m4: New files, from Gnulib.
* src/lisp.h (IEEE_FLOATING_POINT): Now a macro so that it
can be used in #if.
* src/lread.c, src/print.c: Include <ieee754.h> if
IEEE_FLOATING_POINT, not if HAVE_IEEE754_H.
* src/lread.c (string_to_number):
* src/print.c (float_to_string):
Process NaNs only on IEEE hosts, and assume <ieee754.h>
in that case.
---
 .gitignore         |   1 +
 admin/merge-gnulib |   2 +-
 configure.ac       |   1 -
 etc/NEWS           |   6 +-
 lib/gnulib.mk.in   |  29 +++++++
 lib/ieee754.in.h   | 222 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 m4/gnulib-comp.m4  |   4 +
 m4/ieee754-h.m4    |  21 +++++
 src/lisp.h         |  11 +--
 src/lread.c        |   9 +--
 src/print.c        |  25 +-----
 11 files changed, 291 insertions(+), 40 deletions(-)
 create mode 100644 lib/ieee754.in.h
 create mode 100644 m4/ieee754-h.m4

diff --git a/.gitignore b/.gitignore
index d3712b0..26fe4bb 100644
--- a/.gitignore
+++ b/.gitignore
@@ -57,6 +57,7 @@ lib/execinfo.h
 lib/fcntl.h
 lib/getopt.h
 lib/getopt-cdefs.h
+lib/ieee754.h
 lib/inttypes.h
 lib/libgnu.a
 lib/limits.h
diff --git a/admin/merge-gnulib b/admin/merge-gnulib
index 39dfaee..1397ecf 100755
--- a/admin/merge-gnulib
+++ b/admin/merge-gnulib
@@ -35,7 +35,7 @@ GNULIB_MODULES=
   fcntl fcntl-h fdatasync fdopendir
   filemode filevercmp flexmember fpieee fstatat fsusage fsync
   getloadavg getopt-gnu gettime gettimeofday gitlog-to-changelog
-  ignore-value intprops largefile lstat
+  ieee754-h ignore-value intprops largefile lstat
   manywarnings memrchr minmax mkostemp mktime nstrftime
   pipe2 pselect pthread_sigmask putenv qcopy-acl readlink readlinkat
   sig2str socklen stat-time std-gnu11 stdalign stddef stdio
diff --git a/configure.ac b/configure.ac
index dbdcce7..b691867 100644
--- a/configure.ac
+++ b/configure.ac
@@ -1668,7 +1668,6 @@ AC_DEFUN
 
 dnl checks for header files
 AC_CHECK_HEADERS_ONCE(
-  ieee754.h
   linux/fs.h
   malloc.h
   sys/systeminfo.h
diff --git a/etc/NEWS b/etc/NEWS
index 9e7a765..6c79a46 100644
--- a/etc/NEWS
+++ b/etc/NEWS
@@ -880,9 +880,9 @@ Formerly, some of these functions ignored signs and significands of
 NaNs.  Now, all these functions treat NaN signs and significands as
 significant.  For example, (eql 0.0e+NaN -0.0e+NaN) now returns nil
 because the two NaNs have different signs; formerly it returned t.
-Also, on platforms that have <ieee754.h> Emacs now reads and prints
-NaN significands; e.g., if X is a NaN, (format "%s" X) now returns
-"0.0e+NaN", "1.0e+NaN", etc., depending on X's significand.
+Also, Emacs now reads and prints NaN significands; e.g., if X is a
+NaN, (format "%s" X) now returns "0.0e+NaN", "1.0e+NaN", etc.,
+depending on X's significand.
 
 +++
 ** The function 'make-string' accepts an additional optional argument.
diff --git a/lib/gnulib.mk.in b/lib/gnulib.mk.in
index e623921..7d28dcc 100644
--- a/lib/gnulib.mk.in
+++ b/lib/gnulib.mk.in
@@ -95,6 +95,7 @@
 #  gettime \
 #  gettimeofday \
 #  gitlog-to-changelog \
+#  ieee754-h \
 #  ignore-value \
 #  intprops \
 #  largefile \
@@ -220,6 +221,7 @@ GL_GENERATE_ALLOCA_H = @GL_GENERATE_ALLOCA_H@
 GL_GENERATE_BYTESWAP_H = @GL_GENERATE_BYTESWAP_H@
 GL_GENERATE_ERRNO_H = @GL_GENERATE_ERRNO_H@
 GL_GENERATE_EXECINFO_H = @GL_GENERATE_EXECINFO_H@
+GL_GENERATE_IEEE754_H = @GL_GENERATE_IEEE754_H@
 GL_GENERATE_LIMITS_H = @GL_GENERATE_LIMITS_H@
 GL_GENERATE_STDALIGN_H = @GL_GENERATE_STDALIGN_H@
 GL_GENERATE_STDDEF_H = @GL_GENERATE_STDDEF_H@
@@ -646,6 +648,7 @@ HAVE_WINSOCK2_H = @HAVE_WINSOCK2_H@
 HAVE_XSERVER = @HAVE_XSERVER@
 HAVE__EXIT = @HAVE__EXIT@
 HYBRID_MALLOC = @HYBRID_MALLOC@
+IEEE754_H = @IEEE754_H@
 IMAGEMAGICK_CFLAGS = @IMAGEMAGICK_CFLAGS@
 IMAGEMAGICK_LIBS = @IMAGEMAGICK_LIBS@
 INCLUDE_NEXT = @INCLUDE_NEXT@
@@ -1787,6 +1790,32 @@ EXTRA_libgnu_a_SOURCES += group-member.c
 endif
 ## end   gnulib module group-member
 
+## begin gnulib module ieee754-h
+ifeq (,$(OMIT_GNULIB_MODULE_ieee754-h))
+
+BUILT_SOURCES += $(IEEE754_H)
+
+# We need the following in order to create <ieee754.h> when the system
+# doesn't have one that works with the given compiler.
+ifneq (,$(GL_GENERATE_IEEE754_H))
+ieee754.h: ieee754.in.h $(top_builddir)/config.status
+	$(AM_V_GEN)rm -f $@-t && \
+	{ echo '/* DO NOT EDIT! GENERATED AUTOMATICALLY! */'; \
+	  sed -e 's/ifndef _GL_GNULIB_HEADER/if 0/g' \
+	    $(srcdir)/ieee754.in.h; \
+	} > $@-t && \
+	mv -f $@-t $@
+else
+ieee754.h: $(top_builddir)/config.status
+	rm -f $@
+endif
+MOSTLYCLEANFILES += ieee754.h ieee754.h-t
+
+EXTRA_DIST += ieee754.in.h
+
+endif
+## end   gnulib module ieee754-h
+
 ## begin gnulib module ignore-value
 ifeq (,$(OMIT_GNULIB_MODULE_ignore-value))
 
diff --git a/lib/ieee754.in.h b/lib/ieee754.in.h
new file mode 100644
index 0000000..316ac03
--- /dev/null
+++ b/lib/ieee754.in.h
@@ -0,0 +1,222 @@
+/* Copyright (C) 1992-2018 Free Software Foundation, Inc.
+   This file is part of the GNU C Library.
+
+   The GNU C Library is free software; you can redistribute it and/or
+   modify it under the terms of the GNU General Public
+   License as published by the Free Software Foundation; either
+   version 3 of the License, or (at your option) any later version.
+
+   The GNU C Library is distributed in the hope that it will be useful,
+   but WITHOUT ANY WARRANTY; without even the implied warranty of
+   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+   General Public License for more details.
+
+   You should have received a copy of the GNU General Public
+   License along with the GNU C Library; if not, see
+   <http://www.gnu.org/licenses/>.  */
+
+#ifndef _IEEE754_H
+
+#define _IEEE754_H 1
+
+#ifndef _GL_GNULIB_HEADER
+/* Ordinary glibc usage.  */
+# include <features.h>
+# include <endian.h>
+#else
+/* Gnulib usage.  */
+# ifndef __BEGIN_DECLS
+#  ifdef __cplusplus
+#   define __BEGIN_DECLS	extern "C" {
+#   define __END_DECLS		}
+#  else
+#   define __BEGIN_DECLS
+#   define __END_DECLS
+#  endif
+# endif
+# ifndef __FLOAT_WORD_ORDER
+#  define __LITTLE_ENDIAN	1234
+#  define __BIG_ENDIAN		4321
+#  ifdef WORDS_BIGENDIAN
+#   define __BYTE_ORDER __BIG_ENDIAN
+#  else
+#   define __BYTE_ORDER __LITTLE_ENDIAN
+#  endif
+#  define __FLOAT_WORD_ORDER __BYTE_ORDER
+# endif
+#endif
+
+__BEGIN_DECLS
+
+union ieee754_float
+  {
+    float f;
+
+    /* This is the IEEE 754 single-precision format.  */
+    struct
+      {
+#if	__BYTE_ORDER == __BIG_ENDIAN
+	unsigned int negative:1;
+	unsigned int exponent:8;
+	unsigned int mantissa:23;
+#endif				/* Big endian.  */
+#if	__BYTE_ORDER == __LITTLE_ENDIAN
+	unsigned int mantissa:23;
+	unsigned int exponent:8;
+	unsigned int negative:1;
+#endif				/* Little endian.  */
+      } ieee;
+
+    /* This format makes it easier to see if a NaN is a signalling NaN.  */
+    struct
+      {
+#if	__BYTE_ORDER == __BIG_ENDIAN
+	unsigned int negative:1;
+	unsigned int exponent:8;
+	unsigned int quiet_nan:1;
+	unsigned int mantissa:22;
+#endif				/* Big endian.  */
+#if	__BYTE_ORDER == __LITTLE_ENDIAN
+	unsigned int mantissa:22;
+	unsigned int quiet_nan:1;
+	unsigned int exponent:8;
+	unsigned int negative:1;
+#endif				/* Little endian.  */
+      } ieee_nan;
+  };
+
+#define IEEE754_FLOAT_BIAS	0x7f /* Added to exponent.  */
+
+
+union ieee754_double
+  {
+    double d;
+
+    /* This is the IEEE 754 double-precision format.  */
+    struct
+      {
+#if	__BYTE_ORDER == __BIG_ENDIAN
+	unsigned int negative:1;
+	unsigned int exponent:11;
+	/* Together these comprise the mantissa.  */
+	unsigned int mantissa0:20;
+	unsigned int mantissa1:32;
+#endif				/* Big endian.  */
+#if	__BYTE_ORDER == __LITTLE_ENDIAN
+# if	__FLOAT_WORD_ORDER == __BIG_ENDIAN
+	unsigned int mantissa0:20;
+	unsigned int exponent:11;
+	unsigned int negative:1;
+	unsigned int mantissa1:32;
+# else
+	/* Together these comprise the mantissa.  */
+	unsigned int mantissa1:32;
+	unsigned int mantissa0:20;
+	unsigned int exponent:11;
+	unsigned int negative:1;
+# endif
+#endif				/* Little endian.  */
+      } ieee;
+
+    /* This format makes it easier to see if a NaN is a signalling NaN.  */
+    struct
+      {
+#if	__BYTE_ORDER == __BIG_ENDIAN
+	unsigned int negative:1;
+	unsigned int exponent:11;
+	unsigned int quiet_nan:1;
+	/* Together these comprise the mantissa.  */
+	unsigned int mantissa0:19;
+	unsigned int mantissa1:32;
+#else
+# if	__FLOAT_WORD_ORDER == __BIG_ENDIAN
+	unsigned int mantissa0:19;
+	unsigned int quiet_nan:1;
+	unsigned int exponent:11;
+	unsigned int negative:1;
+	unsigned int mantissa1:32;
+# else
+	/* Together these comprise the mantissa.  */
+	unsigned int mantissa1:32;
+	unsigned int mantissa0:19;
+	unsigned int quiet_nan:1;
+	unsigned int exponent:11;
+	unsigned int negative:1;
+# endif
+#endif
+      } ieee_nan;
+  };
+
+#define IEEE754_DOUBLE_BIAS	0x3ff /* Added to exponent.  */
+
+
+union ieee854_long_double
+  {
+    long double d;
+
+    /* This is the IEEE 854 double-extended-precision format.  */
+    struct
+      {
+#if	__BYTE_ORDER == __BIG_ENDIAN
+	unsigned int negative:1;
+	unsigned int exponent:15;
+	unsigned int empty:16;
+	unsigned int mantissa0:32;
+	unsigned int mantissa1:32;
+#endif
+#if	__BYTE_ORDER == __LITTLE_ENDIAN
+# if	__FLOAT_WORD_ORDER == __BIG_ENDIAN
+	unsigned int exponent:15;
+	unsigned int negative:1;
+	unsigned int empty:16;
+	unsigned int mantissa0:32;
+	unsigned int mantissa1:32;
+# else
+	unsigned int mantissa1:32;
+	unsigned int mantissa0:32;
+	unsigned int exponent:15;
+	unsigned int negative:1;
+	unsigned int empty:16;
+# endif
+#endif
+      } ieee;
+
+    /* This is for NaNs in the IEEE 854 double-extended-precision format.  */
+    struct
+      {
+#if	__BYTE_ORDER == __BIG_ENDIAN
+	unsigned int negative:1;
+	unsigned int exponent:15;
+	unsigned int empty:16;
+	unsigned int one:1;
+	unsigned int quiet_nan:1;
+	unsigned int mantissa0:30;
+	unsigned int mantissa1:32;
+#endif
+#if	__BYTE_ORDER == __LITTLE_ENDIAN
+# if	__FLOAT_WORD_ORDER == __BIG_ENDIAN
+	unsigned int exponent:15;
+	unsigned int negative:1;
+	unsigned int empty:16;
+	unsigned int mantissa0:30;
+	unsigned int quiet_nan:1;
+	unsigned int one:1;
+	unsigned int mantissa1:32;
+# else
+	unsigned int mantissa1:32;
+	unsigned int mantissa0:30;
+	unsigned int quiet_nan:1;
+	unsigned int one:1;
+	unsigned int exponent:15;
+	unsigned int negative:1;
+	unsigned int empty:16;
+# endif
+#endif
+      } ieee_nan;
+  };
+
+#define IEEE854_LONG_DOUBLE_BIAS 0x3fff
+
+__END_DECLS
+
+#endif /* ieee754.h */
diff --git a/m4/gnulib-comp.m4 b/m4/gnulib-comp.m4
index a6e3be3..494c77c 100644
--- a/m4/gnulib-comp.m4
+++ b/m4/gnulib-comp.m4
@@ -101,6 +101,7 @@ AC_DEFUN
   # Code from module gettimeofday:
   # Code from module gitlog-to-changelog:
   # Code from module group-member:
+  # Code from module ieee754-h:
   # Code from module ignore-value:
   # Code from module include_next:
   # Code from module intprops:
@@ -295,6 +296,7 @@ AC_DEFUN
     gl_PREREQ_GETTIMEOFDAY
   fi
   gl_SYS_TIME_MODULE_INDICATOR([gettimeofday])
+  gl_IEEE754_H
   gl_INTTYPES_INCOMPLETE
   AC_REQUIRE([gl_LARGEFILE])
   gl_LIMITS_H
@@ -895,6 +897,7 @@ AC_DEFUN
   lib/gettimeofday.c
   lib/gl_openssl.h
   lib/group-member.c
+  lib/ieee754.in.h
   lib/ignore-value.h
   lib/intprops.h
   lib/inttypes.in.h
@@ -1017,6 +1020,7 @@ AC_DEFUN
   m4/gl-openssl.m4
   m4/gnulib-common.m4
   m4/group-member.m4
+  m4/ieee754-h.m4
   m4/include_next.m4
   m4/inttypes.m4
   m4/largefile.m4
diff --git a/m4/ieee754-h.m4 b/m4/ieee754-h.m4
new file mode 100644
index 0000000..bf7c332
--- /dev/null
+++ b/m4/ieee754-h.m4
@@ -0,0 +1,21 @@
+# Configure ieee754-h module
+
+dnl Copyright 2018 Free Software Foundation, Inc.
+dnl This file is free software; the Free Software Foundation
+dnl gives unlimited permission to copy and/or distribute it,
+dnl with or without modifications, as long as this notice is preserved.
+
+AC_DEFUN([gl_IEEE754_H],
+[
+  AC_REQUIRE([AC_C_BIGENDIAN])
+  AC_CHECK_HEADERS_ONCE([ieee754.h])
+  if test $ac_cv_header_ieee754_h = yes; then
+    IEEE754_H=
+  else
+    IEEE754_H=ieee754.h
+    AC_DEFINE([_GL_REPLACE_IEEE754_H], 1,
+              [Define to 1 if <ieee754.h> is missing.])
+  fi
+  AC_SUBST([IEEE754_H])
+  AM_CONDITIONAL([GL_GENERATE_IEEE754_H], [test -n "$IEEE754_H"])
+])
diff --git a/src/lisp.h b/src/lisp.h
index 96de60e..bdece81 100644
--- a/src/lisp.h
+++ b/src/lisp.h
@@ -2670,17 +2670,14 @@ XFLOAT_DATA (Lisp_Object f)
 
 /* Most hosts nowadays use IEEE floating point, so they use IEC 60559
    representations, have infinities and NaNs, and do not trap on
-   exceptions.  Define IEEE_FLOATING_POINT if this host is one of the
+   exceptions.  Define IEEE_FLOATING_POINT to 1 if this host is one of the
    typical ones.  The C11 macro __STDC_IEC_559__ is close to what is
    wanted here, but is not quite right because Emacs does not require
    all the features of C11 Annex F (and does not require C11 at all,
    for that matter).  */
-enum
-  {
-    IEEE_FLOATING_POINT
-      = (FLT_RADIX == 2 && FLT_MANT_DIG == 24
-	 && FLT_MIN_EXP == -125 && FLT_MAX_EXP == 128)
-  };
+
+#define IEEE_FLOATING_POINT (FLT_RADIX == 2 && FLT_MANT_DIG == 24 \
+			     && FLT_MIN_EXP == -125 && FLT_MAX_EXP == 128)
 
 /* A character, declared with the following typedef, is a member
    of some character set associated with the current buffer.  */
diff --git a/src/lread.c b/src/lread.c
index 290b0f6..9a025d8 100644
--- a/src/lread.c
+++ b/src/lread.c
@@ -72,7 +72,7 @@ along with GNU Emacs.  If not, see <https://www.gnu.org/licenses/>.  */
 #define file_tell ftell
 #endif
 
-#if HAVE_IEEE754_H
+#if IEEE_FLOATING_POINT
 # include <ieee754.h>
 #endif
 
@@ -3756,21 +3756,18 @@ string_to_number (char const *string, int base, int flags)
 	      cp += 3;
 	      value = INFINITY;
 	    }
+#if IEEE_FLOATING_POINT
 	  else if (cp[-1] == '+'
 		   && cp[0] == 'N' && cp[1] == 'a' && cp[2] == 'N')
 	    {
 	      state |= E_EXP;
 	      cp += 3;
-#if HAVE_IEEE754_H
 	      union ieee754_double u
 		= { .ieee_nan = { .exponent = -1, .quiet_nan = 1,
 				  .mantissa0 = n >> 31 >> 1, .mantissa1 = n }};
 	      value = u.d;
-#else
-	      /* NAN is a "positive" NaN on all known Emacs hosts.  */
-	      value = NAN;
-#endif
 	    }
+#endif
 	  else
 	    cp = ecp;
 	}
diff --git a/src/print.c b/src/print.c
index add2160..34c7fa1 100644
--- a/src/print.c
+++ b/src/print.c
@@ -40,7 +40,7 @@ along with GNU Emacs.  If not, see <https://www.gnu.org/licenses/>.  */
 #include <ftoastr.h>
 #include <math.h>
 
-#if HAVE_IEEE754_H
+#if IEEE_FLOATING_POINT
 # include <ieee754.h>
 #endif
 
@@ -1013,34 +1013,15 @@ float_to_string (char *buf, double data)
       strcpy (buf, minus_infinity_string + positive);
       return sizeof minus_infinity_string - 1 - positive;
     }
+#if IEEE_FLOATING_POINT
   if (isnan (data))
     {
-#if HAVE_IEEE754_H
       union ieee754_double u = { .d = data };
       uprintmax_t hi = u.ieee_nan.mantissa0;
       return sprintf (buf, &"-%"pMu".0e+NaN"[!u.ieee_nan.negative],
 		      (hi << 31 << 1) + u.ieee_nan.mantissa1);
-#else
-      /* Prepend "-" if the NaN's sign bit is negative.
-	 The sign bit of a double is the bit that is 1 in -0.0.  */
-      static char const NaN_string[] = "0.0e+NaN";
-      int i;
-      union { double d; char c[sizeof (double)]; } u_data, u_minus_zero;
-      bool negative = 0;
-      u_data.d = data;
-      u_minus_zero.d = - 0.0;
-      for (i = 0; i < sizeof (double); i++)
-	if (u_data.c[i] & u_minus_zero.c[i])
-	  {
-	    *buf = '-';
-	    negative = 1;
-	    break;
-	  }
-
-      strcpy (buf + negative, NaN_string);
-      return negative + sizeof NaN_string - 1;
-#endif
     }
+#endif
 
   if (NILP (Vfloat_output_format)
       || !STRINGP (Vfloat_output_format))
-- 
2.7.4

Re: Using the GNU GMP Library for Bignums in Emacs

[Eli Zaretskii]

> Cc: raeburn@raeburn.org, tom@tromey.com, monnier@IRO.UMontreal.CA,
>  rms@gnu.org, emacs-devel@gnu.org
> From: Paul Eggert <eggert@cs.ucla.edu>
> Date: Wed, 1 Aug 2018 19:06:49 -0700
> 
> We can try using ieee754.h on non-GNU platforms and see what happens. In theory, 
> there could be weird platforms where it doesn't work (as no standard specifies 
> IEEE 754 layout); in practice, we'll probably be OK. So I created a Gnulib 
> module for ieee754.h, merged it in, and installed the attached into master.

Thanks, works here on MS-Windows.

Re: Using the GNU GMP Library for Bignums in Emacs

[Achim Gratz]

Ken Raeburn writes:
> If we’re going to treat NaNs as having distinguishable bit patterns so
> we can tell whether two are the same or not, should we also have
> printable/readable forms that distinguish them? If not, how do we
> create and use distinguishable NaNs?

That would expose the NaN payload, using the non-specified bits of the
mantissa of an NaN to transport (sideband) information.  You will
probably find that no one really expects these bits to be used…

> For that matter, are we guaranteed that operations like (/ 0.0 0.0)
> will always generate NaNs with the same bit pattern?

On the same platform? Likely.  On different platforms, surely not.

> I would’ve guessed that it might be preferable to go the other
> direction, and use one canonical NaN value in Lisp, which would thus
> always be eq/eql to all other NaN expressions.

Ignoring the payload bits would probably be OK, but that still gives you
at least two different NaN (positive and negative) to deal with.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

SD adaptation for Waldorf Blofeld V1.15B11:
http://Synth.Stromeko.net/Downloads.html#WaldorfSDada

Re: Using the GNU GMP Library for Bignums in Emacs

[Markus Triska]

"Siraphob (Ben) Phipathananunth" <siraben@disroot.org> writes:

> The benefits of linking GNU GMP are clear. It would make it easier to
> extend the math capabilities of Emacs, while making it faster and less
> error-prone. However, the downsides, if any exist, should be discussed
> as well, to gauge whether pursuing such a task would be fruitful.

Using GMP has a significant downside if you run out of memory: There’s
currently no defined way for the allocation functions to recover from an
error such as out of memory, they must *terminate* program execution.

Please see the following page for details:

   https://gmplib.org/manual/Custom-Allocation.html

Thus, situtations where you can currently throw an Emacs error that can
be handled in user code would instead terminate the Emacs process. This
can for example arise from malicious input that involves very large
integers as results (7^7^7^7 etc.), and of course also elsewhere.

Also, it may become hard to stop long GMP calculations, whereas you can
easily stop computations that are written in Elisp. Thus, long GMP
calculations may lead to denial of editing attacks.

All the best,
Markus

Re: Using the GNU GMP Library for Bignums in Emacs

[Eli Zaretskii]

> From: Markus Triska <triska@metalevel.at>
> Date: Sat, 21 Apr 2018 18:46:07 +0200
> 
> Using GMP has a significant downside if you run out of memory: There’s
> currently no defined way for the allocation functions to recover from an
> error such as out of memory, they must *terminate* program execution.
> 
> Please see the following page for details:
> 
>    https://gmplib.org/manual/Custom-Allocation.html
> 
> Thus, situtations where you can currently throw an Emacs error that can
> be handled in user code would instead terminate the Emacs process. This
> can for example arise from malicious input that involves very large
> integers as results (7^7^7^7 etc.), and of course also elsewhere.
> 
> Also, it may become hard to stop long GMP calculations, whereas you can
> easily stop computations that are written in Elisp. Thus, long GMP
> calculations may lead to denial of editing attacks.

How are those dangers different from using any other external
library.  Like the JSON library, for excample, or libxml2?

Re: Using the GNU GMP Library for Bignums in Emacs

[Markus Triska]

Eli Zaretskii <eliz@gnu.org> writes:

> How are those dangers different from using any other external
> library.  Like the JSON library, for excample, or libxml2?

Please note that in particular the first issue I mentioned is quite
specific to GMP: It could be solved by providing a different API, or by
generalizing the existing API to let applications safely handle such
situations. Some of the libraries you mention may support this.

The second issue I mentioned is also quite specific to GMP, since short
arithmetic expressions may cause long running times for calculations. Of
course, this issue may in principle also arise in other libraries.

All the best,
Markus

Re: Using the GNU GMP Library for Bignums in Emacs

[Paul Eggert]

Markus Triska wrote:

> Using GMP has a significant downside if you run out of memory: There’s
> currently no defined way for the allocation functions to recover from an
> error such as out of memory, they must *terminate* program execution.

This should not be a problem if we use GMP's mpn_* functions. They never 
allocate memory; it is the user's responsibility to allocate it. This should fit 
better with how Emacs does things internally.

> Also, it may become hard to stop long GMP calculations, whereas you can
> easily stop computations that are written in Elisp. Thus, long GMP
> calculations may lead to denial of editing attacks.

Yes, this could be a problem. Perhaps we'll need to extend GMP to fix it. On the 
other hand, no doubt there are other, similar problems in GNU Emacs (just look 
for calls to memcpy :-), and perhaps this problem could just be added to the list.

Re: Using the GNU GMP Library for Bignums in Emacs

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > Yes, this could be a problem. Perhaps we'll need to extend GMP to fix it.

The only things that might interrupt Lisp code are signals.
If C-g is detected by a signal, it can interrupt almost anything in C,
including computation in a GMP function,

-- 
Dr Richard Stallman
President, Free Software Foundation (https://gnu.org, https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)
Skype: No way! See https://stallman.org/skype.html.

Re: Using the GNU GMP Library for Bignums in Emacs

[Eli Zaretskii]

> From: Richard Stallman <rms@gnu.org>
> Date: Sat, 21 Apr 2018 18:42:52 -0400
> Cc: triska@metalevel.at, emacs-devel@gnu.org
> 
> The only things that might interrupt Lisp code are signals.
> If C-g is detected by a signal, it can interrupt almost anything in C,
> including computation in a GMP function,

We've changed how signals are processed in Emacs several years ago.
Nowadays, a signal handler just sets a flag, and the Lisp interpreter
tests that flag "when appropriate" (normally, as part of maybe_quit).

This change was done to avoid non-trivial processing inside signal
handlers.

Re: Using the GNU GMP Library for Bignums in Emacs

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > We've changed how signals are processed in Emacs several years ago.
  > Nowadays, a signal handler just sets a flag, and the Lisp interpreter
  > tests that flag "when appropriate" (normally, as part of maybe_quit).

  > This change was done to avoid non-trivial processing inside signal
  > handlers.

Quiting always worked by setting a flag, tested by the QUIT macro.
However, as I implemented it, some constructs used to specify to quit
immediately straight from the signal handler.

I see that this feature has been deleted.  That is going to cause a
bad results.  There are quite a few places in the C code of Emacs
where execution can get stuck; that's why I added immediate quitting.
The problems I fixed that way have all come back.

What was the reason for this change?

I think we need to revert it, and fix whatever problem
in another way.

-- 
Dr Richard Stallman
President, Free Software Foundation (https://gnu.org, https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)
Skype: No way! See https://stallman.org/skype.html.

Re: Using the GNU GMP Library for Bignums in Emacs

[Paul Eggert]

Richard Stallman wrote:
> There are quite a few places in the C code of Emacs
> where execution can get stuck; that's why I added immediate quitting.
> The problems I fixed that way have all come back.

I hope they haven't come back. We tried to be methodical about preventing Emacs 
from getting stuck.

> What was the reason for this change?

Allowing immediate quitting from arbitrary points in execution causes problems 
when Emacs internals assume that actions are done in a certain order and are not 
just partly done when interrupted by a quit. In the old days we could assume 
that execution order at the machine level was the same order as in the C program 
source code, which meant that as long as the C source code did things in the 
right order Emacs could survive immediate quitting. That assumption is no longer 
true. To do things safely now, we need to either (1) add critical sections to 
Emacs where appropriate, or (2) test the quit flag at appropriate intervals. In 
practice (2) is easier to implement and to maintain.

Re: Using the GNU GMP Library for Bignums in Emacs

[Stefan Monnier]

>> There are quite a few places in the C code of Emacs
>> where execution can get stuck; that's why I added immediate quitting.
>> The problems I fixed that way have all come back.
[...]
>> What was the reason for this change?

The main reason was that you can't do it reliably in X11 because to
detect a C-g requires non-trivial processing which can't be performed
reliably from a signal handler.


        Stefan

Re: Using the GNU GMP Library for Bignums in Emacs

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > Allowing immediate quitting from arbitrary points in execution causes problems 

Immediate quitting was never allowed from arbitrary points in execution.
Only in places where immediate_quit was nonzero.
Those were generally code whose side effects would not matter
unless they were allowed to finish.

  > In the old days we could assume that execution order at the
  > machine level was the same order as in the C program source code,
  > which meant that as long as the C source code did things in the
  > right order Emacs could survive immediate quitting. That
  > assumption is no longer true.

That problem exists in theory.  Did the problem actually occur?

  >  To do things safely now, we need to either (1) add critical
  > sections to Emacs where appropriate, or (2) test the quit flag at
  > appropriate intervals

... in each and every place where immediate_quit was formerly used.

			   In practice (2) is easier to implement and
  > to maintain.

If (2) means to check for quitting as needed in each and every place
code where immediate_quit was formerly used, that would thoroughly do
the job, but did we handle all of those places?



-- 
Dr Richard Stallman
President, Free Software Foundation (https://gnu.org, https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)
Skype: No way! See https://stallman.org/skype.html.

Re: Using the GNU GMP Library for Bignums in Emacs

[Paul Eggert]

Richard Stallman wrote:

>    > In the old days we could assume that execution order at the
>    > machine level was the same order as in the C program source code,
>    > which meant that as long as the C source code did things in the
>    > right order Emacs could survive immediate quitting. That
>    > assumption is no longer true.
> 
> That problem exists in theory.  Did the problem actually occur?

As Eli wrote, we did make the change in response to a bug report. I do not 
recall all the details offhand. The theory is pretty clear, though.
> If (2) means to check for quitting as needed in each and every place
> code where immediate_quit was formerly used, that would thoroughly do
> the job, but did we handle all of those places?

We attempted to check for quitting at every place where a check is needed to 
avoid unbounded or too-long computation, and I recall using the immediate_quits 
in the old code as a cue for where to check for quitting in the new code. We 
tried to be conservative about this, and as I recall we put in some checks even 
when we weren't sure they were needed. Perhaps we missed some places, but if so 
we can fix them as problems arise (and in this sense the new method is similar 
to the old one).

Re: Using the GNU GMP Library for Bignums in Emacs

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

Thanks for explaining.

-- 
Dr Richard Stallman
President, Free Software Foundation (https://gnu.org, https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)
Skype: No way! See https://stallman.org/skype.html.

Re: Using the GNU GMP Library for Bignums in Emacs

[Eli Zaretskii]

> From: Richard Stallman <rms@gnu.org>
> CC: eggert@cs.ucla.edu, triska@metalevel.at, emacs-devel@gnu.org
> Date: Sun, 22 Apr 2018 23:34:58 -0400
> 
> Quiting always worked by setting a flag, tested by the QUIT macro.
> However, as I implemented it, some constructs used to specify to quit
> immediately straight from the signal handler.

This still happens, but much less frequently: only on text-mode
frames, and only when Emacs waits for input.

> I see that this feature has been deleted.  That is going to cause a
> bad results.  There are quite a few places in the C code of Emacs
> where execution can get stuck; that's why I added immediate quitting.
> The problems I fixed that way have all come back.
> 
> What was the reason for this change?

The main reasons were described and discussed in bug#12471, in the
context of making signal handling in Emacs more robust.  I reproduce
that description below.

In addition, the asynchronous input code caused problems and
limitations, the most (in)famous being that we needed to refrain from
calling malloc in places that could be potentially run from the signal
handler.

Also note this discussion about actually making synchronous input the
default one:

  http://lists.gnu.org/archive/html/emacs-devel/2008-03/msg00410.html

And here's your agreement to making that the default:

  http://lists.gnu.org/archive/html/emacs-devel/2008-03/msg01268.html

----------------------------------------------------------------------
 * Signal handlers can interrupt each other, leading to races.

 * Signals can be mishandled if they arrive right in the middle of
   code that is keeping track of whether signals have arrived.

 * Some variables are modified by signal handlers but are not
   declared 'volatile', which means accesses to them could be
   incorrectly optimized.

 * When debugging, the debugging code can get into an infinite
   signal-handling loop if there's a bug in the fatal error handler.

 * There are some bugs involving running out of memory in the middle
   of a vfork, in which signal handlers aren't restored correctly and
   Emacs is likely to misbehave.

 * Signals are always redirected to the main thread, resulting in
   incorrect backtraces when, for example, a subsidiary thread has
   a segmentation violation.  Thread-specific signals like SIGSEGV
   should have thread-specific backtraces.

 * When in batch mode, Emacs doesn't ignore SIGINT if the invoker has
   purposely ignored SIGINT.  Similarly for SIGTERM.  Emacs gets
   SIGHUP right, but the other two signals should be consistent
   with the usual behavior for batch programs.

 * Emacs isn't consistent about what tests it uses to decide whether
   it is in batch mode, leading to glitches when the tests disagree.

 * Emacs catches SIGPIPE, but it's better to ignore it, as this avoids
   races.

 * Emacs catches SIGFPE, but on IEEE hosts catching SIGFPE isn't
   needed and can mask bugs; it's better to catch SIGFPE only on (the
   now quite rare) non-IEEE hosts.

Re: Using the GNU GMP Library for Bignums in Emacs

[Emanuele Santoro]

On 21/04/2018 16:15, Siraphob (Ben) Phipathananunth wrote:
> What is the consensus on linking the GNU GMP library to Emacs so that
> packages such as Emacs Calc (and others) could benefit from using
> native types (i.e. "mpz_t") rather than reinventing the wheel?


My two cents: this should be added as optional feature and its presence,
if wanted/needed, should be enabled at compilation time.

Maybe in a few years we could see if it has been widely adopted and if
so make GNU GMP usage the default setting (and/or discard the old
implementation). This should allow a smoother transition.


Hope this helps,
-- 
Emanuele Santoro

Re: Using the GNU GMP Library for Bignums in Emacs

[Glenn Morris]

I don't know how useful it is today, but there is existing work on
this in other-branches/gerd_big.