bug#32463: 27.0.50; (logior -1) => 4611686018427387903

[Pip Cet <pipcet@gmail.com>]

On Sun, Aug 19, 2018 at 10:59 AM Paul Eggert <eggert@cs.ucla.edu> wrote:
> Pip Cet wrote:
> > Even if memory isn't exhausted, creating a bignum larger than 16 GB
> > (our most-positive-bignum) results in an immediate crash with external
> > libgmp (Linux, x86_64), and that appears not to be easy to fix without
> > modifying gmp.
>
> Is there a libgmp bug report for this? or is there a reasonable way to
> characterize this arbitrary limitation in libgmp, so that Emacs does not go over
> the limit and crash? I've already put in one limit, and we can tighten that
> limit (or add more checks) if we know what libgmp's limits are.

libgmp stores mpzs as an int (this is a known bug, and plans to work
around it are on the GMP site) giving their current size in words (8
bytes each on x86_64). So the maximum bignum is 0x7fffffff * 64 one
bits. When that limit is reached, _mpz_realloc calls abort rather than
the user-provided reallocation function.

Of course, on POSIX systems, we can catch SIGABRT, but then we'd need
a flag to mark whether we're in a GMP function and another flag to
mark that we're calling abort from a signal handler which might have
interrupted the GMP function, and a third flag to mark that stack
overflow occurred in a signal handler which interrupted a GMP function
so we longjmp()ed out of GMP. On most ELF systems, we can probably
preload a library redirecting abort() for GMP. These are all ugly
solutions to what is ultimately a GMP limitation that should probably
be fixed, making GMP interruptible and abort-safe, which essentially
all interactive applications of it require.

I think we can be clever and wrap calls to mpz_mul_2exp (which can
create arbitrary bignums) and whatever Fexpt uses and make our
allocation function signal for bignums >= sqrt(most-positive-bignum)
(such numbers cannot produce the overflow condition if combined with
multiplication, addition, or subtraction). In fact, I'd suggest a much
lower limit until/unless the C-g issue is fixed, perhaps overridable
by a user preference if people really want to use big bignums.

(I've looked at implementing big rational numbers as an Emacs type
using GMP, too, and I think performance issues are more pronounced for
those, since even (absolutely) small numbers can have a huge
representation.)

> > That and left shifts are probably the ones to worry about for now.
> > Creating a large bignum by repeated multiplication will require at
> > least some intermediate bignums, which need to be allocated and copied
> > and thus probably alert the user to something going on.
>
> expt does bignums now too, so that's one more point of failure in this area.

Yes, that's what "that" referred to. Are there other operations that
create large bignums that I've missed?

I'm not sure what a reasonable limit would be, but I think a global
limit of bignum size to something that allows for "immediate"
computations would be best.