Re: Making --with-wide-int the default

[David Kastrup <dak@gnu.org>]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: David Kastrup <dak@gnu.org>
>> Cc: eggert@cs.ucla.edu,  lekktu@gmail.com,  emacs-devel@gnu.org
>> Date: Fri, 16 Oct 2015 16:03:09 +0200
>> 
>> >> Isn't that also changing the size of a Lisp cell?  And of integer
>> >> arithmetic?
>> >
>> > Part of it, yes.  But since a Lisp cell can hold buffer or string
>> > position, what else can you expect?
>> 
>> That buffer or string positions not representable in the 29 bits or so
>> available for integers in a Lisp cell are instead represented using a
>> gpm number for which there is a reference in the Lisp cell.
>
> But a GPM number is nothing more or less that some C data type.  It's
> not some magic wand that can solve problems just by being there.
>
>> How do you think Emacs managed 64-bit doubles "inside" of a 32-bit
>> integral type used for representing Lisp cells?
>
> So you are suggesting to make a Lisp integer represented like a Lisp
> float, i.e. accessible via a pointer?

For values not fitting into 29 bits, yes.

> But that in itself will slow down integer arithmetics, due to the need
> to dereference the pointer, won't it?

For values not fitting into 29 bits, yes.

>> GMP needs to be called only when leaving the range of "small
>> integers" (which is all we even have right now).  64-bit arithmetic
>> in your plan would be required for every single operation.  Yes, when
>> GMP kicks in, it will be slower than operations using exactly 64-bit
>> (not more, not less).  But it's the exception rather than the rule.
>
> But there has to be a test for when the exception happens, and that
> test is going to exert its price on every operation, whether it
> succeeds or fails the test.  I'm not at all sure it will be a net win.

With regard to memory requirements and processing speed for values
fitting easily into 32Bit (including booleans, indirect types, symbols,
various other stuff), it will be a net win.  Not everything in Lisp is
an integer.

>> So much the exception that we could entirely make do without it so
>> far.  It _will_ occur frequently when editing files larger than 1GB
>> or so.  But only in the _Lisp_ representations of those buffer
>> positions.  Everything implemented in C will use the integral data
>> type we choose for that, throwing an error when it gets exceeded.
>
> So you are suggesting to give up the transparent exposure of members
> of Lisp objects that are integers?

I have no idea what you are talking about.  Integers and Lisp cells are
not the same.  Treating them as the same is a bug.  Just ask Stefan.  So
I have no idea what you call "transparent exposure" here.

>> Yes, not all of the possible offsets will be representable in one
>> Lisp cell.
>
> So you are suggesting that a buffer position will not always be a
> simple number, but will sometimes be a cons cell?

No.  It will always be a Lisp integer, but Lisp integers will not always
be stored in a single cell.  But GMP does not store its numbers in cons
cells.

Really, I have no idea what your problem here is.  Smooth degradation to
fit rare large values compactly is not a particularly uncommon concept.
Emacs uses it, for example, in UTF-8 representations of 8-bit encodings.
It represents character codes 0-127 as-is, then switches to non-byte
representation of 128-255.  Why don't we use 32-bit characters from the
get-go?  Because they are rarely needed and we don't want to always pay
the price in memory for the rarely needed cases.

Doubling the size of Lisp cells does not just (roughly) double the size
of all integers.

-- 
David Kastrup