bug#32405: Turning misc objects into pseudovectors

bug#32405: Turning misc objects into pseudovectors

[Stefan Monnier]

> Bug#32405 contains a patch that will get rid of the miscellaneous-object
> category of the Emacs Lisp interpreter, and will change these objects to be
> pseudovectors instead.  The motivation is to simplify the interpreter and
> garbage collector and speed it up slightly, and to simplify potential
> future changes.

I like the idea, but:

AFAIK the main issue with pseudovectors is that their allocation is
slower and suffers more from fragmentation (because we don't use
a size-segregated allocation algorithm (like Linux's SLAB, for example)
for them).

Are you sure the new code is faster overall?

There is also a potential issue in terms of the resulting heap size of
markers (which may bump up from 6 words to 8 words, IIRC, unless your
patch does something to keep it down to 6), tho this is probably of no
real consequence.


        Stefan

bug#32405: Turning misc objects into pseudovectors

[Paul Eggert]

> AFAIK the main issue with pseudovectors is that their allocation is
> slower and suffers more from fragmentation (because we don't use
> a size-segregated allocation algorithm (like Linux's SLAB, for example)
> for them).

Pseudovectors do have size-segregated allocation; see the vector_free_lists 
array. Although it's not as fancy as Linux's SLAB, I hope it's enough for Emacs; 
if not we could of course make it fancier.

To some extent the point of this change is that we don't need a separate 
Lisp_Object tag in order to have size-segregated allocation, since pseudovectors 
already do that.

> Are you sure the new code is faster overall?

That's what I measured with 'make compile-always', yes. Of course this is just 
one benchmark. (My original intuition was that nobody would notice the 
difference....)

> There is also a potential issue in terms of the resulting heap size of
> markers (which may bump up from 6 words to 8 words, IIRC, unless your
> patch does something to keep it down to 6)

On a 64-bit platform the heap size of markers does not grow. The old size is 6 
words (sizeof (union aligned_Lisp_Misc) is 48), and the new size is also 6 words 
(sizeof (struct Lisp_Marker) is also 48).

bug#32405: Turning misc objects into pseudovectors

[Paul Eggert]

Paul Eggert wrote:
> 
>> There is also a potential issue in terms of the resulting heap size of
>> markers (which may bump up from 6 words to 8 words, IIRC, unless your
>> patch does something to keep it down to 6)
> 
> On a 64-bit platform the heap size of markers does not grow. The old size is 6 
> words (sizeof (union aligned_Lisp_Misc) is 48), and the new size is also 6 words 
> (sizeof (struct Lisp_Marker) is also 48).

I should have written that these numbers are for x86-64 on Fedora 28. They might 
differ on other 64-bit platforms, due to alignment issues.

bug#32405: Turning misc objects into pseudovectors

[Stefan Monnier]

>> AFAIK the main issue with pseudovectors is that their allocation is
>> slower and suffers more from fragmentation (because we don't use
>> a size-segregated allocation algorithm (like Linux's SLAB, for example)
>> for them).
>
> Pseudovectors do have size-segregated allocation; see the vector_free_lists
> array. Although it's not as fancy as Linux's SLAB, I hope it's enough for
> Emacs; if not we could of course make it fancier.

Ah, I had forgotten about vector_free_lists.  So I guess it will likely
suffer a bit more from fragmentation, but performance should be
good enough, thanks.

>> Are you sure the new code is faster overall?
> That's what I measured with 'make compile-always', yes.

Cool!

> Of course this is just one benchmark.  (My original intuition was that
> nobody would notice the difference....)

I think you're right.

>> There is also a potential issue in terms of the resulting heap size of
>> markers (which may bump up from 6 words to 8 words, IIRC, unless your
>> patch does something to keep it down to 6)
> On a 64-bit platform the heap size of markers does not grow.  The old size is
> 6 words (sizeof (union aligned_Lisp_Misc) is 48), and the new size is also
> 6 words (sizeof (struct Lisp_Marker) is also 48).

Perfect, thanks,


        Stefan

bug#32405: Turning misc objects into pseudovectors

[Tom Tromey]

>>>>> "Paul" == Paul Eggert <eggert@cs.ucla.edu> writes:

Paul> Bug#32405 contains a patch that will get rid of the
Paul> miscellaneous-object category of the Emacs Lisp interpreter, and will
Paul> change these objects to be pseudovectors instead. The motivation is to
Paul> simplify the interpreter and garbage collector and speed it up
Paul> slightly, and to simplify potential future changes.

This seems reasonable to me.  Do you plan to land it soon?
I am wondering if I should wait for this before merging bignum.
Currently bignums are implemented as misc types, not pseudovectors.

Tom

bug#32405: Turning misc objects into pseudovectors

[Paul Eggert]

Tom Tromey wrote:
> This seems reasonable to me.  Do you plan to land it soon?

Yes, in a day or two unless there are significant objections.

bug#32405: Turning misc objects into pseudovectors

[Eli Zaretskii]

> From: Tom Tromey <tom@tromey.com>
> Date: Thu, 09 Aug 2018 10:31:35 -0600
> Cc: Emacs Development <Emacs-devel@gnu.org>, 32405@debbugs.gnu.org
> 
> I am wondering if I should wait for this before merging bignum.
> Currently bignums are implemented as misc types, not pseudovectors.

I think the bignum merge should happen first, as it is a much more
important change than turning misc objects into pseudovectors.

bug#32405: Turning misc objects into pseudovectors

[Tom Tromey]

>>>>> "Eli" == Eli Zaretskii <eliz@gnu.org> writes:

>> From: Tom Tromey <tom@tromey.com>
>> Date: Thu, 09 Aug 2018 10:31:35 -0600
>> Cc: Emacs Development <Emacs-devel@gnu.org>, 32405@debbugs.gnu.org
>> 
>> I am wondering if I should wait for this before merging bignum.
>> Currently bignums are implemented as misc types, not pseudovectors.

Eli> I think the bignum merge should happen first, as it is a much more
Eli> important change than turning misc objects into pseudovectors.

Ok.

To anyone concerned - please be sure to take a look at the bignum
branch.  I will merge it soon, maybe this weekend depending on my free
time.

Tom

bug#32405: Turning misc objects into pseudovectors

[Paul Eggert]

Tom Tromey wrote:
> To anyone concerned - please be sure to take a look at the bignum
> branch.  I will merge it soon, maybe this weekend depending on my free
> time.

Thanks for doing all that! I am looking forward to using bignums.

After you merged it into master, I rebased the Bug#32405 patches to turn misc 
objects (including bignums) into pseudovectors and installed them into master. 
Closing the bug report.

bug#32405: Turning misc objects into pseudovectors

[Paul Eggert]

[-- Attachment #1: Type: text/plain, Size: 277 bytes --]

On 08/11/2018 06:53 PM, Paul Eggert wrote:
> I rebased the Bug#32405 patches to turn misc objects (including 
> bignums) into pseudovectors and installed them into master.

I discovered a few more traces of the old misc objects, and removed them 
by installing the attached.



[-- Attachment #2: 0001-Remove-more-traces-of-misc-Bug-32405.patch --]
[-- Type: text/x-patch, Size: 6711 bytes --]

From 27a161e4060084ea093773be0716c10534847168 Mon Sep 17 00:00:00 2001
From: Paul Eggert <eggert@cs.ucla.edu>
Date: Tue, 14 Aug 2018 12:07:09 -0700
Subject: [PATCH] Remove more traces of misc (Bug#32405)

Remove misc-objects-consed and the misc component of
memory-use-count, since misc objects no longer exist.
* doc/lispref/internals.texi, etc/NEWS: Mention this,
and adjust better to recent removal of misc objects.
* src/alloc.c (MEM_TYPE_MISC): Remove; no longer used.
(Fmemory_use_counts): Omit misc count, since miscs
no longer exist.
(misc-objects-consed): Remove.
---
 doc/lispref/internals.texi | 25 ++++++++-----------------
 etc/NEWS                   |  8 +++++---
 src/alloc.c                | 10 +---------
 3 files changed, 14 insertions(+), 29 deletions(-)

diff --git a/doc/lispref/internals.texi b/doc/lispref/internals.texi
index c72dbb5079..3fe28446ea 100644
--- a/doc/lispref/internals.texi
+++ b/doc/lispref/internals.texi
@@ -246,8 +246,8 @@ Garbage Collection
 
 @cindex vector-like objects, storage
 @cindex storage of vector-like Lisp objects
-  Beyond the basic vector, a lot of objects like window, buffer, and
-frame are managed as if they were vectors.  The corresponding C data
+  Beyond the basic vector, a lot of objects like markers, overlays and
+buffers are managed as if they were vectors.  The corresponding C data
 structures include the @code{union vectorlike_header} field whose
 @code{size} member contains the subtype enumerated by @code{enum pvec_type}
 and an information about how many @code{Lisp_Object} fields this structure
@@ -579,6 +579,8 @@ Memory Usage
 @defvar vector-cells-consed
 The total number of vector cells that have been allocated so far
 in this Emacs session.
+This includes vector-like objects such as markers and overlays, plus
+certain objects not visible to users.
 @end defvar
 
 @defvar symbols-consed
@@ -591,12 +593,6 @@ Memory Usage
 in this session.
 @end defvar
 
-@defvar misc-objects-consed
-The total number of miscellaneous objects that have been allocated so
-far in this session.  These include markers and overlays, plus
-certain objects not visible to users.
-@end defvar
-
 @defvar intervals-consed
 The total number of intervals that have been allocated so far
 in this Emacs session.
@@ -987,7 +983,7 @@ Object Internals
   In C, the tagged pointer is an object of type @code{Lisp_Object}.  Any
 initialized variable of such a type always holds the value of one of the
 following basic data types: integer, symbol, string, cons cell, float,
-vectorlike or miscellaneous object.  Each of these data types has the
+or vectorlike object.  Each of these data types has the
 corresponding tag value.  All tags are enumerated by @code{enum Lisp_Type}
 and placed into a 3-bit bitfield of the @code{Lisp_Object}.  The rest of the
 bits is the value itself.  Integers are immediate, i.e., directly
@@ -1019,18 +1015,13 @@ Object Internals
 
 @item struct Lisp_Float
 Floating-point value.
-
-@item union Lisp_Misc
-Miscellaneous kinds of objects which don't fit into any of the above.
 @end table
 
   These types are the first-class citizens of an internal type system.
-Since the tag space is limited, all other types are the subtypes of either
-@code{Lisp_Vectorlike} or @code{Lisp_Misc}.  Vector subtypes are enumerated
+Since the tag space is limited, all other types are the subtypes of
+@code{Lisp_Vectorlike}.  Vector subtypes are enumerated
 by @code{enum pvec_type}, and nearly all complex objects like windows, buffers,
-frames, and processes fall into this category.  The rest of special types,
-including markers and overlays, are enumerated by @code{enum Lisp_Misc_Type}
-and form the set of subtypes of @code{Lisp_Misc}.
+frames, and processes fall into this category.
 
   Below there is a description of a few subtypes of @code{Lisp_Vectorlike}.
 Buffer object represents the text to display and edit.  Window is the part
diff --git a/etc/NEWS b/etc/NEWS
index e381a546a9..f1d09a2b63 100644
--- a/etc/NEWS
+++ b/etc/NEWS
@@ -831,9 +831,11 @@ is backwards-compatible with versions of Emacs in which the old function
 exists.  See the node "Displaying Buffers in Side Windows" in the ELisp
 manual for more details.
 
-** The 'garbage-collect' function no longer returns a 'misc' component
-because garbage collection no longer treats miscellaneous objects
-specially; they are now allocated like any other pseudovector.
+** garbage collection no longer treats miscellaneous objects specially;
+they are now allocated like any other pseudovector.  As a result, the
+'garbage-collect' and 'memory-use-count' functions no longer return a
+'misc' component, and the 'misc-objects-consed' variable has been
+removed.
 
 \f
 * Lisp Changes in Emacs 27.1
diff --git a/src/alloc.c b/src/alloc.c
index fb8a8c98b0..6a93821159 100644
--- a/src/alloc.c
+++ b/src/alloc.c
@@ -379,7 +379,6 @@ enum mem_type
   MEM_TYPE_BUFFER,
   MEM_TYPE_CONS,
   MEM_TYPE_STRING,
-  MEM_TYPE_MISC,
   MEM_TYPE_SYMBOL,
   MEM_TYPE_FLOAT,
   /* Since all non-bool pseudovectors are small enough to be
@@ -7023,11 +7022,10 @@ Each of these counters increments for a certain kind of object.
 The counters wrap around from the largest positive integer to zero.
 Garbage collection does not decrease them.
 The elements of the value are as follows:
-  (CONSES FLOATS VECTOR-CELLS SYMBOLS STRING-CHARS MISCS INTERVALS STRINGS)
+  (CONSES FLOATS VECTOR-CELLS SYMBOLS STRING-CHARS INTERVALS STRINGS)
 All are in units of 1 = one object consed
 except for VECTOR-CELLS and STRING-CHARS, which count the total length of
 objects consed.
-MISCS include overlays, markers, and some internal types.
 Frames, windows, buffers, and subprocesses count as vectors
   (but the contents of a buffer's text do not count here).  */)
   (void)
@@ -7038,7 +7036,6 @@ Frames, windows, buffers, and subprocesses count as vectors
 		bounded_number (vector_cells_consed),
 		bounded_number (symbols_consed),
 		bounded_number (string_chars_consed),
-		bounded_number (misc_objects_consed),
 		bounded_number (intervals_consed),
 		bounded_number (strings_consed));
 }
@@ -7297,11 +7294,6 @@ If this portion is smaller than `gc-cons-threshold', this is ignored.  */);
   DEFVAR_INT ("string-chars-consed", string_chars_consed,
 	      doc: /* Number of string characters that have been consed so far.  */);
 
-  DEFVAR_INT ("misc-objects-consed", misc_objects_consed,
-	      doc: /* Number of miscellaneous objects that have been consed so far.
-These include markers and overlays, plus certain objects not visible
-to users.  */);
-
   DEFVAR_INT ("intervals-consed", intervals_consed,
 	      doc: /* Number of intervals that have been consed so far.  */);
 
-- 
2.17.1