Turning misc objects into pseudovectors

Turning misc objects into pseudovectors

[Paul Eggert]

Bug#32405 contains a patch that will get rid of the miscellaneous-object 
category of the Emacs Lisp interpreter, and will change these objects to be 
pseudovectors instead. The motivation is to simplify the interpreter and garbage 
collector and speed it up slightly, and to simplify potential future changes.

This change should be invisible to Emacs users, except that (garbage-collect) 
will now yield a data structure like this:

((conses 16 59897 12062) (symbols 48 15849 0) (strings 32 14605 1081) 
(string-bytes 1 295420) (vectors 16 7106) (vector-slots 8 403952 52398) (floats 
8 46 530) (intervals 56 36 16) (buffers 992 7))

That is, there is no sublist headed 'misc' any more, because there are no longer 
misc objects. If you see any problem with this change please comment in the bug 
report (this email contains a Reply-To: header that should work). Thanks.

bug#32405: Turning misc objects into pseudovectors

[Stefan Monnier]

> Bug#32405 contains a patch that will get rid of the miscellaneous-object
> category of the Emacs Lisp interpreter, and will change these objects to be
> pseudovectors instead.  The motivation is to simplify the interpreter and
> garbage collector and speed it up slightly, and to simplify potential
> future changes.

I like the idea, but:

AFAIK the main issue with pseudovectors is that their allocation is
slower and suffers more from fragmentation (because we don't use
a size-segregated allocation algorithm (like Linux's SLAB, for example)
for them).

Are you sure the new code is faster overall?

There is also a potential issue in terms of the resulting heap size of
markers (which may bump up from 6 words to 8 words, IIRC, unless your
patch does something to keep it down to 6), tho this is probably of no
real consequence.


        Stefan

Re: Turning misc objects into pseudovectors

[Tom Tromey]

>>>>> "Paul" == Paul Eggert <eggert@cs.ucla.edu> writes:

Paul> Bug#32405 contains a patch that will get rid of the
Paul> miscellaneous-object category of the Emacs Lisp interpreter, and will
Paul> change these objects to be pseudovectors instead. The motivation is to
Paul> simplify the interpreter and garbage collector and speed it up
Paul> slightly, and to simplify potential future changes.

This seems reasonable to me.  Do you plan to land it soon?
I am wondering if I should wait for this before merging bignum.
Currently bignums are implemented as misc types, not pseudovectors.

Tom

bug#32405: Turning misc objects into pseudovectors

[Paul Eggert]

Tom Tromey wrote:
> This seems reasonable to me.  Do you plan to land it soon?

Yes, in a day or two unless there are significant objections.

bug#32405: Turning misc objects into pseudovectors

[Eli Zaretskii]

> From: Tom Tromey <tom@tromey.com>
> Date: Thu, 09 Aug 2018 10:31:35 -0600
> Cc: Emacs Development <Emacs-devel@gnu.org>, 32405@debbugs.gnu.org
> 
> I am wondering if I should wait for this before merging bignum.
> Currently bignums are implemented as misc types, not pseudovectors.

I think the bignum merge should happen first, as it is a much more
important change than turning misc objects into pseudovectors.

Re: bug#32405: Turning misc objects into pseudovectors

[Eli Zaretskii]

> From: Tom Tromey <tom@tromey.com>
> Date: Thu, 09 Aug 2018 10:31:35 -0600
> Cc: Emacs Development <Emacs-devel@gnu.org>, 32405@debbugs.gnu.org
> 
> I am wondering if I should wait for this before merging bignum.
> Currently bignums are implemented as misc types, not pseudovectors.

I think the bignum merge should happen first, as it is a much more
important change than turning misc objects into pseudovectors.

bug#32405: Turning misc objects into pseudovectors

[Tom Tromey]

>>>>> "Eli" == Eli Zaretskii <eliz@gnu.org> writes:

>> From: Tom Tromey <tom@tromey.com>
>> Date: Thu, 09 Aug 2018 10:31:35 -0600
>> Cc: Emacs Development <Emacs-devel@gnu.org>, 32405@debbugs.gnu.org
>> 
>> I am wondering if I should wait for this before merging bignum.
>> Currently bignums are implemented as misc types, not pseudovectors.

Eli> I think the bignum merge should happen first, as it is a much more
Eli> important change than turning misc objects into pseudovectors.

Ok.

To anyone concerned - please be sure to take a look at the bignum
branch.  I will merge it soon, maybe this weekend depending on my free
time.

Tom

Re: bug#32405: Turning misc objects into pseudovectors

[Tom Tromey]

>>>>> "Eli" == Eli Zaretskii <eliz@gnu.org> writes:

>> From: Tom Tromey <tom@tromey.com>
>> Date: Thu, 09 Aug 2018 10:31:35 -0600
>> Cc: Emacs Development <Emacs-devel@gnu.org>, 32405@debbugs.gnu.org
>> 
>> I am wondering if I should wait for this before merging bignum.
>> Currently bignums are implemented as misc types, not pseudovectors.

Eli> I think the bignum merge should happen first, as it is a much more
Eli> important change than turning misc objects into pseudovectors.

Ok.

To anyone concerned - please be sure to take a look at the bignum
branch.  I will merge it soon, maybe this weekend depending on my free
time.

Tom

bug#32405: Turning misc objects into pseudovectors

[Paul Eggert]

Tom Tromey wrote:
> To anyone concerned - please be sure to take a look at the bignum
> branch.  I will merge it soon, maybe this weekend depending on my free
> time.

Thanks for doing all that! I am looking forward to using bignums.

After you merged it into master, I rebased the Bug#32405 patches to turn misc 
objects (including bignums) into pseudovectors and installed them into master. 
Closing the bug report.

Re: bug#32405: Turning misc objects into pseudovectors

[Paul Eggert]

Tom Tromey wrote:
> To anyone concerned - please be sure to take a look at the bignum
> branch.  I will merge it soon, maybe this weekend depending on my free
> time.

Thanks for doing all that! I am looking forward to using bignums.

After you merged it into master, I rebased the Bug#32405 patches to turn misc 
objects (including bignums) into pseudovectors and installed them into master. 
Closing the bug report.

Re: bug#32405: Turning misc objects into pseudovectors

[Eli Zaretskii]

> From: Paul Eggert <eggert@cs.ucla.edu>
> Date: Sat, 11 Aug 2018 18:53:43 -0700
> Cc: 32405-done@debbugs.gnu.org, Emacs-devel@gnu.org
> 
> After you merged it into master, I rebased the Bug#32405 patches to turn misc 
> objects (including bignums) into pseudovectors and installed them into master. 

Thanks, but it sounds like .gdbinit needs some update as followup,
because it still uses Lisp_Misc, and various unrelated commands (like
"xtype") barf.

Re: bug#32405: Turning misc objects into pseudovectors

[Paul Eggert]

[-- Attachment #1: Type: text/plain, Size: 142 bytes --]

Eli Zaretskii wrote:
> it sounds like .gdbinit needs some update as followup,

Sure does. Thanks for catching that. I installed the attached.

[-- Attachment #2: 0001-Adjust-.gdbinit-to-removal-of-misc-objects.patch --]
[-- Type: text/x-patch, Size: 2444 bytes --]

From f99ee7378f8529e748f894859f305d4cca2483e4 Mon Sep 17 00:00:00 2001
From: Paul Eggert <eggert@cs.ucla.edu>
Date: Sun, 12 Aug 2018 11:46:07 -0700
Subject: [PATCH] Adjust .gdbinit to removal of misc objects

* src/.gdbinit (xtype, xpr): Adjust.
(xmisctype, xmiscfree): Remove.
---
 src/.gdbinit | 46 +++++++++-------------------------------------
 1 file changed, 9 insertions(+), 37 deletions(-)

diff --git a/src/.gdbinit b/src/.gdbinit
index 3cebdff5e7..ae6f13a103 100644
--- a/src/.gdbinit
+++ b/src/.gdbinit
@@ -649,17 +649,13 @@ define xtype
   xgettype $
   output $type
   echo \n
-  if $type == Lisp_Misc
-    xmisctype
-  else
-    if $type == Lisp_Vectorlike
-      xvectype
-    end
+  if $type == Lisp_Vectorlike
+    xvectype
   end
 end
 document xtype
 Print the type of $, assuming it is an Emacs Lisp value.
-If the first type printed is Lisp_Vector or Lisp_Misc,
+If the first type printed is Lisp_Vectorlike,
 a second line gives the more precise type.
 end
 
@@ -711,15 +707,6 @@ Print the size of $
 This command assumes that $ is a Lisp_Object.
 end
 
-define xmisctype
-  xgetptr $
-  output (enum Lisp_Misc_Type) (((struct Lisp_Free *) $ptr)->type)
-  echo \n
-end
-document xmisctype
-Assume that $ is some misc type and print its specific type.
-end
-
 define xint
   xgetint $
   print $int
@@ -754,15 +741,6 @@ Print $ as a overlay pointer.
 This command assumes that $ is an Emacs Lisp overlay value.
 end
 
-define xmiscfree
-  xgetptr $
-  print (struct Lisp_Free *) $ptr
-end
-document xmiscfree
-Print $ as a misc free-cell pointer.
-This command assumes that $ is an Emacs Lisp Misc value.
-end
-
 define xsymbol
   set $sym = $
   xgetsym $sym
@@ -1015,18 +993,6 @@ define xpr
   if $type == Lisp_Float
     xfloat
   end
-  if $type == Lisp_Misc
-    set $misc = (enum Lisp_Misc_Type) (((struct Lisp_Free *) $ptr)->type)
-    if $misc == Lisp_Misc_Free
-      xmiscfree
-    end
-    if $misc == Lisp_Misc_Marker
-      xmarker
-    end
-    if $misc == Lisp_Misc_Overlay
-      xoverlay
-    end
-  end
   if $type == Lisp_Vectorlike
     set $size = ((struct Lisp_Vector *) $ptr)->header.size
     if ($size & PSEUDOVECTOR_FLAG)
@@ -1034,6 +1000,12 @@ define xpr
       if $vec == PVEC_NORMAL_VECTOR
 	xvector
       end
+      if $vec == PVEC_MARKER
+        xmarker
+      end
+      if $vec == PVEC_OVERLAY
+        xoverlay
+      end
       if $vec == PVEC_PROCESS
 	xprocess
       end
-- 
2.17.1

bug#32405: Turning misc objects into pseudovectors

[Paul Eggert]

[-- Attachment #1: Type: text/plain, Size: 277 bytes --]

On 08/11/2018 06:53 PM, Paul Eggert wrote:
> I rebased the Bug#32405 patches to turn misc objects (including 
> bignums) into pseudovectors and installed them into master.

I discovered a few more traces of the old misc objects, and removed them 
by installing the attached.



[-- Attachment #2: 0001-Remove-more-traces-of-misc-Bug-32405.patch --]
[-- Type: text/x-patch, Size: 6711 bytes --]

From 27a161e4060084ea093773be0716c10534847168 Mon Sep 17 00:00:00 2001
From: Paul Eggert <eggert@cs.ucla.edu>
Date: Tue, 14 Aug 2018 12:07:09 -0700
Subject: [PATCH] Remove more traces of misc (Bug#32405)

Remove misc-objects-consed and the misc component of
memory-use-count, since misc objects no longer exist.
* doc/lispref/internals.texi, etc/NEWS: Mention this,
and adjust better to recent removal of misc objects.
* src/alloc.c (MEM_TYPE_MISC): Remove; no longer used.
(Fmemory_use_counts): Omit misc count, since miscs
no longer exist.
(misc-objects-consed): Remove.
---
 doc/lispref/internals.texi | 25 ++++++++-----------------
 etc/NEWS                   |  8 +++++---
 src/alloc.c                | 10 +---------
 3 files changed, 14 insertions(+), 29 deletions(-)

diff --git a/doc/lispref/internals.texi b/doc/lispref/internals.texi
index c72dbb5079..3fe28446ea 100644
--- a/doc/lispref/internals.texi
+++ b/doc/lispref/internals.texi
@@ -246,8 +246,8 @@ Garbage Collection
 
 @cindex vector-like objects, storage
 @cindex storage of vector-like Lisp objects
-  Beyond the basic vector, a lot of objects like window, buffer, and
-frame are managed as if they were vectors.  The corresponding C data
+  Beyond the basic vector, a lot of objects like markers, overlays and
+buffers are managed as if they were vectors.  The corresponding C data
 structures include the @code{union vectorlike_header} field whose
 @code{size} member contains the subtype enumerated by @code{enum pvec_type}
 and an information about how many @code{Lisp_Object} fields this structure
@@ -579,6 +579,8 @@ Memory Usage
 @defvar vector-cells-consed
 The total number of vector cells that have been allocated so far
 in this Emacs session.
+This includes vector-like objects such as markers and overlays, plus
+certain objects not visible to users.
 @end defvar
 
 @defvar symbols-consed
@@ -591,12 +593,6 @@ Memory Usage
 in this session.
 @end defvar
 
-@defvar misc-objects-consed
-The total number of miscellaneous objects that have been allocated so
-far in this session.  These include markers and overlays, plus
-certain objects not visible to users.
-@end defvar
-
 @defvar intervals-consed
 The total number of intervals that have been allocated so far
 in this Emacs session.
@@ -987,7 +983,7 @@ Object Internals
   In C, the tagged pointer is an object of type @code{Lisp_Object}.  Any
 initialized variable of such a type always holds the value of one of the
 following basic data types: integer, symbol, string, cons cell, float,
-vectorlike or miscellaneous object.  Each of these data types has the
+or vectorlike object.  Each of these data types has the
 corresponding tag value.  All tags are enumerated by @code{enum Lisp_Type}
 and placed into a 3-bit bitfield of the @code{Lisp_Object}.  The rest of the
 bits is the value itself.  Integers are immediate, i.e., directly
@@ -1019,18 +1015,13 @@ Object Internals
 
 @item struct Lisp_Float
 Floating-point value.
-
-@item union Lisp_Misc
-Miscellaneous kinds of objects which don't fit into any of the above.
 @end table
 
   These types are the first-class citizens of an internal type system.
-Since the tag space is limited, all other types are the subtypes of either
-@code{Lisp_Vectorlike} or @code{Lisp_Misc}.  Vector subtypes are enumerated
+Since the tag space is limited, all other types are the subtypes of
+@code{Lisp_Vectorlike}.  Vector subtypes are enumerated
 by @code{enum pvec_type}, and nearly all complex objects like windows, buffers,
-frames, and processes fall into this category.  The rest of special types,
-including markers and overlays, are enumerated by @code{enum Lisp_Misc_Type}
-and form the set of subtypes of @code{Lisp_Misc}.
+frames, and processes fall into this category.
 
   Below there is a description of a few subtypes of @code{Lisp_Vectorlike}.
 Buffer object represents the text to display and edit.  Window is the part
diff --git a/etc/NEWS b/etc/NEWS
index e381a546a9..f1d09a2b63 100644
--- a/etc/NEWS
+++ b/etc/NEWS
@@ -831,9 +831,11 @@ is backwards-compatible with versions of Emacs in which the old function
 exists.  See the node "Displaying Buffers in Side Windows" in the ELisp
 manual for more details.
 
-** The 'garbage-collect' function no longer returns a 'misc' component
-because garbage collection no longer treats miscellaneous objects
-specially; they are now allocated like any other pseudovector.
+** garbage collection no longer treats miscellaneous objects specially;
+they are now allocated like any other pseudovector.  As a result, the
+'garbage-collect' and 'memory-use-count' functions no longer return a
+'misc' component, and the 'misc-objects-consed' variable has been
+removed.
 
 \f
 * Lisp Changes in Emacs 27.1
diff --git a/src/alloc.c b/src/alloc.c
index fb8a8c98b0..6a93821159 100644
--- a/src/alloc.c
+++ b/src/alloc.c
@@ -379,7 +379,6 @@ enum mem_type
   MEM_TYPE_BUFFER,
   MEM_TYPE_CONS,
   MEM_TYPE_STRING,
-  MEM_TYPE_MISC,
   MEM_TYPE_SYMBOL,
   MEM_TYPE_FLOAT,
   /* Since all non-bool pseudovectors are small enough to be
@@ -7023,11 +7022,10 @@ Each of these counters increments for a certain kind of object.
 The counters wrap around from the largest positive integer to zero.
 Garbage collection does not decrease them.
 The elements of the value are as follows:
-  (CONSES FLOATS VECTOR-CELLS SYMBOLS STRING-CHARS MISCS INTERVALS STRINGS)
+  (CONSES FLOATS VECTOR-CELLS SYMBOLS STRING-CHARS INTERVALS STRINGS)
 All are in units of 1 = one object consed
 except for VECTOR-CELLS and STRING-CHARS, which count the total length of
 objects consed.
-MISCS include overlays, markers, and some internal types.
 Frames, windows, buffers, and subprocesses count as vectors
   (but the contents of a buffer's text do not count here).  */)
   (void)
@@ -7038,7 +7036,6 @@ Frames, windows, buffers, and subprocesses count as vectors
 		bounded_number (vector_cells_consed),
 		bounded_number (symbols_consed),
 		bounded_number (string_chars_consed),
-		bounded_number (misc_objects_consed),
 		bounded_number (intervals_consed),
 		bounded_number (strings_consed));
 }
@@ -7297,11 +7294,6 @@ If this portion is smaller than `gc-cons-threshold', this is ignored.  */);
   DEFVAR_INT ("string-chars-consed", string_chars_consed,
 	      doc: /* Number of string characters that have been consed so far.  */);
 
-  DEFVAR_INT ("misc-objects-consed", misc_objects_consed,
-	      doc: /* Number of miscellaneous objects that have been consed so far.
-These include markers and overlays, plus certain objects not visible
-to users.  */);
-
   DEFVAR_INT ("intervals-consed", intervals_consed,
 	      doc: /* Number of intervals that have been consed so far.  */);
 
-- 
2.17.1

bug#32405: Turning misc objects into pseudovectors

[Tom Tromey]

>>>>> "Paul" == Paul Eggert <eggert@cs.ucla.edu> writes:

Paul> Bug#32405 contains a patch that will get rid of the
Paul> miscellaneous-object category of the Emacs Lisp interpreter, and will
Paul> change these objects to be pseudovectors instead. The motivation is to
Paul> simplify the interpreter and garbage collector and speed it up
Paul> slightly, and to simplify potential future changes.

This seems reasonable to me.  Do you plan to land it soon?
I am wondering if I should wait for this before merging bignum.
Currently bignums are implemented as misc types, not pseudovectors.

Tom