unofficial mirror of emacs-devel@gnu.org 
 help / color / mirror / code / Atom feed
* Building Emacs overflowed pure space
@ 2006-07-18  8:38 Mathias Dahl
  2006-07-18  9:02 ` Ralph Moritz
  2006-07-19  0:44 ` Katsumi Yamaoka
  0 siblings, 2 replies; 72+ messages in thread
From: Mathias Dahl @ 2006-07-18  8:38 UTC (permalink / raw)


When starting Emacs after the compile today I got this message:

Warning (initialization): Building Emacs overflowed pure space.  (See the node \
Pure Storage in the Lisp manual for details.)

I have never seen this before, what has happened?

^ permalink raw reply	[flat|nested] 72+ messages in thread

* Re: Building Emacs overflowed pure space
  2006-07-18  8:38 Building Emacs overflowed pure space Mathias Dahl
@ 2006-07-18  9:02 ` Ralph Moritz
  2006-07-18 12:06   ` Mathias Dahl
  2006-07-18 15:00   ` Richard Stallman
  2006-07-19  0:44 ` Katsumi Yamaoka
  1 sibling, 2 replies; 72+ messages in thread
From: Ralph Moritz @ 2006-07-18  9:02 UTC (permalink / raw)
  Cc: emacs-devel

On 7/18/06, Mathias Dahl <mathias.dahl@gmail.com> wrote:
> When starting Emacs after the compile today I got this message:
>
> Warning (initialization): Building Emacs overflowed pure space.  (See the node \
> Pure Storage in the Lisp manual for details.)
>
> I have never seen this before, what has happened?

After a `cvs update' on Sunday morning, I got the same thing. I increased
the amount of memory (by 605500 bytes) allocated in `src/puresize.h' and
did `make clean; make bootstrap' and everything was fine.

I'm guessing that Emacs needs more memory because more files are
being loaded upon initialization (?).

-- 
Ralph Moritz
< ralphm @ members . fsf . org >
Ph: +27/0 317 055 268 (home)
    +27/0 846 269 070 (mobile)
"All this worldly wisdom was once the unamiable heresy of some wise man."
  - Henry David Thoreau

^ permalink raw reply	[flat|nested] 72+ messages in thread

* Re: Building Emacs overflowed pure space
  2006-07-18  9:02 ` Ralph Moritz
@ 2006-07-18 12:06   ` Mathias Dahl
  2006-07-18 13:37     ` David Hansen
  2006-07-18 15:00   ` Richard Stallman
  1 sibling, 1 reply; 72+ messages in thread
From: Mathias Dahl @ 2006-07-18 12:06 UTC (permalink / raw)
  Cc: emacs-devel

> After a `cvs update' on Sunday morning, I got the same thing. I increased
> the amount of memory (by 605500 bytes) allocated in `src/puresize.h' and
> did `make clean; make bootstrap' and everything was fine.

Thanks, I will try that.

Is this something I need to do each time I compile Emacs or will it be
taken care of centrally? I am quite lost when it comes to compiling
programs besides the normal "./configure; make; make install".

^ permalink raw reply	[flat|nested] 72+ messages in thread

* Re: Building Emacs overflowed pure space
  2006-07-18 12:06   ` Mathias Dahl
@ 2006-07-18 13:37     ` David Hansen
  0 siblings, 0 replies; 72+ messages in thread
From: David Hansen @ 2006-07-18 13:37 UTC (permalink / raw)


On Tue, 18 Jul 2006 14:06:00 +0200 Mathias Dahl wrote:

>> After a `cvs update' on Sunday morning, I got the same thing. I increased
>> the amount of memory (by 605500 bytes) allocated in `src/puresize.h' and
>> did `make clean; make bootstrap' and everything was fine.
>
> Thanks, I will try that.
>
> Is this something I need to do each time I compile Emacs or will it be
> taken care of centrally? I am quite lost when it comes to compiling
> programs besides the normal "./configure; make; make install".

As long as a cvs up doesn't indicate any conflicts (afaik
via a 'C' at the beginning of the updating message) your
change to puresize.h is successfully merged.

If there is a conflict it's very likely someone changed
exactly the line you changed.  Either resolve it manually or
simply delete puresize.h and check it out again (and hope
this change fixed your problem).

David

^ permalink raw reply	[flat|nested] 72+ messages in thread

* Re: Building Emacs overflowed pure space
  2006-07-18  9:02 ` Ralph Moritz
  2006-07-18 12:06   ` Mathias Dahl
@ 2006-07-18 15:00   ` Richard Stallman
  2006-07-18 18:55     ` Luc Teirlinck
  2006-07-18 19:29     ` Luc Teirlinck
  1 sibling, 2 replies; 72+ messages in thread
From: Richard Stallman @ 2006-07-18 15:00 UTC (permalink / raw)
  Cc: emacs-devel, mathias.dahl

    I'm guessing that Emacs needs more memory because more files are
    being loaded upon initialization (?).

ARE additional files being loaded now at startup?
If so, which ones -- and why?

^ permalink raw reply	[flat|nested] 72+ messages in thread

* Re: Building Emacs overflowed pure space
  2006-07-18 15:00   ` Richard Stallman
@ 2006-07-18 18:55     ` Luc Teirlinck
  2006-07-18 20:16       ` Eli Zaretskii
  2006-07-18 19:29     ` Luc Teirlinck
  1 sibling, 1 reply; 72+ messages in thread
From: Luc Teirlinck @ 2006-07-18 18:55 UTC (permalink / raw)
  Cc: mathias.dahl, ralphm, emacs-devel

Richard Stallman wrote:

       I'm guessing that Emacs needs more memory because more files are
       being loaded upon initialization (?).

   ARE additional files being loaded now at startup?
   If so, which ones -- and why?

Nothing in the changelogs seems to indicate that additional files are
being preloaded.  But several _changes_ to files that are preloaded
are constantly being made, increasing pure-bytes-used constantly.

What used to be done is that each time there was a pure space
overflow, BASE_PURESIZE got increased by 10000.  This yielded pure
space overflows every couple of months or so.  Now apparently
whenever there is a pure space overflow, BASE_PURESIZE gets
increased by 500.  This means that there will be a new pure space
overflow anywhere from a few minutes to a few days later.

Different people get somewhat different values of pure-bytes-used.
Eli was wondering that this could be caused by some bug.  I must
confess that I do not fully understand the problems (if any) involved,
but somebody suggested that the differences could be due to the way
you compile Emacs.  This may make a difference as to whether and when
cl.el gets loaded during compiling.  cl.el redefines some often used
functions, making their code longer.  This would seem consistent with
the fact that people doing `make bootstrap' seem to get a slightly
higher value than Eli, who I believe does not usually do `make bootstrap'.

Incrementing BASE_PURESIZE by increments of 500 forces people like me
who have a routinely a few tiny uncommitted change to the Emacs source
code to set SITELOAD_PURESIZE_EXTRA to some reasonable value, say 10000.

I suggest to go back to increasing BASE_PURESIZE by increments of 10000.
Any lesser value just will result in too frequent problems.

Sincerely,

Luc.

^ permalink raw reply	[flat|nested] 72+ messages in thread

* Re: Building Emacs overflowed pure space
  2006-07-18 15:00   ` Richard Stallman
  2006-07-18 18:55     ` Luc Teirlinck
@ 2006-07-18 19:29     ` Luc Teirlinck
  2006-07-19  6:04       ` Richard Stallman
  1 sibling, 1 reply; 72+ messages in thread
From: Luc Teirlinck @ 2006-07-18 19:29 UTC (permalink / raw)
  Cc: mathias.dahl, ralphm, emacs-devel

>From my previous reply:

   This may make a difference as to whether and when
   cl.el gets loaded during compiling.  cl.el redefines some often used
   functions, making their code longer.

I should have been more precise here.  cl.el redefines some often used
_macros_ which get inlined when compiled.  So this increases the pure
space used in all compiled versions of files that use those macros.
The difference in pure-bytes-used indeed only seems to occur in
compiled files.

Sincerely,

Luc.

^ permalink raw reply	[flat|nested] 72+ messages in thread

* Re: Building Emacs overflowed pure space
  2006-07-18 18:55     ` Luc Teirlinck
@ 2006-07-18 20:16       ` Eli Zaretskii
  2006-07-18 20:56         ` Stefan Monnier
  2006-07-18 22:01         ` Luc Teirlinck
  0 siblings, 2 replies; 72+ messages in thread
From: Eli Zaretskii @ 2006-07-18 20:16 UTC (permalink / raw)
  Cc: emacs-devel, rms, ralphm, mathias.dahl

> Date: Tue, 18 Jul 2006 13:55:42 -0500 (CDT)
> From: Luc Teirlinck <teirllm@dms.auburn.edu>
> Cc: mathias.dahl@gmail.com, ralphm@members.fsf.org, emacs-devel@gnu.org
> 
> Different people get somewhat different values of pure-bytes-used.
> Eli was wondering that this could be caused by some bug.  I must
> confess that I do not fully understand the problems (if any) involved,
> but somebody suggested that the differences could be due to the way
> you compile Emacs.  This may make a difference as to whether and when
> cl.el gets loaded during compiling.  cl.el redefines some often used
> functions, making their code longer.  This would seem consistent with
> the fact that people doing `make bootstrap' seem to get a slightly
> higher value than Eli, who I believe does not usually do `make bootstrap'.

No, this cannot explain the differences I was trying to investigate,
because I made a point, as part of my testing, of building Emacs both
with and without bootstrap.  I got the same numbers in both cases.

> I suggest to go back to increasing BASE_PURESIZE by increments of 10000.
> Any lesser value just will result in too frequent problems.

I don't think it's reasonable at this stage.  We are not supposed to
install changes that increase the pure size significantly; adding 10K
will just risk wasting memory in the released Emacs.

^ permalink raw reply	[flat|nested] 72+ messages in thread

* Re: Building Emacs overflowed pure space
  2006-07-18 20:16       ` Eli Zaretskii
@ 2006-07-18 20:56         ` Stefan Monnier
  2006-07-19  2:51           ` Eli Zaretskii
  2006-07-19 13:52           ` mituharu
  2006-07-18 22:01         ` Luc Teirlinck
  1 sibling, 2 replies; 72+ messages in thread
From: Stefan Monnier @ 2006-07-18 20:56 UTC (permalink / raw)
  Cc: mathias.dahl, Luc Teirlinck, rms, ralphm, emacs-devel

> I don't think it's reasonable at this stage.  We are not supposed to
> install changes that increase the pure size significantly; adding 10K
> will just risk wasting memory in the released Emacs.

I'm not sure what size other people get, but my `emacs' binary is never
smaller than 2MB (by a long shot), so even if one counts the worst case
20KB of wasted space (10KB * 2 (64bit growth factor)), it's still less than
1%.  Not much to worry about.  If we really care about that kind of memory,
we can do as XEmacs did: dump once to see how much pure space is necessary,
than adjust pure size and redump.

I use here a local hack which saves more pure space than that by simply
using a more compact representation for strings.  I haven't installed it
because I'm not convinced it's worth the trouble (it's more interesting on
64bit systems, tho).  See patch hunk below to get a feel for it ;-)


        Stefan


--- orig/src/lisp.h
+++ mod/src/lisp.h
@@ -709,14 +736,24 @@
 /* Set text properties.  */
 #define STRING_SET_INTERVALS(STR, INT) (XSTRING (STR)->intervals = (INT))
 
+/* If the string's size is smaller than the size of a pointer,
+   we store the data directly in Lisp_String, otherwise, we store it in
+   a separate object.  */
+#define STRING_MAXINLINE (sizeof (unsigned char *))
+
 /* In a string or vector, the sign bit of the `size' is the gc mark bit */
 
 struct Lisp_String
   {
     EMACS_INT size;
-    EMACS_INT size_byte;
+    EMACS_INT size_byte : BITS_PER_EMACS_INT - 1;
+    unsigned inlined : 1;	/* 0 -> ptr, 1 -> chars; in union below.  */
     INTERVAL intervals;		/* text properties in this string */
-    unsigned char *data;
+    union
+    { 
+      unsigned char *ptr;
+      unsigned char chars[STRING_MAXINLINE];
+    } data;
   };
 
 #ifdef offsetof

^ permalink raw reply	[flat|nested] 72+ messages in thread

* Re: Building Emacs overflowed pure space
  2006-07-18 20:16       ` Eli Zaretskii
  2006-07-18 20:56         ` Stefan Monnier
@ 2006-07-18 22:01         ` Luc Teirlinck
  2006-07-18 23:44           ` Chong Yidong
  2006-07-19  2:56           ` Eli Zaretskii
  1 sibling, 2 replies; 72+ messages in thread
From: Luc Teirlinck @ 2006-07-18 22:01 UTC (permalink / raw)
  Cc: mathias.dahl, rms, ralphm, emacs-devel

Eli Zaretskii wrote:

   No, this cannot explain the differences I was trying to investigate,
   because I made a point, as part of my testing, of building Emacs both
   with and without bootstrap.  I got the same numbers in both cases.

On the other hand, I guess that you read the following two messages in
which two people reported seeing a difference in pure-bytes-used
depending on the way they compiled Emacs?

   Date: Thu, 06 Jul 2006 20:01:25 +0900
   From: YAMAMOTO Mitsuharu <mituharu@math.s.chiba-u.ac.jp>
   Organization: Faculty of Science, Chiba University
   Cc: emacs-devel@gnu.org
   Sender: emacs-devel-bounces+teirllm=mail.auburn.edu@gnu.org

   > The question is not what caused the recent addition to the pure
   > space, the question is why some people see overflow, while others
   > don't, on the same platform.

   Maybe I'm not completely following the related threads, but at least
   one can see such a difference as follows:

   $ make bootstrap
   -> 1210372 pure bytes used

   $ cd lisp
   $ make bootstrap-clean
   $ make compile EMACS=../src/emacs
   $ cd ..
   $ make clean
   $ make
   -> 1209036 pure bytes used
   (on Mac OS X 10.4.7, X11)

   I observed that .elc's compiled by bootstrap-emacs have `dolist'
   expanded by cl.el, whereas those compiled by emacs have
   subr.el-version.  Actually, bootstrap-emacs loads cl.el via preloaded
   .el files that contain (eval-when-compile (require 'cl)), where
   `eval-when-compile' just behaves like `progn'.

					YAMAMOTO Mitsuharu
				   mituharu@math.s.chiba-u.ac.jp


   X-Injected-Via-Gmane: http://gmane.org/
   From: Evil Boris <evilborisnet@netscape.net>
   Date: Thu, 06 Jul 2006 07:40:47 -0400
   X-Gmane-NNTP-Posting-Host: 207-38-193-43.c3-0.wsd-ubr1.qens-wsd.ny.cable.rcn.com
   Cancel-Lock: sha1:oxVdsPt4RABMvFYyjLvvrf0IEG0=
   Sender: emacs-devel-bounces+teirllm=mail.auburn.edu@gnu.org


   For what it's worth, I have been using the following recipe (taken
   from INSTALL.CVS, is it still valid advice?) for updating Emacs.  After
	cvs update 
   I do:

   % gmake
   % cd lisp
   % gmake recompile EMACS=../src/emacs
   % cd ..
   % gmake

   I have noticed that the amount of pure storage needed in the initial
   call to gmake is different from the one needed after recompile.
   (sparc-sun-solaris2.7, X toolkit).  Is this to be expected?

   Specifically, the last build I tried (Jul 4) required 1210824 pure
   bytes in second iteration and 1208688 in the first (so the latest
   increase does not suffice).

   Cheers,
	   --Boris

^ permalink raw reply	[flat|nested] 72+ messages in thread

* Re: Building Emacs overflowed pure space
  2006-07-18 22:01         ` Luc Teirlinck
@ 2006-07-18 23:44           ` Chong Yidong
  2006-07-19  0:06             ` Luc Teirlinck
  2006-07-19  2:56           ` Eli Zaretskii
  1 sibling, 1 reply; 72+ messages in thread
From: Chong Yidong @ 2006-07-18 23:44 UTC (permalink / raw)


>    I observed that .elc's compiled by bootstrap-emacs have `dolist'
>    expanded by cl.el, whereas those compiled by emacs have
>    subr.el-version.  Actually, bootstrap-emacs loads cl.el via preloaded
>    .el files that contain (eval-when-compile (require 'cl)), where
>    `eval-when-compile' just behaves like `progn'.

Incidentally, why do we still have separate `dotimes' and `dolist'
functions in cl-macs.el?  Would we ever lose anything by using the
subr.el version?

^ permalink raw reply	[flat|nested] 72+ messages in thread

* Re: Building Emacs overflowed pure space
  2006-07-18 23:44           ` Chong Yidong
@ 2006-07-19  0:06             ` Luc Teirlinck
  2006-07-19  0:14               ` Luc Teirlinck
  2006-07-19  0:21               ` Luc Teirlinck
  0 siblings, 2 replies; 72+ messages in thread
From: Luc Teirlinck @ 2006-07-19  0:06 UTC (permalink / raw)
  Cc: emacs-devel

Ching Yidong wrote:

   Incidentally, why do we still have separate `dotimes' and `dolist'
   functions in cl-macs.el?  Would we ever lose anything by using the
   subr.el version?

I believe that it is to make the bodies of dotimes and dolist like a
`tagbody' and to surround dotimes and dolist with an implicit block
called `nil' to allow the CL function `return' to be used.  So one
would loose something by eliminating the cl version and using the subr
version.  It would break all code that loads cl and then uses tags or
`return' inside a dolist or dotimes.

See http://www.lisp.org/HyperSpec/Body/mac_dolist.html

Sincerely,

Luc.

^ permalink raw reply	[flat|nested] 72+ messages in thread

* Re: Building Emacs overflowed pure space
  2006-07-19  0:06             ` Luc Teirlinck
@ 2006-07-19  0:14               ` Luc Teirlinck
  2006-07-19  0:21               ` Luc Teirlinck
  1 sibling, 0 replies; 72+ messages in thread
From: Luc Teirlinck @ 2006-07-19  0:14 UTC (permalink / raw)
  Cc: emacs-devel

>From my previous reply:

   I believe that it is to make the bodies of dotimes and dolist like a
   `tagbody' and to surround dotimes and dolist with an implicit block
   called `nil' to allow the CL function `return' to be used.

After looking slightly closer at it, apparently only the latter: cl.el
apparently does not implement the CL function `tagbody', but it _does_
implement `return'.

   So one would loose something by eliminating the cl version and
   using the subr version.  It would break all code that loads cl and
   then uses tags or `return' inside a dolist or dotimes.

It would not break code using dolist or dotimes as a `tagbody',
because such code already does not work now, but it would break code
using the CL function `return', which is bad enough.

Sincerely,

Luc.

^ permalink raw reply	[flat|nested] 72+ messages in thread

* Re: Building Emacs overflowed pure space
  2006-07-19  0:06             ` Luc Teirlinck
  2006-07-19  0:14               ` Luc Teirlinck
@ 2006-07-19  0:21               ` Luc Teirlinck
  1 sibling, 0 replies; 72+ messages in thread
From: Luc Teirlinck @ 2006-07-19  0:21 UTC (permalink / raw)
  Cc: emacs-devel

In my previous reply, I left out an `m' in Chong's address.  So if you
reply, reply to this version of the message, or email is going to
bounce.  Sorry about that.  The below is just a literal repeat of my
previous reply:

>From my previous reply:

   I believe that it is to make the bodies of dotimes and dolist like a
   `tagbody' and to surround dotimes and dolist with an implicit block
   called `nil' to allow the CL function `return' to be used.

After looking slightly closer at it, apparently only the latter: cl.el
apparently does not implement the CL function `tagbody', but it _does_
implement `return'.

   So one would loose something by eliminating the cl version and
   using the subr version.  It would break all code that loads cl and
   then uses tags or `return' inside a dolist or dotimes.

It would not break code using dolist or dotimes as a `tagbody',
because such code already does not work now, but it would break code
using the CL function `return', which is bad enough.

Sincerely,

Luc.

^ permalink raw reply	[flat|nested] 72+ messages in thread

* Re: Building Emacs overflowed pure space
  2006-07-18  8:38 Building Emacs overflowed pure space Mathias Dahl
  2006-07-18  9:02 ` Ralph Moritz
@ 2006-07-19  0:44 ` Katsumi Yamaoka
  1 sibling, 0 replies; 72+ messages in thread
From: Katsumi Yamaoka @ 2006-07-19  0:44 UTC (permalink / raw)


Hi,

Also building Emacs failed with `make bootstrap' in the Fedora
Core 5 system.  Today I got:

  Dumping under names emacs and emacs-22.0.50.1
  emacs:0:Pure Lisp storage overflow (approx. 1211144 bytes needed)
  144 pure bytes used

Despite that it didn't happen with the Solaris 2.6 system:

  Dumping under names emacs and emacs-22.0.50.1
  1210896 pure bytes used

I don't use GTK.  The value of `system-configuration-options' is:

  "'--verbose' '--without-xim'"

Regards,

^ permalink raw reply	[flat|nested] 72+ messages in thread

* Re: Building Emacs overflowed pure space
  2006-07-18 20:56         ` Stefan Monnier
@ 2006-07-19  2:51           ` Eli Zaretskii
  2006-07-19  4:35             ` Stefan Monnier
  2006-07-19  4:42             ` Miles Bader
  2006-07-19 13:52           ` mituharu
  1 sibling, 2 replies; 72+ messages in thread
From: Eli Zaretskii @ 2006-07-19  2:51 UTC (permalink / raw)
  Cc: mathias.dahl, ralphm, emacs-devel

> Cc: Luc Teirlinck <teirllm@dms.auburn.edu>,  emacs-devel@gnu.org,  rms@gnu.org,
> 	  ralphm@members.fsf.org,  mathias.dahl@gmail.com
> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Date: Tue, 18 Jul 2006 16:56:36 -0400
> 
> I'm not sure what size other people get, but my `emacs' binary is never
> smaller than 2MB (by a long shot), so even if one counts the worst case
> 20KB of wasted space (10KB * 2 (64bit growth factor)), it's still less than
> 1%.  Not much to worry about.

We've been through this: memory waste is not measured in percents in
my book, it's measured in bytes.

^ permalink raw reply	[flat|nested] 72+ messages in thread

* Re: Building Emacs overflowed pure space
  2006-07-18 22:01         ` Luc Teirlinck
  2006-07-18 23:44           ` Chong Yidong
@ 2006-07-19  2:56           ` Eli Zaretskii
  1 sibling, 0 replies; 72+ messages in thread
From: Eli Zaretskii @ 2006-07-19  2:56 UTC (permalink / raw)
  Cc: mathias.dahl, rms, ralphm, emacs-devel

> Date: Tue, 18 Jul 2006 17:01:41 -0500 (CDT)
> From: Luc Teirlinck <teirllm@dms.auburn.edu>
> CC: emacs-devel@gnu.org, rms@gnu.org, ralphm@members.fsf.org,
>         mathias.dahl@gmail.com
> 
> Eli Zaretskii wrote:
> 
>    No, this cannot explain the differences I was trying to investigate,
>    because I made a point, as part of my testing, of building Emacs both
>    with and without bootstrap.  I got the same numbers in both cases.
> 
> On the other hand, I guess that you read the following two messages in
> which two people reported seeing a difference in pure-bytes-used
> depending on the way they compiled Emacs?

Yes, I've read it.  But note that even those messages talk about a
small discrepancy (1300 to 2200 bytes), whereas the differences I was
looking into were much larger.

^ permalink raw reply	[flat|nested] 72+ messages in thread

* Re: Building Emacs overflowed pure space
  2006-07-19  2:51           ` Eli Zaretskii
@ 2006-07-19  4:35             ` Stefan Monnier
  2006-07-19  4:42             ` Miles Bader
  1 sibling, 0 replies; 72+ messages in thread
From: Stefan Monnier @ 2006-07-19  4:35 UTC (permalink / raw)
  Cc: mathias.dahl, ralphm, emacs-devel

>> I'm not sure what size other people get, but my `emacs' binary is never
>> smaller than 2MB (by a long shot), so even if one counts the worst case
>> 20KB of wasted space (10KB * 2 (64bit growth factor)), it's still less than
>> 1%.  Not much to worry about.

> We've been through this: memory waste is not measured in percents in
> my book, it's measured in bytes.

You must have an earlier edition.


        Stefan

^ permalink raw reply	[flat|nested] 72+ messages in thread

* Re: Building Emacs overflowed pure space
  2006-07-19  2:51           ` Eli Zaretskii
  2006-07-19  4:35             ` Stefan Monnier
@ 2006-07-19  4:42             ` Miles Bader
  2006-07-19 18:13               ` Eli Zaretskii
  1 sibling, 1 reply; 72+ messages in thread
From: Miles Bader @ 2006-07-19  4:42 UTC (permalink / raw)
  Cc: emacs-devel, Stefan Monnier, ralphm, mathias.dahl

Eli Zaretskii <eliz@gnu.org> writes:
> We've been through this: memory waste is not measured in percents in
> my book, it's measured in bytes.

Er, just out of curiosity, how much memory do you (or your users) have...?

[My home machine is rather quaint by modern standards (450 Mhz Piii :-),
but still, it has 512 MB of RAM...]

-Miles
-- 
Come now, if we were really planning to harm you, would we be waiting here,
 beside the path, in the very darkest part of the forest?

^ permalink raw reply	[flat|nested] 72+ messages in thread

* Re: Building Emacs overflowed pure space
  2006-07-18 19:29     ` Luc Teirlinck
@ 2006-07-19  6:04       ` Richard Stallman
  2006-07-19  9:22         ` YAMAMOTO Mitsuharu
  0 siblings, 1 reply; 72+ messages in thread
From: Richard Stallman @ 2006-07-19  6:04 UTC (permalink / raw)
  Cc: mathias.dahl, ralphm, emacs-devel

    I should have been more precise here.  cl.el redefines some often used
    _macros_ which get inlined when compiled.  So this increases the pure
    space used in all compiled versions of files that use those macros.

I don't quite follow.  Why does the redefinition made by cl.el
alter the size of the expansion of these macro calls?

Can you give an example case where that occurs?

^ permalink raw reply	[flat|nested] 72+ messages in thread

* Re: Building Emacs overflowed pure space
  2006-07-19  6:04       ` Richard Stallman
@ 2006-07-19  9:22         ` YAMAMOTO Mitsuharu
  2006-07-19  9:46           ` YAMAMOTO Mitsuharu
  2006-07-19 21:15           ` Richard Stallman
  0 siblings, 2 replies; 72+ messages in thread
From: YAMAMOTO Mitsuharu @ 2006-07-19  9:22 UTC (permalink / raw)
  Cc: emacs-devel, Luc Teirlinck, ralphm, mathias.dahl

>>>>> On Wed, 19 Jul 2006 02:04:54 -0400, Richard Stallman <rms@gnu.org> said:

> I don't quite follow.  Why does the redefinition made by cl.el alter
> the size of the expansion of these macro calls?

The major difference seems to come from the length of the uninterned
symbol name that is allocated for each expanded `dolist'.

static POINTER_TYPE *
pure_alloc (size, type)
     size_t size;
     int type;
{
  POINTER_TYPE *result;
#ifdef USE_LSB_TAG
  size_t alignment = (1 << GCTYPEBITS);
#else
  size_t alignment = sizeof (EMACS_INT);

The length of the name of '--dolist-temp-- (in subr.el) is 15, and it
takes 2 units in 8(== (1 << GCTYPEBITS))-byte block including the
terminating nul char.  On the other hand, the name of
'--cl-dolist-temp-- takes 3 units.

Here's the result of some experiments:

  Dumped with dolist in subr.el: 1209808 bytes used
              dolist in cl.el:  >1211000 bytes used (overflow)
              dolist in cl.el with the modification
                '--cl-dolist-temp-- -> '--dolist-temp-- :
                                 1209832 bytes used

				     YAMAMOTO Mitsuharu
				mituharu@math.s.chiba-u.ac.jp

^ permalink raw reply	[flat|nested] 72+ messages in thread

* Re: Building Emacs overflowed pure space
  2006-07-19  9:22         ` YAMAMOTO Mitsuharu
@ 2006-07-19  9:46           ` YAMAMOTO Mitsuharu
  2006-07-19 15:05             ` Stefan Monnier
  2006-07-19 21:15           ` Richard Stallman
  1 sibling, 1 reply; 72+ messages in thread
From: YAMAMOTO Mitsuharu @ 2006-07-19  9:46 UTC (permalink / raw)
  Cc: mathias.dahl, Luc Teirlinck, ralphm, emacs-devel

>>>>> On Wed, 19 Jul 2006 18:22:44 +0900, YAMAMOTO Mitsuharu <mituharu@math.s.chiba-u.ac.jp> said:

>   Dumped with dolist in subr.el: 1209808 bytes used
>               dolist in cl.el:  >1211000 bytes used (overflow)
>               dolist in cl.el with the modification
>                 '--cl-dolist-temp-- -> '--dolist-temp-- :
>                                  1209832 bytes used

Sorry, I should have not used quote for uninterned symbols.  I meant
`dolist in cl-macs.el with the modification "--cl-dolist-temp--" ->
"--dolist-temp--"'.

BTW, I noticed some inefficiency of dolist in subr.el:

(defmacro dolist (spec &rest body)
  "Loop over a list.
Evaluate BODY with VAR bound to each car from LIST, in turn.
Then evaluate RESULT to get return value, default nil.

\(fn (VAR LIST [RESULT]) BODY...)"
  (declare (indent 1) (debug ((symbolp form &optional form) body)))
  (let ((temp (make-symbol "--dolist-temp--")))
    `(let ((,temp ,(nth 1 spec))
	   ,(car spec))
       (while ,temp
	 (setq ,(car spec) (car ,temp))
	 (setq ,temp (cdr ,temp))
	 ,@body)
       ,@(if (cdr (cdr spec))
	     `((setq ,(car spec) nil) ,@(cdr (cdr spec)))))))

In the above definition, `(setq ,temp (cdr ,temp))' is not placed at
the end of the loop body, so it disables the following optimization in
byte-opt.el:

	      ;; X: varref-Y    ...     varset-Y goto-X  -->
	      ;; X: varref-Y Z: ... dup varset-Y goto-Z
	      ;; (varset-X goto-BACK, BACK: varref-X --> copy the varref down.)
	      ;; (This is so usual for while loops that it is worth handling).

Is it OK to place `(setq ...)' after `,@body' as in the CL-version?

				     YAMAMOTO Mitsuharu
				mituharu@math.s.chiba-u.ac.jp

^ permalink raw reply	[flat|nested] 72+ messages in thread

* Re: Building Emacs overflowed pure space
  2006-07-18 20:56         ` Stefan Monnier
  2006-07-19  2:51           ` Eli Zaretskii
@ 2006-07-19 13:52           ` mituharu
  2006-07-19 15:07             ` Stefan Monnier
                               ` (2 more replies)
  1 sibling, 3 replies; 72+ messages in thread
From: mituharu @ 2006-07-19 13:52 UTC (permalink / raw)
  Cc: Luc Teirlinck, rms, ralphm, emacs-devel, Eli Zaretskii,
	mathias.dahl

>> I don't think it's reasonable at this stage.  We are not supposed to
>> install changes that increase the pure size significantly; adding 10K
>> will just risk wasting memory in the released Emacs.
>
> I'm not sure what size other people get, but my `emacs' binary is never
> smaller than 2MB (by a long shot), so even if one counts the worst case
> 20KB of wasted space (10KB * 2 (64bit growth factor)), it's still less
> than
> 1%.  Not much to worry about.  If we really care about that kind of
> memory,
> we can do as XEmacs did: dump once to see how much pure space is
> necessary,
> than adjust pure size and redump.
>
> I use here a local hack which saves more pure space than that by simply
> using a more compact representation for strings.  I haven't installed it
> because I'm not convinced it's worth the trouble (it's more interesting on
> 64bit systems, tho).  See patch hunk below to get a feel for it ;-)

Does it make sense to allocate Lisp objects (with alignment)
forward from the beginning and non-Lisp objects (without
alignment) backward from the end in the pure storage?  If the
following patch correct, pure-bytes-used decreases by ~70KB.

				     YAMAMOTO Mitsuharu
				mituharu@math.s.chiba-u.ac.jp
Index: src/alloc.c
===================================================================
RCS file: /cvsroot/emacs/emacs/src/alloc.c,v
retrieving revision 1.396
diff -c -p -r1.396 alloc.c
*** src/alloc.c	18 Jul 2006 13:25:40 -0000	1.396
--- src/alloc.c	19 Jul 2006 13:36:31 -0000
*************** static size_t pure_bytes_used_before_ove
*** 289,298 ****
        && ((PNTR_COMPARISON_TYPE) (P)				?
  	  >= (PNTR_COMPARISON_TYPE) purebeg))

! /* Index in pure at which next pure object will be allocated.. */

  EMACS_INT pure_bytes_used;

  /* If nonzero, this is a warning delivered by malloc and not yet
     displayed.  */

--- 289,306 ----
        && ((PNTR_COMPARISON_TYPE) (P)				?
  	  >= (PNTR_COMPARISON_TYPE) purebeg))

! /* Total number of bytes allocated in pure storage. */

  EMACS_INT pure_bytes_used;

+ /* Index in pure at which next pure Lisp object will be allocated.. */
+
+ static EMACS_INT pure_bytes_used_lisp;
+
+ /* Number of bytes allocated for non-Lisp objects in pure storage.  */
+
+ static EMACS_INT pure_bytes_used_non_lisp;
+
  /* If nonzero, this is a warning delivered by malloc and not yet
     displayed.  */

*************** pure_alloc (size, type)
*** 4720,4727 ****
  #endif

   again:
!   result = ALIGN (purebeg + pure_bytes_used, alignment);
!   pure_bytes_used = ((char *)result - (char *)purebeg) + size;

    if (pure_bytes_used <= pure_size)
      return result;
--- 4728,4746 ----
  #endif

   again:
!   if (type >= 0)
!     {
!       /* Lisp object.  */
!       result = ALIGN (purebeg + pure_bytes_used_lisp, alignment);
!       pure_bytes_used_lisp = ((char *)result - (char *)purebeg) + size;
!     }
!   else
!     {
!       /* Non-Lisp object.  */
!       pure_bytes_used_non_lisp += size;
!       result = purebeg + pure_size - pure_bytes_used_non_lisp;
!     }
!   pure_bytes_used = pure_bytes_used_lisp + pure_bytes_used_non_lisp;

    if (pure_bytes_used <= pure_size)
      return result;
*************** pure_alloc (size, type)
*** 4733,4738 ****
--- 4752,4758 ----
    pure_size = 10000;
    pure_bytes_used_before_overflow += pure_bytes_used - size;
    pure_bytes_used = 0;
+   pure_bytes_used_lisp = pure_bytes_used_non_lisp = 0;
    goto again;
  }

*************** init_alloc_once ()
*** 6225,6230 ****
--- 6245,6251 ----
    purebeg = PUREBEG;
    pure_size = PURESIZE;
    pure_bytes_used = 0;
+   pure_bytes_used_lisp = pure_bytes_used_non_lisp = 0;
    pure_bytes_used_before_overflow = 0;

    /* Initialize the list of free aligned blocks.  */

^ permalink raw reply	[flat|nested] 72+ messages in thread

* Re: Building Emacs overflowed pure space
  2006-07-19  9:46           ` YAMAMOTO Mitsuharu
@ 2006-07-19 15:05             ` Stefan Monnier
  2006-07-19 23:13               ` Kim F. Storm
                                 ` (3 more replies)
  0 siblings, 4 replies; 72+ messages in thread
From: Stefan Monnier @ 2006-07-19 15:05 UTC (permalink / raw)
  Cc: emacs-devel, Luc Teirlinck, rms, ralphm, mathias.dahl

>> Dumped with dolist in subr.el: 1209808 bytes used
>> dolist in cl.el:  >1211000 bytes used (overflow)
>> dolist in cl.el with the modification
>> '--cl-dolist-temp-- -> '--dolist-temp-- :
>> 1209832 bytes used

Interesting.  I'm wondering why you say "multiple of 8", since the pointer
to the data part of a string is not a Lisp_Object but a plain pointer
without any special alignment constraint AFAIK.

Also `make-symbol' allocates a fixed amount of memory AFAIK, independent
from the symbol name's length (it keeps a ref to the string argument,
instead, so that (eq X (symbol-name (make-symbol X))) is always true), and
the "--cl-dolist-temp--" string should only be allocated once (when reading
the macro definition).  So the only explanation that I can find is that the
uninterned symbol names get copied to purespace at some point (don't know
when or why), at which point the single shared string "--cl-dolist-temp--"
gets copied N times.

> (defmacro dolist (spec &rest body)
>   "Loop over a list.
> Evaluate BODY with VAR bound to each car from LIST, in turn.
> Then evaluate RESULT to get return value, default nil.

> \(fn (VAR LIST [RESULT]) BODY...)"
>   (declare (indent 1) (debug ((symbolp form &optional form) body)))
>   (let ((temp (make-symbol "--dolist-temp--")))
>     `(let ((,temp ,(nth 1 spec))
> 	   ,(car spec))
>        (while ,temp
> 	 (setq ,(car spec) (car ,temp))
> 	 (setq ,temp (cdr ,temp))
> 	 ,@body)
>        ,@(if (cdr (cdr spec))
> 	     `((setq ,(car spec) nil) ,@(cdr (cdr spec)))))))

> In the above definition, `(setq ,temp (cdr ,temp))' is not placed at
> the end of the loop body, so it disables the following optimization in
> byte-opt.el:

> 	      ;; X: varref-Y    ...     varset-Y goto-X  -->
> 	      ;; X: varref-Y Z: ... dup varset-Y goto-Z
> 	      ;; (varset-X goto-BACK, BACK: varref-X --> copy the varref down.)
> 	      ;; (This is so usual for while loops that it is worth handling).

> Is it OK to place `(setq ...)' after `,@body' as in the CL-version?

The setq can be placed after, but the cdr can't, in case the `body' does
a setcdr on it.  So maybe

   (defmacro dolist (spec &rest body)
     "Loop over a list.
   Evaluate BODY with VAR bound to each car from LIST, in turn.
   Then evaluate RESULT to get return value, default nil.
   
   \(fn (VAR LIST [RESULT]) BODY...)"
     (declare (indent 1) (debug ((symbolp form &optional form) body)))
     (let ((temp (make-symbol "--dolist-temp--")))
       `(let ((,temp ,(nth 1 spec))
              ,(car spec))
          (while ,temp
            (setq ,(car spec) (car ,temp))
            (setq ,temp (prog1 (cdr ,temp) ,@body)))
          ,@(if (cdr (cdr spec))
                `((setq ,(car spec) nil) ,@(cdr (cdr spec)))))))

will do the trick?


        Stefan

^ permalink raw reply	[flat|nested] 72+ messages in thread

* Re: Building Emacs overflowed pure space
  2006-07-19 13:52           ` mituharu
@ 2006-07-19 15:07             ` Stefan Monnier
  2006-07-19 21:16             ` Richard Stallman
  2006-07-22  9:06             ` YAMAMOTO Mitsuharu
  2 siblings, 0 replies; 72+ messages in thread
From: Stefan Monnier @ 2006-07-19 15:07 UTC (permalink / raw)
  Cc: Luc Teirlinck, rms, ralphm, emacs-devel, Eli Zaretskii,
	mathias.dahl

> Does it make sense to allocate Lisp objects (with alignment)
> forward from the beginning and non-Lisp objects (without
> alignment) backward from the end in the pure storage?  If the
> following patch correct, pure-bytes-used decreases by ~70KB.

Yes, I believe it makes perfect sense.


        Stefan

^ permalink raw reply	[flat|nested] 72+ messages in thread

* Re: Building Emacs overflowed pure space
  2006-07-19  4:42             ` Miles Bader
@ 2006-07-19 18:13               ` Eli Zaretskii
  0 siblings, 0 replies; 72+ messages in thread
From: Eli Zaretskii @ 2006-07-19 18:13 UTC (permalink / raw)
  Cc: emacs-devel, ralphm, mathias.dahl

> Cc: Stefan Monnier <monnier@iro.umontreal.ca>, mathias.dahl@gmail.com,
>         ralphm@members.fsf.org, emacs-devel@gnu.org
> From: Miles Bader <miles.bader@necel.com>
> Date: Wed, 19 Jul 2006 13:42:22 +0900
> 
> Er, just out of curiosity, how much memory do you (or your users) have...?

512MB, and it's all used up by programs that are running as long as
the machine is up.

^ permalink raw reply	[flat|nested] 72+ messages in thread

* Re: Building Emacs overflowed pure space
  2006-07-19  9:22         ` YAMAMOTO Mitsuharu
  2006-07-19  9:46           ` YAMAMOTO Mitsuharu
@ 2006-07-19 21:15           ` Richard Stallman
  2006-07-19 22:47             ` Kim F. Storm
                               ` (2 more replies)
  1 sibling, 3 replies; 72+ messages in thread
From: Richard Stallman @ 2006-07-19 21:15 UTC (permalink / raw)
  Cc: emacs-devel, teirllm, ralphm, mathias.dahl

I see no good reason why each expansion of `dolist' should use a
different symbol.  Maybe we can arrange to use one symbol over and
over again.  Does this work?



(defvar dolist-temp-var nil)

(defmacro dolist (spec &rest body)
  "Loop over a list.
Evaluate BODY with VAR bound to each car from LIST, in turn.
Then evaluate RESULT to get return value, default nil.

\(fn (VAR LIST [RESULT]) BODY...)"
  (declare (indent 1) (debug ((symbolp form &optional form) body)))
  (unless dolist-temp-var
    (setq dolist-temp-var (make-symbol "--dolist-temp--")))
  (let ((temp dolist-temp-var))
    `(let ((,temp ,(nth 1 spec))
	   ,(car spec))
       (while ,temp
	 (setq ,(car spec) (car ,temp))
	 (setq ,temp (cdr ,temp))
	 ,@body)
       ,@(if (cdr (cdr spec))
	     `((setq ,(car spec) nil) ,@(cdr (cdr spec)))))))

^ permalink raw reply	[flat|nested] 72+ messages in thread

* Re: Building Emacs overflowed pure space
  2006-07-19 13:52           ` mituharu
  2006-07-19 15:07             ` Stefan Monnier
@ 2006-07-19 21:16             ` Richard Stallman
  2006-07-20  1:12               ` YAMAMOTO Mitsuharu
  2006-07-22  9:06             ` YAMAMOTO Mitsuharu
  2 siblings, 1 reply; 72+ messages in thread
From: Richard Stallman @ 2006-07-19 21:16 UTC (permalink / raw)
  Cc: teirllm, ralphm, emacs-devel, monnier, eliz, mathias.dahl

      If the
    following patch correct, pure-bytes-used decreases by ~70KB.

Clever idea; please install it.

^ permalink raw reply	[flat|nested] 72+ messages in thread

* Re: Building Emacs overflowed pure space
  2006-07-19 21:15           ` Richard Stallman
@ 2006-07-19 22:47             ` Kim F. Storm
  2006-07-20 15:33               ` Richard Stallman
  2006-07-20  9:34             ` Kim F. Storm
  2006-07-20 16:03             ` Stefan Monnier
  2 siblings, 1 reply; 72+ messages in thread
From: Kim F. Storm @ 2006-07-19 22:47 UTC (permalink / raw)
  Cc: mathias.dahl, teirllm, ralphm, YAMAMOTO Mitsuharu, emacs-devel

Richard Stallman <rms@gnu.org> writes:

> I see no good reason why each expansion of `dolist' should use a
> different symbol.  Maybe we can arrange to use one symbol over and
> over again.  Does this work?

Does that really work with nested dolist calls?

-- 
Kim F. Storm <storm@cua.dk> http://www.cua.dk

^ permalink raw reply	[flat|nested] 72+ messages in thread

* Re: Building Emacs overflowed pure space
  2006-07-19 15:05             ` Stefan Monnier
@ 2006-07-19 23:13               ` Kim F. Storm
  2006-07-20  4:05                 ` Stefan Monnier
  2006-07-20  2:13               ` YAMAMOTO Mitsuharu
                                 ` (2 subsequent siblings)
  3 siblings, 1 reply; 72+ messages in thread
From: Kim F. Storm @ 2006-07-19 23:13 UTC (permalink / raw)
  Cc: Luc Teirlinck, rms, ralphm, emacs-devel, YAMAMOTO Mitsuharu,
	mathias.dahl

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> Is it OK to place `(setq ...)' after `,@body' as in the CL-version?
>
> The setq can be placed after, but the cdr can't, in case the `body' does
> a setcdr on it.

Really?
I thought the whole point of using make-symbol was to create an
anonymous loop-var to prevent BODY from accessing/modifying the list.

Besides, if the CL version is correct, why cannot the subr version of
dolist do the same?

-- 
Kim F. Storm <storm@cua.dk> http://www.cua.dk

^ permalink raw reply	[flat|nested] 72+ messages in thread

* Re: Building Emacs overflowed pure space
  2006-07-19 21:16             ` Richard Stallman
@ 2006-07-20  1:12               ` YAMAMOTO Mitsuharu
  0 siblings, 0 replies; 72+ messages in thread
From: YAMAMOTO Mitsuharu @ 2006-07-20  1:12 UTC (permalink / raw)
  Cc: teirllm, ralphm, emacs-devel, monnier, eliz, mathias.dahl

>>>>> On Wed, 19 Jul 2006 17:16:05 -0400, Richard Stallman <rms@gnu.org> said:

> Clever idea; please install it.

Done.  BASE_PURESIZE is now set to 1141000 (was 1211000).

				     YAMAMOTO Mitsuharu
				mituharu@math.s.chiba-u.ac.jp

^ permalink raw reply	[flat|nested] 72+ messages in thread

* Re: Building Emacs overflowed pure space
  2006-07-19 15:05             ` Stefan Monnier
  2006-07-19 23:13               ` Kim F. Storm
@ 2006-07-20  2:13               ` YAMAMOTO Mitsuharu
  2006-07-20  2:21               ` Richard Stallman
  2006-07-20  2:21               ` Richard Stallman
  3 siblings, 0 replies; 72+ messages in thread
From: YAMAMOTO Mitsuharu @ 2006-07-20  2:13 UTC (permalink / raw)
  Cc: emacs-devel, Luc Teirlinck, rms, ralphm, mathias.dahl

>>>>> On Wed, 19 Jul 2006 11:05:39 -0400, Stefan Monnier <monnier@iro.umontreal.ca> said:

> Also `make-symbol' allocates a fixed amount of memory AFAIK,
> independent from the symbol name's length (it keeps a ref to the
> string argument, instead, so that (eq X (symbol-name (make-symbol
> X))) is always true), and the "--cl-dolist-temp--" string should
> only be allocated once (when reading the macro definition).  So the
> only explanation that I can find is that the uninterned symbol names
> get copied to purespace at some point (don't know when or why), at
> which point the single shared string "--cl-dolist-temp--" gets
> copied N times.

It is allocated when .elc file is loaded.  Each compiled `dolist'
contains the uninterned symbol #:--cl-dolist-temp--, and its name is
allocated through the following path:

lread.c:read1
2780		  Lisp_Object result = uninterned_symbol ? make_symbol (read_buffer)
2781		    : intern (read_buffer);

lread.c:make_symbol
3294	  return Fmake_symbol ((!NILP (Vpurify_flag)
3295				? make_pure_string (str, len, len, 0)
3296				: make_string (str, len)));

alloc.c:make_pure_string
4787	  s = (struct Lisp_String *) pure_alloc (sizeof *s, Lisp_String);
4788	  s->data = (unsigned char *) pure_alloc (nbytes + 1, -1);

				     YAMAMOTO Mitsuharu
				mituharu@math.s.chiba-u.ac.jp

^ permalink raw reply	[flat|nested] 72+ messages in thread

* Re: Building Emacs overflowed pure space
  2006-07-19 15:05             ` Stefan Monnier
  2006-07-19 23:13               ` Kim F. Storm
  2006-07-20  2:13               ` YAMAMOTO Mitsuharu
@ 2006-07-20  2:21               ` Richard Stallman
  2006-07-20  2:21               ` Richard Stallman
  3 siblings, 0 replies; 72+ messages in thread
From: Richard Stallman @ 2006-07-20  2:21 UTC (permalink / raw)
  Cc: emacs-devel, teirllm, ralphm, mituharu, mathias.dahl

Symbols are not copied to pure space at all.
This is because their slots are not constant.
However, make_symbol copies uninterned symbol names to pure space.
Since the existing dolist macro generates a new symbol each time,
these symbols share nothing in the compiled output; they are different
uninterned symbols whose names happen to be the same.  So lread.c
copies the name to pure space each time.

^ permalink raw reply	[flat|nested] 72+ messages in thread

* Re: Building Emacs overflowed pure space
  2006-07-19 15:05             ` Stefan Monnier
                                 ` (2 preceding siblings ...)
  2006-07-20  2:21               ` Richard Stallman
@ 2006-07-20  2:21               ` Richard Stallman
  2006-07-20  4:07                 ` Stefan Monnier
  3 siblings, 1 reply; 72+ messages in thread
From: Richard Stallman @ 2006-07-20  2:21 UTC (permalink / raw)
  Cc: emacs-devel, teirllm, ralphm, mituharu, mathias.dahl

    The setq can be placed after, but the cdr can't, in case the `body' does
    a setcdr on it.

It is (cdr ,temp), referring to the uninterned symbol.
The body cannot refer to that uninterned symbol (that's the whole
point of using one).  So it is safe to take the cdr at the end.

^ permalink raw reply	[flat|nested] 72+ messages in thread

* Re: Building Emacs overflowed pure space
  2006-07-19 23:13               ` Kim F. Storm
@ 2006-07-20  4:05                 ` Stefan Monnier
  0 siblings, 0 replies; 72+ messages in thread
From: Stefan Monnier @ 2006-07-20  4:05 UTC (permalink / raw)
  Cc: Luc Teirlinck, rms, ralphm, emacs-devel, YAMAMOTO Mitsuharu,
	mathias.dahl

>>> Is it OK to place `(setq ...)' after `,@body' as in the CL-version?
>> The setq can be placed after, but the cdr can't, in case the `body' does
>> a setcdr on it.
> Really?
> I thought the whole point of using make-symbol was to create an
> anonymous loop-var to prevent BODY from accessing/modifying the list.

Indeed.  That's why the setq can be moved.  But the `cdr' is applied to
a cons-cell, not to the anonymous variable, and the cons-cell is
not anonymous.

> Besides, if the CL version is correct, why cannot the subr version of
> dolist do the same?

Then maybe it can.  I was just pointing out that it can change the behavior
in the case where the loop body changes the list destructively as it goes
through it.


        Stefan

^ permalink raw reply	[flat|nested] 72+ messages in thread

* Re: Building Emacs overflowed pure space
  2006-07-20  2:21               ` Richard Stallman
@ 2006-07-20  4:07                 ` Stefan Monnier
  2006-07-20  8:14                   ` Kim F. Storm
  0 siblings, 1 reply; 72+ messages in thread
From: Stefan Monnier @ 2006-07-20  4:07 UTC (permalink / raw)
  Cc: emacs-devel, teirllm, ralphm, mituharu, mathias.dahl

>     The setq can be placed after, but the cdr can't, in case the `body' does
>     a setcdr on it.

> It is (cdr ,temp), referring to the uninterned symbol.

(cdr ,temp) contains two operations: one is to look up the var to get its
content, and the other is to extract the cdr part of that content.  Only the
first part is protected by the fact that the symbol is uninterned.

> The body cannot refer to that uninterned symbol (that's the whole
> point of using one).

Indeed, but it may refer to the cons cell.

> So it is safe to take the cdr at the end.


Maybe it is, but it will subtly change the semantics in case the loop body
modifies the list it's loooping over.


        Stefan

^ permalink raw reply	[flat|nested] 72+ messages in thread

* Re: Building Emacs overflowed pure space
  2006-07-20  4:07                 ` Stefan Monnier
@ 2006-07-20  8:14                   ` Kim F. Storm
  2006-07-20 15:26                     ` Stefan Monnier
  0 siblings, 1 reply; 72+ messages in thread
From: Kim F. Storm @ 2006-07-20  8:14 UTC (permalink / raw)
  Cc: teirllm, rms, ralphm, emacs-devel, mituharu, mathias.dahl

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> So it is safe to take the cdr at the end.
>
> Maybe it is, but it will subtly change the semantics in case the loop body
> modifies the list it's loooping over.

But that subtle difference already exists between the cl and subr versions.
IMO, we better make them consistent.

As you point out, it is not safe to move the `cdr' if BODY modifies
the list.  But we could just document this fact in the docstring, e.g.:

dolist is a Lisp macro in `subr.el'.
(dolist (var list [result]) body...)

Loop over a list.
Evaluate BODY with VAR bound to each car from LIST, in turn.
Then evaluate RESULT to get return value, default nil.
The result is undefined if BODY modifies the list.

-- 
Kim F. Storm <storm@cua.dk> http://www.cua.dk

^ permalink raw reply	[flat|nested] 72+ messages in thread

* Re: Building Emacs overflowed pure space
  2006-07-19 21:15           ` Richard Stallman
  2006-07-19 22:47             ` Kim F. Storm
@ 2006-07-20  9:34             ` Kim F. Storm
  2006-07-20  9:58               ` David Kastrup
  2006-07-20 18:16               ` Richard Stallman
  2006-07-20 16:03             ` Stefan Monnier
  2 siblings, 2 replies; 72+ messages in thread
From: Kim F. Storm @ 2006-07-20  9:34 UTC (permalink / raw)
  Cc: mathias.dahl, teirllm, ralphm, YAMAMOTO Mitsuharu, emacs-devel

Richard Stallman <rms@gnu.org> writes:

> I see no good reason why each expansion of `dolist' should use a
> different symbol.  Maybe we can arrange to use one symbol over and
> over again.  Does this work?

I'm getting confused now (the summer heat here is draining).

Why is make-symbol used in dolist at all?  The benefit seems
infinitesimal to me, and in particular if you now suggest that we
should intern another variable just to hold the uninterned symbol.


Current version:

(defmacro dolist (spec &rest body)
  "Loop over a list.
Evaluate BODY with VAR bound to each car from LIST, in turn.
Then evaluate RESULT to get return value, default nil.

\(fn (VAR LIST [RESULT]) BODY...)"
  (declare (indent 1) (debug ((symbolp form &optional form) body)))
  (let ((temp (make-symbol "--dolist-temp--")))
    `(let ((,temp ,(nth 1 spec))
	   ,(car spec))
       (while ,temp
	 (setq ,(car spec) (car ,temp))
	 (setq ,temp (cdr ,temp))
	 ,@body)
       ,@(if (cdr (cdr spec))
	     `((setq ,(car spec) nil) ,@(cdr (cdr spec)))))))


Why isn't the following version just as good?

(defmacro dolist (spec &rest body)
  "Loop over a list.
Evaluate BODY with VAR bound to each car from LIST, in turn.
Then evaluate RESULT to get return value, default nil.

\(fn (VAR LIST [RESULT]) BODY...)"
  (declare (indent 1) (debug ((symbolp form &optional form) body)))
  `(let ((--dolist-temp-- ,(nth 1 spec))
	 ,(car spec))
     (while --dolist-temp--
       (setq ,(car spec) (car --dolist-temp--))
       (setq --dolist-temp-- (cdr --dolist-temp--))
       ,@body)
     ,@(if (cdr (cdr spec))
	   `((setq ,(car spec) nil) ,@(cdr (cdr spec))))))


>
>
>
> (defvar dolist-temp-var nil)
>
> (defmacro dolist (spec &rest body)
>   "Loop over a list.
> Evaluate BODY with VAR bound to each car from LIST, in turn.
> Then evaluate RESULT to get return value, default nil.
>
> \(fn (VAR LIST [RESULT]) BODY...)"
>   (declare (indent 1) (debug ((symbolp form &optional form) body)))
>   (unless dolist-temp-var
>     (setq dolist-temp-var (make-symbol "--dolist-temp--")))
>   (let ((temp dolist-temp-var))
>     `(let ((,temp ,(nth 1 spec))
> 	   ,(car spec))
>        (while ,temp
> 	 (setq ,(car spec) (car ,temp))
> 	 (setq ,temp (cdr ,temp))
> 	 ,@body)
>        ,@(if (cdr (cdr spec))
> 	     `((setq ,(car spec) nil) ,@(cdr (cdr spec)))))))

-- 
Kim F. Storm <storm@cua.dk> http://www.cua.dk

^ permalink raw reply	[flat|nested] 72+ messages in thread

* Re: Building Emacs overflowed pure space
  2006-07-20  9:34             ` Kim F. Storm
@ 2006-07-20  9:58               ` David Kastrup
  2006-07-20 11:35                 ` Kim F. Storm
  2006-07-20 18:16               ` Richard Stallman
  1 sibling, 1 reply; 72+ messages in thread
From: David Kastrup @ 2006-07-20  9:58 UTC (permalink / raw)
  Cc: teirllm, rms, ralphm, emacs-devel, YAMAMOTO Mitsuharu,
	mathias.dahl

storm@cua.dk (Kim F. Storm) writes:

> Richard Stallman <rms@gnu.org> writes:
>
>> I see no good reason why each expansion of `dolist' should use a
>> different symbol.  Maybe we can arrange to use one symbol over and
>> over again.  Does this work?
>
> I'm getting confused now (the summer heat here is draining).
>
> Why is make-symbol used in dolist at all?  The benefit seems
> infinitesimal to me, and in particular if you now suggest that we
> should intern another variable just to hold the uninterned symbol.
>
>
> Current version:
>
> (defmacro dolist (spec &rest body)
>   "Loop over a list.
> Evaluate BODY with VAR bound to each car from LIST, in turn.
> Then evaluate RESULT to get return value, default nil.
>
> \(fn (VAR LIST [RESULT]) BODY...)"
>   (declare (indent 1) (debug ((symbolp form &optional form) body)))
>   (let ((temp (make-symbol "--dolist-temp--")))
>     `(let ((,temp ,(nth 1 spec))
> 	   ,(car spec))
>        (while ,temp
> 	 (setq ,(car spec) (car ,temp))
> 	 (setq ,temp (cdr ,temp))
> 	 ,@body)
>        ,@(if (cdr (cdr spec))
> 	     `((setq ,(car spec) nil) ,@(cdr (cdr spec)))))))
>
>
> Why isn't the following version just as good?
>
> (defmacro dolist (spec &rest body)
>   "Loop over a list.
> Evaluate BODY with VAR bound to each car from LIST, in turn.
> Then evaluate RESULT to get return value, default nil.
>
> \(fn (VAR LIST [RESULT]) BODY...)"
>   (declare (indent 1) (debug ((symbolp form &optional form) body)))
>   `(let ((--dolist-temp-- ,(nth 1 spec))
> 	 ,(car spec))
>      (while --dolist-temp--
>        (setq ,(car spec) (car --dolist-temp--))
>        (setq --dolist-temp-- (cdr --dolist-temp--))
>        ,@body)
>      ,@(if (cdr (cdr spec))
> 	   `((setq ,(car spec) nil) ,@(cdr (cdr spec))))))

(dolist (i '(1 2))
  (dolist (j '(3 4))
    [...]))

>> (defvar dolist-temp-var nil)
>>
>> (defmacro dolist (spec &rest body)
>>   "Loop over a list.
>> Evaluate BODY with VAR bound to each car from LIST, in turn.
>> Then evaluate RESULT to get return value, default nil.
>>
>> \(fn (VAR LIST [RESULT]) BODY...)"
>>   (declare (indent 1) (debug ((symbolp form &optional form) body)))
>>   (unless dolist-temp-var
>>     (setq dolist-temp-var (make-symbol "--dolist-temp--")))
>>   (let ((temp dolist-temp-var))
>>     `(let ((,temp ,(nth 1 spec))
>> 	   ,(car spec))
>>        (while ,temp
>> 	 (setq ,(car spec) (car ,temp))
>> 	 (setq ,temp (cdr ,temp))
>> 	 ,@body)
>>        ,@(if (cdr (cdr spec))
>> 	     `((setq ,(car spec) nil) ,@(cdr (cdr spec)))))))

This version does not suffer the same problem since the macro is
exited before a nested dolist is _expanded_ in a similar way.

-- 
David Kastrup, Kriemhildstr. 15, 44793 Bochum

^ permalink raw reply	[flat|nested] 72+ messages in thread

* Re: Building Emacs overflowed pure space
  2006-07-20  9:58               ` David Kastrup
@ 2006-07-20 11:35                 ` Kim F. Storm
  2006-07-20 13:46                   ` David Kastrup
  0 siblings, 1 reply; 72+ messages in thread
From: Kim F. Storm @ 2006-07-20 11:35 UTC (permalink / raw)
  Cc: teirllm, rms, ralphm, emacs-devel, YAMAMOTO Mitsuharu,
	mathias.dahl

David Kastrup <dak@gnu.org> writes:

>
> (dolist (i '(1 2))
>   (dolist (j '(3 4))
>     [...]))

Huh?

(dolist (i '(1 2))
  (dolist (j '(3 4))
    (print (cons i j))))

=> (1 . 3) (1 . 4) (2 . 3) (2 . 4)


(defmacro dolist1 (spec &rest body)
  "Loop over a list.
Evaluate BODY with VAR bound to each car from LIST, in turn.
Then evaluate RESULT to get return value, default nil.

\(fn (VAR LIST [RESULT]) BODY...)"
  (declare (indent 1) (debug ((symbolp form &optional form) body)))
  `(let ((--dolist-temp-- ,(nth 1 spec))
	 ,(car spec))
     (while --dolist-temp--
       (setq ,(car spec) (car --dolist-temp--))
       (setq --dolist-temp-- (cdr --dolist-temp--))
       ,@body)
     ,@(if (cdr (cdr spec))
	   `((setq ,(car spec) nil) ,@(cdr (cdr spec))))))

(dolist1 (i '(1 2))
  (dolist1 (j '(3 4))
    (print (cons i j))))

=> (1 . 3) (1 . 4) (2 . 3) (2 . 4)

.. so where's the difference?

-- 
Kim F. Storm <storm@cua.dk> http://www.cua.dk

^ permalink raw reply	[flat|nested] 72+ messages in thread

* Re: Building Emacs overflowed pure space
  2006-07-20 11:35                 ` Kim F. Storm
@ 2006-07-20 13:46                   ` David Kastrup
  0 siblings, 0 replies; 72+ messages in thread
From: David Kastrup @ 2006-07-20 13:46 UTC (permalink / raw)
  Cc: teirllm, rms, ralphm, emacs-devel, YAMAMOTO Mitsuharu,
	mathias.dahl

storm@cua.dk (Kim F. Storm) writes:

> David Kastrup <dak@gnu.org> writes:
>
>>
>> (dolist (i '(1 2))
>>   (dolist (j '(3 4))
>>     [...]))
>
> Huh?
>
> (dolist (i '(1 2))
>   (dolist (j '(3 4))
>     (print (cons i j))))
>
> => (1 . 3) (1 . 4) (2 . 3) (2 . 4)
>
>
> (defmacro dolist1 (spec &rest body)
>   "Loop over a list.
> Evaluate BODY with VAR bound to each car from LIST, in turn.
> Then evaluate RESULT to get return value, default nil.
>
> \(fn (VAR LIST [RESULT]) BODY...)"
>   (declare (indent 1) (debug ((symbolp form &optional form) body)))
>   `(let ((--dolist-temp-- ,(nth 1 spec))
> 	 ,(car spec))
>      (while --dolist-temp--
>        (setq ,(car spec) (car --dolist-temp--))
>        (setq --dolist-temp-- (cdr --dolist-temp--))
>        ,@body)
>      ,@(if (cdr (cdr spec))
> 	   `((setq ,(car spec) nil) ,@(cdr (cdr spec))))))
>
> (dolist1 (i '(1 2))
>   (dolist1 (j '(3 4))
>     (print (cons i j))))
>
> => (1 . 3) (1 . 4) (2 . 3) (2 . 4)
>
> .. so where's the difference?

Uh yes.

-- 
David Kastrup, Kriemhildstr. 15, 44793 Bochum

^ permalink raw reply	[flat|nested] 72+ messages in thread

* Re: Building Emacs overflowed pure space
  2006-07-20  8:14                   ` Kim F. Storm
@ 2006-07-20 15:26                     ` Stefan Monnier
  2006-07-20 18:16                       ` Richard Stallman
  0 siblings, 1 reply; 72+ messages in thread
From: Stefan Monnier @ 2006-07-20 15:26 UTC (permalink / raw)
  Cc: teirllm, rms, ralphm, emacs-devel, mituharu, mathias.dahl

>>> So it is safe to take the cdr at the end.
>> 
>> Maybe it is, but it will subtly change the semantics in case the loop body
>> modifies the list it's loooping over.

> But that subtle difference already exists between the cl and subr versions.
> IMO, we better make them consistent.

> As you point out, it is not safe to move the `cdr' if BODY modifies
> the list.  But we could just document this fact in the docstring, e.g.:

> dolist is a Lisp macro in `subr.el'.
> (dolist (var list [result]) body...)

> Loop over a list.
> Evaluate BODY with VAR bound to each car from LIST, in turn.
> Then evaluate RESULT to get return value, default nil.
> The result is undefined if BODY modifies the list.

Sure.  But I currently see no reason to prefer the cl-macs.el behavior over
the subr.el one (AFAICT my suggested code is just as efficient as the
cl-macs.el one), so I'm not sure whether we should change cl-macs.el or
subr.el's behavior.


        Stefan

^ permalink raw reply	[flat|nested] 72+ messages in thread

* Re: Building Emacs overflowed pure space
  2006-07-19 22:47             ` Kim F. Storm
@ 2006-07-20 15:33               ` Richard Stallman
  0 siblings, 0 replies; 72+ messages in thread
From: Richard Stallman @ 2006-07-20 15:33 UTC (permalink / raw)
  Cc: mathias.dahl, teirllm, ralphm, mituharu, emacs-devel

    > I see no good reason why each expansion of `dolist' should use a
    > different symbol.  Maybe we can arrange to use one symbol over and
    > over again.  Does this work?

    Does that really work with nested dolist calls?

It ought to; each one will refer to its own binding.

It would work with an ordinary interned symbol, too.
The only reason to use an uninterned symbol is to avoid
interfering with any possible user program.

^ permalink raw reply	[flat|nested] 72+ messages in thread

* Re: Building Emacs overflowed pure space
  2006-07-19 21:15           ` Richard Stallman
  2006-07-19 22:47             ` Kim F. Storm
  2006-07-20  9:34             ` Kim F. Storm
@ 2006-07-20 16:03             ` Stefan Monnier
  2006-07-20 21:46               ` Richard Stallman
  2 siblings, 1 reply; 72+ messages in thread
From: Stefan Monnier @ 2006-07-20 16:03 UTC (permalink / raw)
  Cc: mathias.dahl, teirllm, ralphm, YAMAMOTO Mitsuharu, emacs-devel

> I see no good reason why each expansion of `dolist' should use a
> different symbol.  Maybe we can arrange to use one symbol over and
> over again.  Does this work?

I doubt it'll make any difference.  When reading the .elc file, the symbols
will get copied anyway for the reason that they're not interned.


        Stefan

^ permalink raw reply	[flat|nested] 72+ messages in thread

* Re: Building Emacs overflowed pure space
  2006-07-20  9:34             ` Kim F. Storm
  2006-07-20  9:58               ` David Kastrup
@ 2006-07-20 18:16               ` Richard Stallman
  2006-07-21  8:36                 ` Kim F. Storm
  1 sibling, 1 reply; 72+ messages in thread
From: Richard Stallman @ 2006-07-20 18:16 UTC (permalink / raw)
  Cc: mathias.dahl, teirllm, ralphm, mituharu, emacs-devel

    Why isn't the following version just as good?

    (defmacro dolist (spec &rest body)
      "Loop over a list.
    Evaluate BODY with VAR bound to each car from LIST, in turn.
    Then evaluate RESULT to get return value, default nil.

    \(fn (VAR LIST [RESULT]) BODY...)"
      (declare (indent 1) (debug ((symbolp form &optional form) body)))
      `(let ((--dolist-temp-- ,(nth 1 spec))
	     ,(car spec))
	 (while --dolist-temp--

The only flaw of this is that BODY can see the variable --dolist-temp--.
Practically speaking, it may not ever matter; users are unlikely to
use that variable by accident.  However, it is cleaner to prevent that
by using an uninterned symbol, assuming we can get rid of the current
inefficiency that that causes.

^ permalink raw reply	[flat|nested] 72+ messages in thread

* Re: Building Emacs overflowed pure space
  2006-07-20 15:26                     ` Stefan Monnier
@ 2006-07-20 18:16                       ` Richard Stallman
  0 siblings, 0 replies; 72+ messages in thread
From: Richard Stallman @ 2006-07-20 18:16 UTC (permalink / raw)
  Cc: teirllm, ralphm, emacs-devel, storm, mituharu, mathias.dahl

    Sure.  But I currently see no reason to prefer the cl-macs.el behavior over
    the subr.el one (AFAICT my suggested code is just as efficient as the
    cl-macs.el one), so I'm not sure whether we should change cl-macs.el or
    subr.el's behavior.

To modify the list that dolist is scanning is such an abuse that I am
not sure we need to specify what it should do.  But if I have
to express a preference between the two behaviors, I think that taking
the cdr at the end is a little more "correct", a little less surprising,
than taking the cdr before the body.

^ permalink raw reply	[flat|nested] 72+ messages in thread

* Re: Building Emacs overflowed pure space
  2006-07-20 16:03             ` Stefan Monnier
@ 2006-07-20 21:46               ` Richard Stallman
  2006-07-20 22:11                 ` Stefan Monnier
  0 siblings, 1 reply; 72+ messages in thread
From: Richard Stallman @ 2006-07-20 21:46 UTC (permalink / raw)
  Cc: mathias.dahl, teirllm, ralphm, mituharu, emacs-devel

    I doubt it'll make any difference.  When reading the .elc file, the symbols
    will get copied anyway for the reason that they're not interned.

If the table that arranges for all references to one symbol within
a function to come out as the same symbol
is preserved between functions, that ought to do the job.

However, this might be inconvenient to do.  We might want to set up a
new reader mechanism for a specified list of uninterned symbols.  That
would be a little work, but not hard at all.

^ permalink raw reply	[flat|nested] 72+ messages in thread

* Re: Building Emacs overflowed pure space
  2006-07-20 21:46               ` Richard Stallman
@ 2006-07-20 22:11                 ` Stefan Monnier
  2006-07-21  4:46                   ` Richard Stallman
  0 siblings, 1 reply; 72+ messages in thread
From: Stefan Monnier @ 2006-07-20 22:11 UTC (permalink / raw)
  Cc: mathias.dahl, teirllm, ralphm, mituharu, emacs-devel

>     I doubt it'll make any difference.  When reading the .elc file, the
>     symbols will get copied anyway for the reason that they're
>     not interned.

> If the table that arranges for all references to one symbol within
> a function to come out as the same symbol is preserved between functions,
> that ought to do the job.

> However, this might be inconvenient to do.

Indeed: it make conflate two different symbols into one and thus
re-introduce the error that the use of uninterned symbol is often meant
to solve.


        Stefan

^ permalink raw reply	[flat|nested] 72+ messages in thread

* Re: Building Emacs overflowed pure space
  2006-07-20 22:11                 ` Stefan Monnier
@ 2006-07-21  4:46                   ` Richard Stallman
  2006-07-21 12:57                     ` Stefan Monnier
  0 siblings, 1 reply; 72+ messages in thread
From: Richard Stallman @ 2006-07-21  4:46 UTC (permalink / raw)
  Cc: mathias.dahl, teirllm, ralphm, mituharu, emacs-devel

    > If the table that arranges for all references to one symbol within
    > a function to come out as the same symbol is preserved between functions,
    > that ought to do the job.

    > However, this might be inconvenient to do.

    Indeed: it make conflate two different symbols into one 

No it wouldn't.  With my change in the macro, compilation of a single
source file, indeed all compilation in a single Emacs session, would
use only one symbol for this.

The only missing unsolved part of this problem is how to arrange
the proper read syntax in the .elc file so that what's one symbol
at compile time will read back as one symbol.

^ permalink raw reply	[flat|nested] 72+ messages in thread

* Re: Building Emacs overflowed pure space
  2006-07-20 18:16               ` Richard Stallman
@ 2006-07-21  8:36                 ` Kim F. Storm
  2006-07-21 19:36                   ` Richard Stallman
  0 siblings, 1 reply; 72+ messages in thread
From: Kim F. Storm @ 2006-07-21  8:36 UTC (permalink / raw)
  Cc: emacs-devel, teirllm, mituharu, ralphm, mathias.dahl

Richard Stallman <rms@gnu.org> writes:

>     Why isn't the following version just as good?
>
>     (defmacro dolist (spec &rest body)
>       "Loop over a list.
>     Evaluate BODY with VAR bound to each car from LIST, in turn.
>     Then evaluate RESULT to get return value, default nil.
>
>     \(fn (VAR LIST [RESULT]) BODY...)"
>       (declare (indent 1) (debug ((symbolp form &optional form) body)))
>       `(let ((--dolist-temp-- ,(nth 1 spec))
> 	     ,(car spec))
> 	 (while --dolist-temp--
>
> The only flaw of this is that BODY can see the variable --dolist-temp--.
> Practically speaking, it may not ever matter; users are unlikely to
> use that variable by accident.  However, it is cleaner to prevent that
> by using an uninterned symbol, assuming we can get rid of the current
> inefficiency that that causes.

So what about using shorter symbol names than --dolist-temp--
--dotimes-temp--, --cl-dolist-temp-- etc.

And for the "defconst-tmp-var" symbol used in byte-compile-defvar.

E.g use "*" instead of the longer name.

That'll save a few KBytes.


But to follow your suggestion, why don't we just have _one_ generic
uninterned "tmp" symbol for all such uses.

E.g.

(defconst uninterned-tmp (make-symbol "tmp")
  "Uninterned symbol for use in macros.")

And then modify dolist (and friends) to:

(defmacro dolist (spec &rest body)
  "Loop over a list.
Evaluate BODY with VAR bound to each car from LIST, in turn.
Then evaluate RESULT to get return value, default nil.

\(fn (VAR LIST [RESULT]) BODY...)"
  (declare (indent 1) (debug ((symbolp form &optional form) body)))
  (let ((temp uninterned-tmp))
    `(let ((,temp ,(nth 1 spec))
	   ,(car spec))
       (while ,temp
	 (setq ,(car spec) (car ,temp))
	 (setq ,temp (cdr ,temp))
	 ,@body)
       ,@(if (cdr (cdr spec))
	     `((setq ,(car spec) nil) ,@(cdr (cdr spec)))))))


-- 
Kim F. Storm <storm@cua.dk> http://www.cua.dk

^ permalink raw reply	[flat|nested] 72+ messages in thread

* Re: Building Emacs overflowed pure space
  2006-07-21  4:46                   ` Richard Stallman
@ 2006-07-21 12:57                     ` Stefan Monnier
  2006-07-21 19:37                       ` Richard Stallman
  0 siblings, 1 reply; 72+ messages in thread
From: Stefan Monnier @ 2006-07-21 12:57 UTC (permalink / raw)
  Cc: mathias.dahl, teirllm, ralphm, mituharu, emacs-devel

>> If the table that arranges for all references to one symbol within
>> a function to come out as the same symbol is preserved between functions,
>> that ought to do the job.

>> However, this might be inconvenient to do.

>     Indeed: it make conflate two different symbols into one 

> No it wouldn't.  With my change in the macro, compilation of a single
> source file, indeed all compilation in a single Emacs session, would
> use only one symbol for this.

Well, that's only true if the change in the reader only applies to dolist.
If it applies more generally, it may conflate distinct symbols used in
*other* macros.


        Stefan

^ permalink raw reply	[flat|nested] 72+ messages in thread

* Re: Building Emacs overflowed pure space
  2006-07-21  8:36                 ` Kim F. Storm
@ 2006-07-21 19:36                   ` Richard Stallman
  2006-07-21 19:48                     ` David Kastrup
  0 siblings, 1 reply; 72+ messages in thread
From: Richard Stallman @ 2006-07-21 19:36 UTC (permalink / raw)
  Cc: emacs-devel, teirllm, mituharu, ralphm, mathias.dahl

    But to follow your suggestion, why don't we just have _one_ generic
    uninterned "tmp" symbol for all such uses.

We could do that.  It would work.

It would be a little better to have a list of ten of them and pick at
random.  That would facilitate examining values in the Lisp debugger
on those occasions when you really want to.

^ permalink raw reply	[flat|nested] 72+ messages in thread

* Re: Building Emacs overflowed pure space
  2006-07-21 12:57                     ` Stefan Monnier
@ 2006-07-21 19:37                       ` Richard Stallman
  0 siblings, 0 replies; 72+ messages in thread
From: Richard Stallman @ 2006-07-21 19:37 UTC (permalink / raw)
  Cc: mathias.dahl, teirllm, ralphm, mituharu, emacs-devel

    > No it wouldn't.  With my change in the macro, compilation of a single
    > source file, indeed all compilation in a single Emacs session, would
    > use only one symbol for this.

    Well, that's only true if the change in the reader only applies to dolist.
    If it applies more generally, it may conflate distinct symbols used in
    *other* macros.

Perhaps we are thinking of different kinds of change.
What I have in mind would only avoid splitting of one symbol
(as inserted in macro expansions) into multiple symbols
in the process of writing the compiled code into a file and reading it again.
That can't do any harm.

^ permalink raw reply	[flat|nested] 72+ messages in thread

* Re: Building Emacs overflowed pure space
  2006-07-21 19:36                   ` Richard Stallman
@ 2006-07-21 19:48                     ` David Kastrup
  2006-07-21 20:40                       ` Andreas Schwab
  2006-07-22 15:49                       ` Richard Stallman
  0 siblings, 2 replies; 72+ messages in thread
From: David Kastrup @ 2006-07-21 19:48 UTC (permalink / raw)
  Cc: teirllm, ralphm, emacs-devel, Kim F. Storm, mituharu,
	mathias.dahl

Richard Stallman <rms@gnu.org> writes:

>     But to follow your suggestion, why don't we just have _one_ generic
>     uninterned "tmp" symbol for all such uses.
>
> We could do that.  It would work.
>
> It would be a little better to have a list of ten of them and pick at
> random.  That would facilitate examining values in the Lisp debugger
> on those occasions when you really want to.

I don't see how.  They are uninterned, after all.  And "picking at
random" means that it becomes unpredictable what loop combinations
will happen to be debuggable, and what combinations will not.  And it
will change from compilation to compilation.  I don't like that at
all.

Frankly speaking: I don't know what we are supposed to buy ourselves
if a loop body can access its internal variable by (symbol-value
tmp-loop-var) instead of just tmp-loop-var.

It seems like a pointless exercise in obfuscation.  It does not buy us
any encapsulation, and makes debugging and understanding the code
harder when disassembling it.

-- 
David Kastrup, Kriemhildstr. 15, 44793 Bochum

^ permalink raw reply	[flat|nested] 72+ messages in thread

* Re: Building Emacs overflowed pure space
  2006-07-21 19:48                     ` David Kastrup
@ 2006-07-21 20:40                       ` Andreas Schwab
  2006-07-21 20:49                         ` David Kastrup
  2006-07-22 15:49                       ` Richard Stallman
  1 sibling, 1 reply; 72+ messages in thread
From: Andreas Schwab @ 2006-07-21 20:40 UTC (permalink / raw)
  Cc: teirllm, rms, ralphm, emacs-devel, Kim F. Storm, mituharu,
	mathias.dahl

David Kastrup <dak@gnu.org> writes:

> Frankly speaking: I don't know what we are supposed to buy ourselves
> if a loop body can access its internal variable by (symbol-value
> tmp-loop-var) instead of just tmp-loop-var.

If we use an uninterned symbol the body can't access it at all.

Andreas.

-- 
Andreas Schwab, SuSE Labs, schwab@suse.de
SuSE Linux Products GmbH, Maxfeldstraße 5, 90409 Nürnberg, Germany
PGP key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

^ permalink raw reply	[flat|nested] 72+ messages in thread

* Re: Building Emacs overflowed pure space
  2006-07-21 20:40                       ` Andreas Schwab
@ 2006-07-21 20:49                         ` David Kastrup
  0 siblings, 0 replies; 72+ messages in thread
From: David Kastrup @ 2006-07-21 20:49 UTC (permalink / raw)
  Cc: teirllm, rms, ralphm, emacs-devel, Kim F. Storm, mituharu,
	mathias.dahl

Andreas Schwab <schwab@suse.de> writes:

> David Kastrup <dak@gnu.org> writes:
>
>> Frankly speaking: I don't know what we are supposed to buy ourselves
>> if a loop body can access its internal variable by (symbol-value
>> tmp-loop-var) instead of just tmp-loop-var.
>
> If we use an uninterned symbol the body can't access it at all.

The proposal was to use the same uninterned symbol for all loops and
keep it in a global variable like tmp-loop-var.  And that means, of
course, that everybody able to access that global variable can also
use it to access the loop variable.  You don't need to go through
obarray, the symbol _is_ already available in tmp-loop-var without
lookup.

-- 
David Kastrup, Kriemhildstr. 15, 44793 Bochum

^ permalink raw reply	[flat|nested] 72+ messages in thread

* Re: Building Emacs overflowed pure space
  2006-07-19 13:52           ` mituharu
  2006-07-19 15:07             ` Stefan Monnier
  2006-07-19 21:16             ` Richard Stallman
@ 2006-07-22  9:06             ` YAMAMOTO Mitsuharu
  2006-07-22 16:03               ` Stefan Monnier
  2006-07-23  5:26               ` Richard Stallman
  2 siblings, 2 replies; 72+ messages in thread
From: YAMAMOTO Mitsuharu @ 2006-07-22  9:06 UTC (permalink / raw)
  Cc: Luc Teirlinck, rms, ralphm, emacs-devel, Eli Zaretskii,
	mathias.dahl

>>>>> On Wed, 19 Jul 2006 22:52:15 +0900 (JST), mituharu@math.s.chiba-u.ac.jp said:

> Does it make sense to allocate Lisp objects (with alignment) forward
> from the beginning and non-Lisp objects (without alignment) backward
> from the end in the pure storage?  If the following patch correct,
> pure-bytes-used decreases by ~70KB.

If the string data in the pure storage can be assumed to be completely
read-only (including the preloading stage), another hack could be
considered.  That is, sharing postfixes of string data among multiple
pure strings.  The following experimental patch shows another ~40KB
reduction of the pure storage usage.  (A slow and naive but reliable
version not using Boyer-Moore also shows the same size of reduction.)

				     YAMAMOTO Mitsuharu
				mituharu@math.s.chiba-u.ac.jp

Index: src/alloc.c
===================================================================
RCS file: /cvsroot/emacs/emacs/src/alloc.c,v
retrieving revision 1.397
diff -c -p -r1.397 alloc.c
*** src/alloc.c	20 Jul 2006 01:01:04 -0000	1.397
--- src/alloc.c	22 Jul 2006 08:28:47 -0000
*************** check_pure_size ()
*** 4767,4772 ****
--- 4767,4837 ----
  }
  
  
+ /* Find the byte sequence {DATA[0], ..., DATA[NBYTES-1], '\0'} from
+    the non-Lisp data pool of the pure storage, and return its start
+    address.  Return NULL if not found.  */
+ 
+ static char *
+ find_string_data_in_pure (data, nbytes)
+      unsigned char *data;
+      int nbytes;
+ {
+   int i, skip, bm_skip[256], last_char_skip, infinity, start, start_max;
+   unsigned char *p;
+   char *non_lisp_head;
+ 
+   /* Set up the Boyer-Moore table. */
+   skip = nbytes + 1;
+   for (i = 0; i < 256; i++)
+     bm_skip[i] = skip;
+ 
+   p = data;
+   while (--skip > 0)
+     bm_skip[*p++] = skip;
+ 
+   last_char_skip = bm_skip['\0'];
+ 
+   non_lisp_head = purebeg + pure_size - pure_bytes_used_non_lisp;
+   start_max = pure_bytes_used_non_lisp - (nbytes + 1);
+ 
+   /* See the comments in the function `boyer_moore' (search.c) for the
+      use of `infinity'.  */
+   infinity = pure_bytes_used_non_lisp + 1;
+   bm_skip['\0'] = infinity;
+ 
+   p = non_lisp_head + nbytes;
+   start = 0;
+   do
+     {
+       /* Check the last character (== '\0'). */
+       do
+ 	{
+ 	  start += bm_skip[*(p + start)];
+ 	}
+       while (start <= start_max);
+ 
+       if (start < infinity)
+ 	/* Couldn't find the last character.  */
+ 	return NULL;
+ 
+       /* No less than `infinity' means we could find the last
+ 	 character at `s[start - infinity]'.  */
+       start -= infinity;
+ 
+       /* Check the remaining characters.  */
+       if (memcmp (data, non_lisp_head + start, nbytes) == 0)
+ 	/* Found.  */
+ 	return non_lisp_head + start;
+ 
+       /* Not found. */
+       start += last_char_skip;
+     }
+   while (start <= start_max);
+ 
+   return NULL;
+ }
+ 
+ 
  /* Return a string allocated in pure space.  DATA is a buffer holding
     NCHARS characters, and NBYTES bytes of string data.  MULTIBYTE
     non-zero means make the result string multibyte.
*************** make_pure_string (data, nchars, nbytes, 
*** 4785,4795 ****
    struct Lisp_String *s;
  
    s = (struct Lisp_String *) pure_alloc (sizeof *s, Lisp_String);
!   s->data = (unsigned char *) pure_alloc (nbytes + 1, -1);
    s->size = nchars;
    s->size_byte = multibyte ? nbytes : -1;
-   bcopy (data, s->data, nbytes);
-   s->data[nbytes] = '\0';
    s->intervals = NULL_INTERVAL;
    XSETSTRING (string, s);
    return string;
--- 4850,4864 ----
    struct Lisp_String *s;
  
    s = (struct Lisp_String *) pure_alloc (sizeof *s, Lisp_String);
!   s->data = find_string_data_in_pure (data, nbytes);
!   if (s->data == NULL)
!     {
!       s->data = (unsigned char *) pure_alloc (nbytes + 1, -1);
!       bcopy (data, s->data, nbytes);
!       s->data[nbytes] = '\0';
!     }
    s->size = nchars;
    s->size_byte = multibyte ? nbytes : -1;
    s->intervals = NULL_INTERVAL;
    XSETSTRING (string, s);
    return string;

^ permalink raw reply	[flat|nested] 72+ messages in thread

* Re: Building Emacs overflowed pure space
  2006-07-21 19:48                     ` David Kastrup
  2006-07-21 20:40                       ` Andreas Schwab
@ 2006-07-22 15:49                       ` Richard Stallman
  1 sibling, 0 replies; 72+ messages in thread
From: Richard Stallman @ 2006-07-22 15:49 UTC (permalink / raw)
  Cc: teirllm, ralphm, emacs-devel, storm, mituharu, mathias.dahl

    > It would be a little better to have a list of ten of them and pick at
    > random.  That would facilitate examining values in the Lisp debugger
    > on those occasions when you really want to.

    I don't see how.  They are uninterned, after all.

You can use symbol-value to examine an uninterned symbol.
If it uses names like `uninterned-6', and you know to find that
as (nth 6 uninterned-vars-list), you can examine it.  But this is
not something user programs are likely to do by accident.

      And "picking at
    random" means that it becomes unpredictable what loop combinations
    will happen to be debuggable, and what combinations will not.

It is not as good as if we had a feature in the debugger to do this
job 100%.  Ideal would be having a feature to examine a variable in
the debugger's selected stack frame.  However, that would be a lot of
work.  So I proposed something that would give a little bit more
debuggability in a pinch, while being very easy to do.

    It seems like a pointless exercise in obfuscation.  It does not buy us
    any encapsulation, and makes debugging and understanding the code
    harder when disassembling it.

Maybe you are right.  Lots of other internal values are exposed inside
Emacs in ways that are not quite clean; two more for dolist and
dotimes would not be much risk.

Does anyone argue against?

^ permalink raw reply	[flat|nested] 72+ messages in thread

* Re: Building Emacs overflowed pure space
  2006-07-22  9:06             ` YAMAMOTO Mitsuharu
@ 2006-07-22 16:03               ` Stefan Monnier
  2006-07-22 23:11                 ` Kim F. Storm
  2006-07-23  5:26               ` Richard Stallman
  1 sibling, 1 reply; 72+ messages in thread
From: Stefan Monnier @ 2006-07-22 16:03 UTC (permalink / raw)
  Cc: Luc Teirlinck, rms, ralphm, emacs-devel, Eli Zaretskii,
	mathias.dahl

> If the string data in the pure storage can be assumed to be completely
> read-only (including the preloading stage), another hack could be
> considered.  That is, sharing postfixes of string data among multiple
> pure strings.  The following experimental patch shows another ~40KB
> reduction of the pure storage usage.  (A slow and naive but reliable
> version not using Boyer-Moore also shows the same size of reduction.)

Cute,
And it does remove the --cl-dolist-temp-- vs --dolist-temp-- difference.


        Stefan

^ permalink raw reply	[flat|nested] 72+ messages in thread

* Re: Building Emacs overflowed pure space
  2006-07-22 16:03               ` Stefan Monnier
@ 2006-07-22 23:11                 ` Kim F. Storm
  2006-07-23  3:47                   ` Stefan Monnier
  2006-07-25  1:14                   ` YAMAMOTO Mitsuharu
  0 siblings, 2 replies; 72+ messages in thread
From: Kim F. Storm @ 2006-07-22 23:11 UTC (permalink / raw)
  Cc: Luc Teirlinck, rms, ralphm, emacs-devel, Eli Zaretskii,
	YAMAMOTO Mitsuharu, mathias.dahl

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> If the string data in the pure storage can be assumed to be completely
>> read-only (including the preloading stage), another hack could be
>> considered.  That is, sharing postfixes of string data among multiple
>> pure strings.  The following experimental patch shows another ~40KB
>> reduction of the pure storage usage.  (A slow and naive but reliable
>> version not using Boyer-Moore also shows the same size of reduction.)
>
I tried the exact the same thing (using the naive version), but the
dumped emacs crashed immediately after loadup, thinking that "nil" was
a filename...  So I didn't think it was safe to do this.


> Cute,
> And it does remove the --cl-dolist-temp-- vs --dolist-temp-- difference.

How?  --do... != c-do...  !?


-- 
Kim F. Storm <storm@cua.dk> http://www.cua.dk

^ permalink raw reply	[flat|nested] 72+ messages in thread

* Re: Building Emacs overflowed pure space
  2006-07-22 23:11                 ` Kim F. Storm
@ 2006-07-23  3:47                   ` Stefan Monnier
  2006-07-25  1:14                   ` YAMAMOTO Mitsuharu
  1 sibling, 0 replies; 72+ messages in thread
From: Stefan Monnier @ 2006-07-23  3:47 UTC (permalink / raw)
  Cc: Luc Teirlinck, rms, ralphm, emacs-devel, Eli Zaretskii,
	YAMAMOTO Mitsuharu, mathias.dahl

>>> If the string data in the pure storage can be assumed to be completely
>>> read-only (including the preloading stage), another hack could be
>>> considered.  That is, sharing postfixes of string data among multiple
>>> pure strings.  The following experimental patch shows another ~40KB
>>> reduction of the pure storage usage.  (A slow and naive but reliable
>>> version not using Boyer-Moore also shows the same size of reduction.)
> I tried the exact the same thing (using the naive version), but the
> dumped emacs crashed immediately after loadup, thinking that "nil" was
> a filename...  So I didn't think it was safe to do this.

There may be one or two unexpected details to work out, but I think the
approach is fundamentally pretty safe.

>> Cute,
>> And it does remove the --cl-dolist-temp-- vs --dolist-temp-- difference.

> How?  --do... != c-do...  !?

One of the problems mentioned in this thread is that not only each uninterned
symbol used by dolist is a separate object, but that the symbol's name
(i.e. the string "--cl-dolist-temp--") is also duplicated when reading the
.elc file, which is why the length of the symbol's name makes a difference
in the amount of pure storage used.  If all strings are basically
hash-consed, then this problem disappears.


        Stefan

^ permalink raw reply	[flat|nested] 72+ messages in thread

* Re: Building Emacs overflowed pure space
  2006-07-22  9:06             ` YAMAMOTO Mitsuharu
  2006-07-22 16:03               ` Stefan Monnier
@ 2006-07-23  5:26               ` Richard Stallman
  1 sibling, 0 replies; 72+ messages in thread
From: Richard Stallman @ 2006-07-23  5:26 UTC (permalink / raw)
  Cc: teirllm, ralphm, emacs-devel, monnier, eliz, mathias.dahl

Clever.  Please install this too, if nobody objects.

^ permalink raw reply	[flat|nested] 72+ messages in thread

* Re: Building Emacs overflowed pure space
  2006-07-22 23:11                 ` Kim F. Storm
  2006-07-23  3:47                   ` Stefan Monnier
@ 2006-07-25  1:14                   ` YAMAMOTO Mitsuharu
  2006-07-25  4:13                     ` Richard Stallman
  1 sibling, 1 reply; 72+ messages in thread
From: YAMAMOTO Mitsuharu @ 2006-07-25  1:14 UTC (permalink / raw)
  Cc: Luc Teirlinck, rms, ralphm, emacs-devel, Stefan Monnier,
	Eli Zaretskii, mathias.dahl

>>>>> On Sun, 23 Jul 2006 01:11:48 +0200, storm@cua.dk (Kim F. Storm) said:

>>> If the string data in the pure storage can be assumed to be
>>> completely read-only (including the preloading stage), another
>>> hack could be considered.  That is, sharing postfixes of string
>>> data among multiple pure strings.  The following experimental
>>> patch shows another ~40KB reduction of the pure storage usage.  (A
>>> slow and naive but reliable version not using Boyer-Moore also
>>> shows the same size of reduction.)
>> 
> I tried the exact the same thing (using the naive version), but the
> dumped emacs crashed immediately after loadup, thinking that "nil"
> was a filename...  So I didn't think it was safe to do this.

I'm not sure if this should be counted as an objection.  If so, can I
have a look at the code just to make sure?

BTW, a condition was missing at the entrance of the function in my
previous patch.  Below is a revised one:

				     YAMAMOTO Mitsuharu
				mituharu@math.s.chiba-u.ac.jp

Index: src/alloc.c
===================================================================
RCS file: /cvsroot/emacs/emacs/src/alloc.c,v
retrieving revision 1.397
diff -c -p -r1.397 alloc.c
*** src/alloc.c	20 Jul 2006 01:01:04 -0000	1.397
--- src/alloc.c	23 Jul 2006 04:21:49 -0000
*************** check_pure_size ()
*** 4767,4772 ****
--- 4767,4839 ----
  }
  
  
+ /* Find the byte sequence {DATA[0], ..., DATA[NBYTES-1], '\0'} from
+    the non-Lisp data pool of the pure storage, and return its start
+    address.  Return NULL if not found.  */
+ 
+ static char *
+ find_string_data_in_pure (data, nbytes)
+      char *data;
+      int nbytes;
+ {
+   int i, skip, bm_skip[256], last_char_skip, infinity, start, start_max;
+   unsigned char *p;
+   char *non_lisp_beg;
+ 
+   if (pure_bytes_used_non_lisp < nbytes + 1)
+     return NULL;
+ 
+   /* Set up the Boyer-Moore table.  */
+   skip = nbytes + 1;
+   for (i = 0; i < 256; i++)
+     bm_skip[i] = skip;
+ 
+   p = (unsigned char *) data;
+   while (--skip > 0)
+     bm_skip[*p++] = skip;
+ 
+   last_char_skip = bm_skip['\0'];
+ 
+   non_lisp_beg = purebeg + pure_size - pure_bytes_used_non_lisp;
+   start_max = pure_bytes_used_non_lisp - (nbytes + 1);
+ 
+   /* See the comments in the function `boyer_moore' (search.c) for the
+      use of `infinity'.  */
+   infinity = pure_bytes_used_non_lisp + 1;
+   bm_skip['\0'] = infinity;
+ 
+   p = (unsigned char *) non_lisp_beg + nbytes;
+   start = 0;
+   do
+     {
+       /* Check the last character (== '\0').  */
+       do
+ 	{
+ 	  start += bm_skip[*(p + start)];
+ 	}
+       while (start <= start_max);
+ 
+       if (start < infinity)
+ 	/* Couldn't find the last character.  */
+ 	return NULL;
+ 
+       /* No less than `infinity' means we could find the last
+ 	 character at `p[start - infinity]'.  */
+       start -= infinity;
+ 
+       /* Check the remaining characters.  */
+       if (memcmp (data, non_lisp_beg + start, nbytes) == 0)
+ 	/* Found.  */
+ 	return non_lisp_beg + start;
+ 
+       start += last_char_skip;
+     }
+   while (start <= start_max);
+ 
+   return NULL;
+ }
+ 
+ 
  /* Return a string allocated in pure space.  DATA is a buffer holding
     NCHARS characters, and NBYTES bytes of string data.  MULTIBYTE
     non-zero means make the result string multibyte.
*************** make_pure_string (data, nchars, nbytes, 
*** 4785,4795 ****
    struct Lisp_String *s;
  
    s = (struct Lisp_String *) pure_alloc (sizeof *s, Lisp_String);
!   s->data = (unsigned char *) pure_alloc (nbytes + 1, -1);
    s->size = nchars;
    s->size_byte = multibyte ? nbytes : -1;
-   bcopy (data, s->data, nbytes);
-   s->data[nbytes] = '\0';
    s->intervals = NULL_INTERVAL;
    XSETSTRING (string, s);
    return string;
--- 4852,4866 ----
    struct Lisp_String *s;
  
    s = (struct Lisp_String *) pure_alloc (sizeof *s, Lisp_String);
!   s->data = find_string_data_in_pure (data, nbytes);
!   if (s->data == NULL)
!     {
!       s->data = (unsigned char *) pure_alloc (nbytes + 1, -1);
!       bcopy (data, s->data, nbytes);
!       s->data[nbytes] = '\0';
!     }
    s->size = nchars;
    s->size_byte = multibyte ? nbytes : -1;
    s->intervals = NULL_INTERVAL;
    XSETSTRING (string, s);
    return string;

^ permalink raw reply	[flat|nested] 72+ messages in thread

* Re: Building Emacs overflowed pure space
  2006-07-25  1:14                   ` YAMAMOTO Mitsuharu
@ 2006-07-25  4:13                     ` Richard Stallman
  2006-07-25 10:41                       ` YAMAMOTO Mitsuharu
  0 siblings, 1 reply; 72+ messages in thread
From: Richard Stallman @ 2006-07-25  4:13 UTC (permalink / raw)
  Cc: teirllm, ralphm, emacs-devel, monnier, storm, eliz, mathias.dahl

    > I tried the exact the same thing (using the naive version), but the
    > dumped emacs crashed immediately after loadup, thinking that "nil"
    > was a filename...  So I didn't think it was safe to do this.

    I'm not sure if this should be counted as an objection.

The code he wrote must have a bug in it.  If your code works,
I guess it does not have a similar bug.

^ permalink raw reply	[flat|nested] 72+ messages in thread

* Re: Building Emacs overflowed pure space
  2006-07-25  4:13                     ` Richard Stallman
@ 2006-07-25 10:41                       ` YAMAMOTO Mitsuharu
  2006-07-25 10:56                         ` David Kastrup
  0 siblings, 1 reply; 72+ messages in thread
From: YAMAMOTO Mitsuharu @ 2006-07-25 10:41 UTC (permalink / raw)
  Cc: teirllm, ralphm, emacs-devel, monnier, storm, eliz, mathias.dahl

>>>>> On Tue, 25 Jul 2006 00:13:51 -0400, Richard Stallman <rms@gnu.org> said:

>> I tried the exact the same thing (using the naive version), but the
>> dumped emacs crashed immediately after loadup, thinking that "nil"
>> was a filename...  So I didn't think it was safe to do this.

>     I'm not sure if this should be counted as an objection.

> The code he wrote must have a bug in it.  If your code works, I
> guess it does not have a similar bug.

I installed the patch.  It works as far as I tested on multiple
platforms (though that doesn't prove that the prerequisite for sharing
is completely satisfied by the current code, of course).

				     YAMAMOTO Mitsuharu
				mituharu@math.s.chiba-u.ac.jp

^ permalink raw reply	[flat|nested] 72+ messages in thread

* Re: Building Emacs overflowed pure space
  2006-07-25 10:41                       ` YAMAMOTO Mitsuharu
@ 2006-07-25 10:56                         ` David Kastrup
  2006-07-25 11:00                           ` Andreas Schwab
  0 siblings, 1 reply; 72+ messages in thread
From: David Kastrup @ 2006-07-25 10:56 UTC (permalink / raw)
  Cc: teirllm, rms, ralphm, emacs-devel, monnier, storm, eliz,
	mathias.dahl

YAMAMOTO Mitsuharu <mituharu@math.s.chiba-u.ac.jp> writes:

>>>>>> On Tue, 25 Jul 2006 00:13:51 -0400, Richard Stallman <rms@gnu.org> said:
>
>>> I tried the exact the same thing (using the naive version), but the
>>> dumped emacs crashed immediately after loadup, thinking that "nil"
>>> was a filename...  So I didn't think it was safe to do this.
>
>>     I'm not sure if this should be counted as an objection.
>
>> The code he wrote must have a bug in it.  If your code works, I
>> guess it does not have a similar bug.
>
> I installed the patch.  It works as far as I tested on multiple
> platforms (though that doesn't prove that the prerequisite for sharing
> is completely satisfied by the current code, of course).

Actually, I don't see how we can satisfy the prerequisite: a user
might always choose to use `aset' on an existing string.  Of course,
this objection would hold equally for sharing equal strings, not just
strings with a common suffix.

Am I overlooking something?  Or do we have some mechanism in place
against tampering with strings in pure space?  I just tried messing up
`search-whitespace-regexp' with `aset', and was successful.

If we merge strings in pure space, should we not disallow modifying
them?

Everything else seems like a recipe for disaster.

-- 
David Kastrup, Kriemhildstr. 15, 44793 Bochum

^ permalink raw reply	[flat|nested] 72+ messages in thread

* Re: Building Emacs overflowed pure space
  2006-07-25 10:56                         ` David Kastrup
@ 2006-07-25 11:00                           ` Andreas Schwab
  2006-07-25 11:05                             ` David Kastrup
       [not found]                             ` <85ejwahrqi.fsf@lola.goethe.zz>
  0 siblings, 2 replies; 72+ messages in thread
From: Andreas Schwab @ 2006-07-25 11:00 UTC (permalink / raw)
  Cc: teirllm, rms, ralphm, emacs-devel, monnier, storm, eliz,
	YAMAMOTO Mitsuharu, mathias.dahl

David Kastrup <dak@gnu.org> writes:

> If we merge strings in pure space, should we not disallow modifying
> them?

We already do, see CHECK_IMPURE.

Andreas.

-- 
Andreas Schwab, SuSE Labs, schwab@suse.de
SuSE Linux Products GmbH, Maxfeldstraße 5, 90409 Nürnberg, Germany
PGP key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

^ permalink raw reply	[flat|nested] 72+ messages in thread

* Re: Building Emacs overflowed pure space
  2006-07-25 11:00                           ` Andreas Schwab
@ 2006-07-25 11:05                             ` David Kastrup
       [not found]                             ` <85ejwahrqi.fsf@lola.goethe.zz>
  1 sibling, 0 replies; 72+ messages in thread
From: David Kastrup @ 2006-07-25 11:05 UTC (permalink / raw)


Andreas Schwab <schwab@suse.de> writes:

> David Kastrup <dak@gnu.org> writes:
>
>> If we merge strings in pure space, should we not disallow modifying
>> them?
>
> We already do, see CHECK_IMPURE.
>
> Andreas.

So why does
(aset search-whitespace-regexp 0 ?x)
not fail?

isearch.el is loaded from loadup.el.

-- 
David Kastrup, Kriemhildstr. 15, 44793 Bochum

^ permalink raw reply	[flat|nested] 72+ messages in thread

* Re: Building Emacs overflowed pure space
       [not found]                             ` <85ejwahrqi.fsf@lola.goethe.zz>
@ 2006-07-25 11:08                               ` Andreas Schwab
  2006-07-25 11:23                                 ` David Kastrup
  0 siblings, 1 reply; 72+ messages in thread
From: Andreas Schwab @ 2006-07-25 11:08 UTC (permalink / raw)
  Cc: teirllm, rms, ralphm, emacs-devel, monnier, storm, eliz,
	YAMAMOTO Mitsuharu, mathias.dahl

David Kastrup <dak@gnu.org> writes:

> So why does
> (aset search-whitespace-regexp 0 ?x)
> not fail?

Because nobody has put search-whitespace-regexp in pure space.

> isearch.el is loaded from loadup.el.

Loading from loadup.el does not mean putting all strings in pure space.
Only explicit calls to purecopy do that.  There are only a few types
objects that are implicitly put in pure space, like byte code objects or
function bodies.

Andreas.

-- 
Andreas Schwab, SuSE Labs, schwab@suse.de
SuSE Linux Products GmbH, Maxfeldstraße 5, 90409 Nürnberg, Germany
PGP key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

^ permalink raw reply	[flat|nested] 72+ messages in thread

* Re: Building Emacs overflowed pure space
  2006-07-25 11:08                               ` Andreas Schwab
@ 2006-07-25 11:23                                 ` David Kastrup
  0 siblings, 0 replies; 72+ messages in thread
From: David Kastrup @ 2006-07-25 11:23 UTC (permalink / raw)


Andreas Schwab <schwab@suse.de> writes:

> David Kastrup <dak@gnu.org> writes:
>
>> So why does
>> (aset search-whitespace-regexp 0 ?x)
>> not fail?
>
> Because nobody has put search-whitespace-regexp in pure space.
>
>> isearch.el is loaded from loadup.el.
>
> Loading from loadup.el does not mean putting all strings in pure
> space.  Only explicit calls to purecopy do that.  There are only a
> few types objects that are implicitly put in pure space, like byte
> code objects or function bodies.

And then we get something like

Debugger entered--Lisp error: (error "Attempt to modify read-only object")
  aset("isearch-forward" 0 111)
  eval((aset (symbol-name (quote isearch-forward)) 0 111))
  eval-expression((aset (symbol-name (quote isearch-forward)) 0 111) nil)
  call-interactively(eval-expression)

correct?

-- 
David Kastrup, Kriemhildstr. 15, 44793 Bochum

^ permalink raw reply	[flat|nested] 72+ messages in thread

* Building Emacs overflowed pure space
@ 2006-08-07 13:30 Klaus Zeitler
  2006-08-08  9:20 ` Klaus Zeitler
  0 siblings, 1 reply; 72+ messages in thread
From: Klaus Zeitler @ 2006-08-07 13:30 UTC (permalink / raw)


Since today I'm seeing the following warning, when I start emacs:
Warning (initialization): Building Emacs overflowed pure space.
(See the node Pure Storage in the Lisp manual for details.)

Is this a new warning for an already existing problem or is this a new
feature?
The docu says to increase PURESIZE, but I'm not sure to or by what.
Maybe it'd be better to increase BASE_PURESIZE or one of these ..._EXTRA
constants, but then again I'm not sure how much.

BTW this is a Solaris 5.8 build.

Klaus

-- 
 ------------------------------------------
|  Klaus Zeitler      Lucent Technologies  |
|  Email:             kzeitler@lucent.com  |
 ------------------------------------------
---
Machine-Independent, adj.: Does not run on any existing machine.

^ permalink raw reply	[flat|nested] 72+ messages in thread

* Re: Building Emacs overflowed pure space
  2006-08-07 13:30 Klaus Zeitler
@ 2006-08-08  9:20 ` Klaus Zeitler
  0 siblings, 0 replies; 72+ messages in thread
From: Klaus Zeitler @ 2006-08-08  9:20 UTC (permalink / raw)


>>>>> "Klaus" == Klaus Zeitler <kzeitler@lucent.com> writes:
    Klaus> 
    Klaus> Since today I'm seeing the following warning, when I start emacs:
    Klaus> Warning (initialization): Building Emacs overflowed pure space.
    Klaus> (See the node Pure Storage in the Lisp manual for details.)

With the new value for BASE_PURESIZE (1120000 + ...) the warning is gone
for the Solaris 5.8.build. Thanks

-- 
 ------------------------------------------
|  Klaus Zeitler      Lucent Technologies  |
|  Email:             kzeitler@lucent.com  |
 ------------------------------------------
---
The only possible interpretation of any research whatever in the
`social sciences' is: some do, some don't.  -- Ernest Rutherford

^ permalink raw reply	[flat|nested] 72+ messages in thread

end of thread, other threads:[~2006-08-08  9:20 UTC | newest]

Thread overview: 72+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2006-07-18  8:38 Building Emacs overflowed pure space Mathias Dahl
2006-07-18  9:02 ` Ralph Moritz
2006-07-18 12:06   ` Mathias Dahl
2006-07-18 13:37     ` David Hansen
2006-07-18 15:00   ` Richard Stallman
2006-07-18 18:55     ` Luc Teirlinck
2006-07-18 20:16       ` Eli Zaretskii
2006-07-18 20:56         ` Stefan Monnier
2006-07-19  2:51           ` Eli Zaretskii
2006-07-19  4:35             ` Stefan Monnier
2006-07-19  4:42             ` Miles Bader
2006-07-19 18:13               ` Eli Zaretskii
2006-07-19 13:52           ` mituharu
2006-07-19 15:07             ` Stefan Monnier
2006-07-19 21:16             ` Richard Stallman
2006-07-20  1:12               ` YAMAMOTO Mitsuharu
2006-07-22  9:06             ` YAMAMOTO Mitsuharu
2006-07-22 16:03               ` Stefan Monnier
2006-07-22 23:11                 ` Kim F. Storm
2006-07-23  3:47                   ` Stefan Monnier
2006-07-25  1:14                   ` YAMAMOTO Mitsuharu
2006-07-25  4:13                     ` Richard Stallman
2006-07-25 10:41                       ` YAMAMOTO Mitsuharu
2006-07-25 10:56                         ` David Kastrup
2006-07-25 11:00                           ` Andreas Schwab
2006-07-25 11:05                             ` David Kastrup
     [not found]                             ` <85ejwahrqi.fsf@lola.goethe.zz>
2006-07-25 11:08                               ` Andreas Schwab
2006-07-25 11:23                                 ` David Kastrup
2006-07-23  5:26               ` Richard Stallman
2006-07-18 22:01         ` Luc Teirlinck
2006-07-18 23:44           ` Chong Yidong
2006-07-19  0:06             ` Luc Teirlinck
2006-07-19  0:14               ` Luc Teirlinck
2006-07-19  0:21               ` Luc Teirlinck
2006-07-19  2:56           ` Eli Zaretskii
2006-07-18 19:29     ` Luc Teirlinck
2006-07-19  6:04       ` Richard Stallman
2006-07-19  9:22         ` YAMAMOTO Mitsuharu
2006-07-19  9:46           ` YAMAMOTO Mitsuharu
2006-07-19 15:05             ` Stefan Monnier
2006-07-19 23:13               ` Kim F. Storm
2006-07-20  4:05                 ` Stefan Monnier
2006-07-20  2:13               ` YAMAMOTO Mitsuharu
2006-07-20  2:21               ` Richard Stallman
2006-07-20  2:21               ` Richard Stallman
2006-07-20  4:07                 ` Stefan Monnier
2006-07-20  8:14                   ` Kim F. Storm
2006-07-20 15:26                     ` Stefan Monnier
2006-07-20 18:16                       ` Richard Stallman
2006-07-19 21:15           ` Richard Stallman
2006-07-19 22:47             ` Kim F. Storm
2006-07-20 15:33               ` Richard Stallman
2006-07-20  9:34             ` Kim F. Storm
2006-07-20  9:58               ` David Kastrup
2006-07-20 11:35                 ` Kim F. Storm
2006-07-20 13:46                   ` David Kastrup
2006-07-20 18:16               ` Richard Stallman
2006-07-21  8:36                 ` Kim F. Storm
2006-07-21 19:36                   ` Richard Stallman
2006-07-21 19:48                     ` David Kastrup
2006-07-21 20:40                       ` Andreas Schwab
2006-07-21 20:49                         ` David Kastrup
2006-07-22 15:49                       ` Richard Stallman
2006-07-20 16:03             ` Stefan Monnier
2006-07-20 21:46               ` Richard Stallman
2006-07-20 22:11                 ` Stefan Monnier
2006-07-21  4:46                   ` Richard Stallman
2006-07-21 12:57                     ` Stefan Monnier
2006-07-21 19:37                       ` Richard Stallman
2006-07-19  0:44 ` Katsumi Yamaoka
  -- strict thread matches above, loose matches on Subject: below --
2006-08-07 13:30 Klaus Zeitler
2006-08-08  9:20 ` Klaus Zeitler

Code repositories for project(s) associated with this public inbox

	https://git.savannah.gnu.org/cgit/emacs.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).