ehelp woes, or why I hate a module that I love so much

ehelp woes, or why I hate a module that I love so much

[Juanma Barranquero]

I'm a big fan of electric buffers and windows; I mean that I like help
windows and temporary buffers to be as reduced and inobstrusive as
posible, and like the ability to dismiss temporary information with a
simple keystroke.

To that end I use ehelp and try to electric-helpify as many information
functions as I can (not counting lobbing module developers to use
`fit-window-to-buffer', `shrink-window-if-larger-than-buffer' and
`electric-helpify' :)

However, it seems like ehelp is not much used, and so not much
maintained, and it's suffering from several shortcomings.

A short list of current problems with ehelp:

- When an electric-helpify'ed function splits the current window
  (because it was the only one), first it moves the cursor to the first
  visible character in the window (to keep the buffer from scrolling, I think).
  The cursor position is restored if the user (q)uits the function, but it
  is *not* if the user exits the function by (r)etaining its output buffer.

- Also, (as stated in ehelp's comments), after (r)etaining, the
  resulting buffer is not a *Help* buffer.

- If you electric-helpify a function whose output has text attributes,
  those are not correctly shown. For example, with global-font-lock-mode
  active, define

  (defun my-describe-char ()
    (interactive)
    (electric-helpify #'describe-char))

  and do M-x my-describe-char over a font-locked character. You'll get
  something like:

>   character: h (0150, 104, 0x68)
>     charset: ascii (ASCII (ISO646 IRV))
>  code point: 104
>      syntax: w 	which means: word
>    category: a:ASCII   l:Latin  
> buffer code: 0x68
>   file code: 68 (encoded by coding system iso-latin-9-dos)
>        font: -outline-Courier New-normal-r-normal-normal-12-90-96-96-c-70-iso8859-1
> 
> There are text properties here:
>   fontified            t
>   face                 font-lock-function-name-face

  where "fontified" and "face" are shown int the default font. However,
  if you (r)etain or otherwise bypass the electric-command-loop (like by
  doing M-:), those strings are immediately shown in font-lock-face
  (italic by default).

- Following links in the *Help* buffer does not work right. For example,
  if you do

  M-x electric-describe-function defun

  and move the cursor over `interactive' and press Return, the buffer
  changes to show the documentation of `interactive', and we still get
  the message "Press SPC to scroll, q to exit, r to retain". However,
  those keys do *not* work (q does not exit, r does not retain). Moreover,
  following the [back] button moves back, and we get again the "Press q to
  exit, r to retain" message, but the only way to exit cleanly is C-g.

- If you invoke an electric-helpify'ed function and then kill the buffer
  (I do that mistake with surprising regularity), the
  electric-command-loop remains active. Yes, that's probably not a bug,
  but still is very annoying.

I'm not entirely sure what I'm ranting about here. I guess what I feel
is that perhaps ehelp should be less electric, in the sense that,
instead of having its own command-loop, it should define local
keybindings for (q)uitting, (r)etaining, etc. that would do the job just
fine and present less inconvenients... 


                                                           /L/e/k/t/u

Re: ehelp woes, or why I hate a module that I love so much

[D. Goel]

Juanma



> - If you invoke an electric-helpify'ed function and then kill the
> buffer (I do that mistake with surprising regularity), the
> electric-command-loop remains active. Yes, that's probably not a
> bug, but still is very annoying.

FWIW, i think i finally understood a way to get out of this loop ---
invoke a ehelp-buffer, and then press 'r' to retain it, and the
previous loop is gone...

(am a big user or ehelp my C-h-->'ehelp, and have observed all the
same points you mentioned, and imho too, as you said, the buffer
retained upon quitting should be a help buffer, the follow-links
should work, the quit should be clean, the 'ehelp-loop' would do
better to be a 'local' thing somehow... )


DG				   
-- 

Re: ehelp woes, or why I hate a module that I love so much

[Juanma Barranquero]

On 25 Jun 2002 11:06:54 -0400, "D. Goel" <deego@glue.umd.edu> wrote:

> FWIW, i think i finally understood a way to get out of this loop ---
> invoke a ehelp-buffer, and then press 'r' to retain it, and the
> previous loop is gone...

He, thanks, I think I'm gonna use it often ;-)

> (am a big user or ehelp my C-h-->'ehelp, and have observed all the
> same points you mentioned, and imho too, as you said, the buffer
> retained upon quitting should be a help buffer, the follow-links
> should work, the quit should be clean, the 'ehelp-loop' would do
> better to be a 'local' thing somehow... )

Yes. OTOH, I know pretty well that saying "should" is a way of musing "I
should" :)

What restrains me is that I've studied ehelp.el (and electric.el) for
quite a while, and the more I look at it, the more I feel a full rewrite
would be the best answer...


                                                           /L/e/k/t/u

Re: ehelp woes, or why I hate a module that I love so much

[Richard Stallman]

I don't think anyone else is working on this, so if you want to
develop these changes, please do.

Re: ehelp woes, or why I hate a module that I love so much

[Juanma Barranquero]

On Wed, 26 Jun 2002 18:24:24 -0400, Richard Stallman <rms@gnu.org> wrote:

> I don't think anyone else is working on this, so if you want to
> develop these changes, please do.

First I'd like to get some input from people around here regarding why
it is necessary for ehelp to use its own command-loop. I sure would like
to get rid of it.

AFAICS, the command-loop is used only to (sort of) "force" the user to
do only pg-up, pg-down, quit, retain, etc. and show a prompt in the echo
area.

But in fact it is perfectly posible to switch to another window, kill
the electrified buffer, etc., so the command-loop is more of a nuisance
than any kind of help.

I think ehelp is conflating two different things: a framework to make
temporary, small-sized windows at the bottom of the frame and restore
the window configuration afterwards, and an "electric" interface to get
a reminding prompt and to dismiss the window with just one keystroke.

IMHO, there should be a splicit framework for temporary, small-sized
windows (using fit-window-to-buffer, etc), but its "electricity" should
be limited to defining a few useful keybindings. Moving around is
self-evident (I mean, all modes have a way to move around), and quitting
and remaining are just particular cases of burying or killing the buffer
or switching to another window. Or am I missing something important here?


                                                           /L/e/k/t/u

Re: ehelp woes, or why I hate a module that I love so much

[Stefan Monnier]

> First I'd like to get some input from people around here regarding why
> it is necessary for ehelp to use its own command-loop. I sure would like
> to get rid of it.

I don't think there's a very strong reason why the code uses its
own command-loop.  In other settings an explicit while loop calling
`read-event' is used (in query-replace, for example), or some
special keymap is installed on the overriding-terminal-local-map
(in isearch).
I'm sure that overriding-minor-mode-map-alist could be used as well.

I don't like `read-event' because it fails to obey function-key-map
and key-translation-map and because it doesn't run post/pre-command-hook
and other such things.

Note that all those different approaches behave subtly differently,
so the trade-offs are not always obvious.  It might be worth trying
out something like overriding-terminal-local-map and see how it works.
But you might end up finding out that the same problems show up
(after all, a recursive command loop is really not very different
from a (while ... (read-key-sequence) ...) or from temporarily
setting the overriding-terminal-local-map).


	Stefan

Re: ehelp woes, or why I hate a module that I love so much

[Juanma Barranquero]

On Thu, 27 Jun 2002 15:57:16 -0400, "Stefan Monnier" <monnier+gnu/emacs@rum.cs.yale.edu> wrote:

> I'm sure that overriding-minor-mode-map-alist could be used as well.

I'm unable to find anything about overriding-minor-mode-map-alist. I
assume you're talking about overriding-local-map...

> I don't like `read-event' because it fails to obey function-key-map
> and key-translation-map and because it doesn't run post/pre-command-hook
> and other such things.

I don't like either. I don't see any gain switching from the electric.el
prepackaged command-loop to an explicit `read-event' or
`read-key-sequence' loop. I'm talking more of a keymap-oriented approach,
perhaps via `overriding-local-map' or something 

> Note that all those different approaches behave subtly differently,
> so the trade-offs are not always obvious.  It might be worth trying
> out something like overriding-terminal-local-map and see how it works.

Yes, I'll try to take a look into it.

Thanks,

                                                           /L/e/k/t/u

Re: ehelp woes, or why I hate a module that I love so much

[Stefan Monnier]

> > I'm sure that overriding-minor-mode-map-alist could be used as well.
> I'm unable to find anything about overriding-minor-mode-map-alist. I
> assume you're talking about overriding-local-map...

No, it was minor-mode-overriding-map-alist.  Sorry.


	Stefan

Re: ehelp woes, or why I hate a module that I love so much

[Juanma Barranquero]

On Tue, 02 Jul 2002 11:18:39 -0400, "Stefan Monnier" <monnier+gnu/emacs@rum.cs.yale.edu> wrote:

> No, it was minor-mode-overriding-map-alist.  Sorry.

Ah, OK. Thanks.


                                                           /L/e/k/t/u

Re: ehelp woes, or why I hate a module that I love so much

[Richard Stallman]

I never used ehelp or any of the packages that use ehelp, so I don't
have much in the way of opinions about it.  But I tend to think that
there are other areas which are much more important to work on than
this one.

Re: ehelp woes, or why I hate a module that I love so much

[Juanma Barranquero]

On Sat, 29 Jun 2002 02:41:47 -0600 (MDT), Richard Stallman <rms@gnu.org> wrote:

> I never used ehelp or any of the packages that use ehelp, so I don't
> have much in the way of opinions about it.  But I tend to think that
> there are other areas which are much more important to work on than
> this one.

Oh, sure, but for those of us who do use ehelp or ehelp-related packages
(term, terminal, emulation/edt, eshell/em-pred and textmodes/ispell,
IIRC), there's a lot of itching there to scratch.

Anyway, I've taken a look into etc/TODO, and many items seem to be a bit
over my head. Just out of curiosity, what does exactly mean
"Internationalize Emacs's messages."?


                                                           /L/e/k/t/u

Re: ehelp woes, or why I hate a module that I love so much

[Eli Zaretskii]

On Tue, 2 Jul 2002, Juanma Barranquero wrote:

> Anyway, I've taken a look into etc/TODO, and many items seem to be a bit
> over my head. Just out of curiosity, what does exactly mean
> "Internationalize Emacs's messages."?

It means teach Emacs to display messages in the locale-specific language, 
not just in US English.  It's the equivalent of gettext-izing Emacs 
(except that in Emacs, this is more complicated since parts of the source 
are in Lisp).

There was a long discussion on emacs-devel about the various aspects of 
this several months ago.

Re: ehelp woes, or why I hate a module that I love so much

[Juanma Barranquero]

On Tue, 2 Jul 2002 18:49:42 +0300 (IDT), Eli Zaretskii <eliz@is.elta.co.il> wrote:

> It means teach Emacs to display messages in the locale-specific language, 
> not just in US English.  It's the equivalent of gettext-izing Emacs 
> (except that in Emacs, this is more complicated since parts of the source 
> are in Lisp).

OK, thanks.

> There was a long discussion on emacs-devel about the various aspects of 
> this several months ago.

Hmm. I have the archived list from Sept 2000 and I've not found anything.
I'll take a closer look.

Thanks,


                                                           /L/e/k/t/u

Re: ehelp woes, or why I hate a module that I love so much

[Richard Stallman]

    Just out of curiosity, what does exactly mean
    "Internationalize Emacs's messages."?

It means to make Emacs use gettext or something equivalent.

In terms of benefit for *all* users, this is at least a thousand times
as important as any work on ehelp.  Maybe a hundred thousand times as
important.  Improvingd on ehelp can only be only useful for writing
other fairly obscure packages for advanced users.  Internationalizing
the messages Emacs outputs would be useful to every potential Emacs
who does not speak English.

Re: ehelp woes, or why I hate a module that I love so much

[Simon Josefsson]

Richard Stallman <rms@gnu.org> writes:

> In terms of benefit for *all* users, this is at least a thousand times
> as important as any work on ehelp.  Maybe a hundred thousand times as
> important.  Improvingd on ehelp can only be only useful for writing
> other fairly obscure packages for advanced users.  Internationalizing
> the messages Emacs outputs would be useful to every potential Emacs
> who does not speak English.

Has anyone started working on this?  I'm having some troubles groking
how it really would work, gettext() is for C but many messages in
Emacs come from elisp code.  gettext.el?

I notice that people have thought about this problem for Guile (e.g.,
http://www.glug.org/patches/2001-07-29.gettext-support/).  Guile
support would be nice..

Re: ehelp woes, or why I hate a module that I love so much

[Eli Zaretskii]

On Wed, 3 Jul 2002, Simon Josefsson wrote:

> Has anyone started working on this?

I don't think so, but I might be wrong.

> I'm having some troubles groking
> how it really would work, gettext() is for C but many messages in
> Emacs come from elisp code.  gettext.el?

Figuring this out is part of the job, IMHO.

Again, I suggest to start by reading past discussions.  Here's a beginning:

  http://mail.gnu.org/pipermail/emacs-devel/2001-December/003666.html

Re: ehelp woes, or why I hate a module that I love so much

[Juanma Barranquero]

On Wed, 3 Jul 2002 14:57:25 -0600 (MDT), Richard Stallman <rms@gnu.org> wrote:

> In terms of benefit for *all* users, this is at least a thousand times
> as important as any work on ehelp.  Maybe a hundred thousand times as
> important.

Yeah, well, with that logic we all should sort together the TODO in
order of decreasing importance and start implementing things from the
top down, never stepping aside nor jumping over boring things...

But that kind of situation tends to make developers lose interest really
fast. How many have left Emacs development because the things they were
interested in are (sometimes actively) discouraged? And, with your
argument, what's the relative importance of integrating Guile, when
Emacs already has a fine and tested extension language? Why replace
something that works, instead of focusing on developing new features more
oriented towards the *user*, as oposed as the developer (who's the only
one that will, perhaps, get any improvement from Guile)? The hundred of
thousands of users implied in your message aren't interested in
programming Emacs, just using it and customizing it (through Customize).
Fixing/expanding desktop.el and figuring a way to save and restore
sessions in a proper and well-defined way is a lot more useful to them
that half the items in etc/TODO.

I'm not trying to be polemic here, just pointing out that we don't have
a clear-cut model of development where it is perfectly clear what's
important and what's not. Perhaps we need it, I don't know. Maybe we
should establish a Master Plan and start assigning tasks and actively
pursuing them. But meanwhile, every fix and every improvement that does
not conflict with the Emacs philosophy and that makes the developer feel
proud is good in my view, even if it seems insignificant. Yesterday I
spent two hours hunting down typos. I'm *sure* most Emacs users won't
ever notice, but personally I feel a lot more confortable when I do C-h C-f
some-function and don't see mispellings in its docstring. With your view,
I should be ashamed of spending the time on this, instead of doing some
worthwhile work.

Sure internationalizing Emacs is a lot more important than hacking ehelp.
OTOH, perhaps I'm able to fix ehelp's problems by myself but I wouldn't
be able to gettextize Emacs unless I have the help of a more experienced
developer...

Anyway, if an expert with some knowledge of gettext plans to start
internationalizing Emacs in the near future, I promise to help in that
as much as I can.


                                                           /L/e/k/t/u

Re: ehelp woes, or why I hate a module that I love so much

[Eli Zaretskii]

On Thu, 4 Jul 2002, Juanma Barranquero wrote:

> Yeah, well, with that logic we all should sort together the TODO in
> order of decreasing importance

AFAIK, it already is sorted.

> Yesterday I
> spent two hours hunting down typos. I'm *sure* most Emacs users won't
> ever notice, but personally I feel a lot more confortable when I do C-h C-f
> some-function and don't see mispellings in its docstring.

Don't worry about this, your contributions are most welcome, as those of 
anyone else.  You have every reason to be proud.

> With your view,
> I should be ashamed of spending the time on this, instead of doing some
> worthwhile work.

Richard will surely reply, but I'm confident he didn't mean that you 
should be ashamed for working on Emacs.

Re: ehelp woes, or why I hate a module that I love so much

[Juanma Barranquero]

On Thu, 4 Jul 2002 13:13:46 +0300 (IDT), Eli Zaretskii <eliz@is.elta.co.il> wrote:

> AFAIK, it already is sorted.

Top-down? Do you mean that "implementing a smoother vertical scroll
facility" has more importance than "internationalize Emacs's messages"?
:)

> Richard will surely reply, but I'm confident he didn't mean that you 
> should be ashamed for working on Emacs.

Well, my Spanish must be getting through my poor English, and we
Spaniards use stronger terms in arguments that Anglo-Saxon people ;-)

What I meant was that, with Richard's POV, I should be continually
thinking if the bugs I'm hunting, or the new features I'm thinking
(which often are small user conveniences) are really important or should
I be doing another, more worthwhile, Emacs thing.

But the fact is that I'm a (small) Emacs developer, but also a user, and
as a user, every time that I use ehelp I get somewhat irritated (though
not as irritated as if I had to use Emacs without ehelp ;) It makes
perfect sense to try and fix those problems, if I can, irrespective of
its relevance in the Great Master Plan.


                                                           /L/e/k/t/u

Re: ehelp woes, or why I hate a module that I love so much

[Eli Zaretskii]

> Date: Thu, 04 Jul 2002 13:52:36 +0200
> From: Juanma Barranquero <lektu@terra.es>
> 
> > AFAIK, it already is sorted.
> 
> Top-down? Do you mean that "implementing a smoother vertical scroll
> facility" has more importance than "internationalize Emacs's messages"?

The sorting is not perfect: the first 10 or so entries are the most
important ones; the rest are unsorted AFAIK.

Re: ehelp woes, or why I hate a module that I love so much

[Simon Josefsson]

Juanma Barranquero <lektu@terra.es> writes:

> But that kind of situation tends to make developers lose interest really
> fast. How many have left Emacs development because the things they were
> interested in are (sometimes actively) discouraged? And, with your
> argument, what's the relative importance of integrating Guile, when
> Emacs already has a fine and tested extension language? Why replace
> something that works, instead of focusing on developing new features more
> oriented towards the *user*, as oposed as the developer (who's the only
> one that will, perhaps, get any improvement from Guile)? The hundred of
> thousands of users implied in your message aren't interested in
> programming Emacs, just using it and customizing it (through Customize).

I think having Emacs support Guile would be very visible to users, as
it likely would mean having threads which UI wise is probably the most
important new feature added to Emacs since it started supporting
windowing systems.  IMHO it is more important than GTK/Gnome or
Unicode support.

But generally I agree, most developers help a project because they
find something fun to work on, and if that is discouraged they get
bored and do something else instead (like, say, forks the application
and starts working on that ;-)).

Re: ehelp woes, or why I hate a module that I love so much

[Juanma Barranquero]

On Thu, 04 Jul 2002 13:02:12 +0200, Simon Josefsson <jas@extundo.com> wrote:

> I think having Emacs support Guile would be very visible to users, as
> it likely would mean having threads which UI wise is probably the most
> important new feature added to Emacs since it started supporting
> windowing systems.

OK, threads per se are good, but why should they be contingent on using
Guile? They are now, because the plan is switching to Guile someday and
it's absurd to duplicate efforts to discard them soon after. But perhaps
adding thread support to Emacs/elisp is easier than doing the conversion
from Emacs/elisp to Emacs/Guile. Who knows?

From my (admittedly not very informed) perspective, the Emacs/Guile
thing seems to be going nowhere (or, if it is going somewhere, it's
doing it very quietly and off side). Meanwhile, some developments seem
stagnated (though I'm very glad to see the lexbind work), and some
people has abandoned Emacs *because* of Guile.

I'm not going to reopen that Pandora box. It seems a decission was
reached some time ago and it's not going to change. But I wonder, if a
poll where made into the people who's actively developing Emacs (and not
just Guile), how many would think the conversion is worth the trouble.

> But generally I agree, most developers help a project because they
> find something fun to work on, and if that is discouraged they get
> bored and do something else instead

Don't take me wrong. I agree with that, but I also believe that thinking
on a bigger scale and having goals is good. It's just that one issue and
the other shouldn't be (felt) such at odds.

(Disclaimer: the longer the message, the less understandable is my
English. Sorry about that.)

                                                           /L/e/k/t/u

Re: ehelp woes, or why I hate a module that I love so much

[Miles Bader]

Juanma Barranquero <lektu@terra.es> writes:
> But perhaps adding thread support to Emacs/elisp is easier than doing
> the conversion from Emacs/elisp to Emacs/Guile. Who knows?

I rather doubt that switching to guile will suddenly make emacs one big
happy multi-threaded application -- I suspect the biggest effort will be
in making various lisp applications thread-safe, not in the low-level (C
level) thread support, so it will be incurred regardless of how threads
are added.

-Miles
-- 
`The suburb is an obsolete and contradictory form of human settlement'

Re: ehelp woes, or why I hate a module that I love so much

[Juanma Barranquero]

On 04 Jul 2002 21:19:54 +0900, Miles Bader <miles@gnu.org> wrote:

> I suspect the biggest effort will be
> in making various lisp applications thread-safe, not in the low-level (C
> level) thread support, so it will be incurred regardless of how threads
> are added.

Yes, sure.

So Guile looks even less of a necessity.


                                                           /L/e/k/t/u

Re: ehelp woes, or why I hate a module that I love so much

[Simon Josefsson]

Miles Bader <miles@gnu.org> writes:

> Juanma Barranquero <lektu@terra.es> writes:
>> But perhaps adding thread support to Emacs/elisp is easier than doing
>> the conversion from Emacs/elisp to Emacs/Guile. Who knows?
>
> I rather doubt that switching to guile will suddenly make emacs one big
> happy multi-threaded application -- I suspect the biggest effort will be
> in making various lisp applications thread-safe, not in the low-level (C
> level) thread support, so it will be incurred regardless of how threads
> are added.

Yes, and I am eager to develop patches making Gnus a threaded
application, but until Emacs has threads I can't do that.

Re: ehelp woes, or why I hate a module that I love so much

[Simon Josefsson]

Juanma Barranquero <lektu@terra.es> writes:

> On Thu, 04 Jul 2002 13:02:12 +0200, Simon Josefsson <jas@extundo.com> wrote:
>
>> I think having Emacs support Guile would be very visible to users, as
>> it likely would mean having threads which UI wise is probably the most
>> important new feature added to Emacs since it started supporting
>> windowing systems.
>
> OK, threads per se are good, but why should they be contingent on using
> Guile? They are now, because the plan is switching to Guile someday and
> it's absurd to duplicate efforts to discard them soon after. But perhaps
> adding thread support to Emacs/elisp is easier than doing the conversion
> from Emacs/elisp to Emacs/Guile. Who knows?

Yes, you are right, but there seem to be several advantages of using
guile (threads, i18n?, re-use between other applications, GTK
bindings, etc) and I strongly doubt that implementing all those things
in emacs is easier than switching to guile.

> From my (admittedly not very informed) perspective, the Emacs/Guile
> thing seems to be going nowhere (or, if it is going somewhere, it's
> doing it very quietly and off side).

I perceive the situation the same.  What happened with the Guile Emacs
project?  It seems to have died after the 0.5 release.

Re: ehelp woes, or why I hate a module that I love so much

[Juanma Barranquero]

On Thu, 04 Jul 2002 16:00:13 +0200, Simon Josefsson <jas@extundo.com> wrote:

> Yes, you are right, but there seem to be several advantages of using
> guile (threads, i18n?, re-use between other applications, GTK
> bindings, etc) and I strongly doubt that implementing all those things
> in emacs is easier than switching to guile.

Well, you said that threads would be the main advantage, user-wise, of
the Emacs/Guile integration.

i18n is certainly a big plus, but I don't think Guile will make that
*much* easier: on one hand, because you have to maintain elisp
compatibility and supply the current .el files, so they won't be
magically translated/internationalized. On the other, because there's
still the C part (though that can be gettextized, it seems). The bigger
problem to internationalize Emacs doesn't seem to be the sheer amount of
manual work, but coming with a practical scheme (no pun intended ;) to
do the adaptation.

> I perceive the situation the same.  What happened with the Guile Emacs
> project?  It seems to have died after the 0.5 release.

From the mailing-list at
http://sources.redhat.com/ml/guile-emacs/2002-q1/msg00003.html:

>From: Keisuke Nishida <knishida at sky dot sannet dot ne dot jp> 

<snip>

> I have stopped working on guile-emacs.  My idea was to embed
> Guile into Emacs, which was not a good idea.
> 
> Later on I started a new project, called emacs-guile ;), which
> intended to use Guile from Emacs as an external language processor.
> With emacs-guile, you can use Guile's module libraries from Emacs
> by importing them into the Lisp world.
> 
> See the following package for more info:
> 
>   http://prdownloads.sourceforge.net/gemacs/emacs-guile-0.2.tar.gz
> 
> Not me, but Ken Raeburn is working on a project that is replacing
> Emacs's core evaluator by Guile's one.  

Ken Raeburn has a page on Guile-based Emacs (http://www.mit.edu/~raeburn/guilemacs/),
but it was last updated on 1999-07-07. Ken is subscribed to this list,
so I suppose he can comment on it.

                                                           /L/e/k/t/u

emacs and guile (Re: ehelp woes, or why I hate a module that I love so much)

[Ken Raeburn]

Juanma Barranquero <lektu@terra.es> writes:
> Ken Raeburn has a page on Guile-based Emacs (http://www.mit.edu/~raeburn/guilemacs/),
> but it was last updated on 1999-07-07. Ken is subscribed to this list,
> so I suppose he can comment on it.

Yeah, I was just looking today at how old that page is....

Yes, I've been working on it slowly and quietly, as my free time
allows.  I'm working on it because it interests me, I'd like more
opportunities to work with Scheme, I'd like to encourage others to
work with Scheme, I think it would get Guile more attention (including
its deficiencies in some areas, which hopefully would be readily
identified with a huge application like Emacs and addressed), and I
really want Simon to write those Gnus patches :-).  How threads at the
Scheme level might translate up to the UI, though, I have no idea.

The first batch of work I did, which resulted in the state described
on that web page, caused a fairly large and painful divergence, and
keeping up with new stuff being added to Emacs proved to be a big time
sink.  So I stopped that work, and started a different approach,
namely making some of the Lisp internals in the main FSF sources
easier to isolate and replace.  Hence the ENABLE_CHECKING,
HIDE_LISP_IMPLEMENTATION, and other changes to add all sorts of
compile-time and run-time checks, and the associated (functionally
invisible, I hope) transformations I've been applying to various
patterns throughout the sources to make more use of lisp.h macros.

(BTW, I've got a patch that adds configures option to turn on
ENABLE_CHECKING, or just the use of a union for Lisp_Object.  Should I
check it in?  I've noticed one or two people have been checking in
fixes for Lisp_Object/int mixups, for which I'm grateful, but I'd like
to encourage more of that checking as people write the code in the
first place.)

At this point, int, float and cons values, and to a lesser degree
strings, should be easy to rip out and reimplement.  Converting other
types to Guile's SCM type would be trickier because of a difference in
type handling.  Guile's C-level type extension system involves
allocating Scheme objects which have some application type information
and a pointer to the application data.  In the current Emacs system,
on the other hand, most of the type and pointer information is all in
the Lisp_Object value, and the "==" comparisons require that the
handle value be unique; you could call XSETBUFFER on a Lisp_String
pointer and it wouldn't complain.  So, should XSETBUFFER for Guile
create one of these Scheme objects, or should it use one allocated
earlier?

My approach last time around was to add a SCM field to most of the
"interesting" data types (symbol, misc, and vectorlike), fill it in
when the object was created, and make XSETFOO just copy the value out.
Rather crude, and wasteful of storage space, but it got the job done,
and it minimized the intrusion in random Emacs code and kept it
running with both object representations.

I want to try eliminating more of the XSETFOO calls outside of object
creation, in places where for example a simple assignment will do
instead.  But if I can't get most of them, I may have to use that same
approach again.

Since with the string macro stuff I've been changing in the last few
days, I've nearly reached the point I had in my old development tree
with regard to isolating those types I thought I could easily replace.
(I was starting to work on strings but hadn't finished them.  Vectors
would be easy except for the bits of code that group them with the
"vectorlike" types like buffers.)  So I'm thinking about starting a
branch soon and starting on some actual Guile code again.  Once the
branch is made, it should be much, much easier to get other people
involved.

Smooth interaction between the languages will need a lot more time,
but an Emacs that uses the Guile storage system and can communicate
with Scheme code, even if only by passing a few simple object types,
is certainly doable; I had a proof-of-concept implementation up and
limping long ago, but not remotely ready to be evaluated in terms of
memory use, performance, etc.

So, that's roughly what the current status is of my work....

Ken

Re: emacs and guile (Re: ehelp woes, or why I hate a module that I love so much)

[Juanma Barranquero]

On Tue, 16 Jul 2002 22:58:13 -0400, Ken Raeburn <raeburn@raeburn.org> wrote:

> So, that's roughly what the current status is of my work....

Thanks *a lot* for your comments.


                                                           /L/e/k/t/u

Re: emacs and guile (Re: ehelp woes, or why I hate a module that I love so much)

[Kai Großjohann]

Ken Raeburn <raeburn@raeburn.org> writes:

> Smooth interaction between the languages will need a lot more time,

Is there a way to use Guile (Scheme?) to emulate Emacs Lisp?  I think
somebody has written a Common Lisp module that emulates Emacs Lisp,
but I haven't heard about Guile/Scheme.

Or is the idea to have two languages side-by-side in Emacs?  I think
this side-by-side feature is a strong point of Guile.  But I don't
know much about Guile, so this could be totally wrong.

kai
-- 
A large number of young women don't trust men with beards.  (BFBS Radio)

Re: emacs and guile (Re: ehelp woes, or why I hate a module that I love so much)

[Richard Stallman]

    Is there a way to use Guile (Scheme?) to emulate Emacs Lisp?  I think
    somebody has written a Common Lisp module that emulates Emacs Lisp,
    but I haven't heard about Guile/Scheme.

This is a very important and useful area to work on.

    Or is the idea to have two languages side-by-side in Emacs?

All else being equal, I think it would be best if we could support
both languages on one engine.  That would make Emacs simpler.
However, if that is not feasible, as a fall back we could use
both engines.

Re: emacs and guile (Re: ehelp woes, or why I hate a module that I love so much)

[Ken Raeburn]

> Is there a way to use Guile (Scheme?) to emulate Emacs Lisp?  I think
> somebody has written a Common Lisp module that emulates Emacs Lisp,
> but I haven't heard about Guile/Scheme.

I think as a proof of concept you could write a machine-code and
system-call emulator in Scheme, and run the current Emacs binary
including its Lisp interpreter, so, yes.  Of course, the interesting
trick is to do it more efficiently than that. :-)

People have definitely worked on it.

The Guile CVS repository contains some code that loads quite a bit of
elisp.  I haven't looked at it in a while, so I can't tell you
anything about the performance, but I think it did depend on some
extension code written in C; why, I'm not sure, but that costs some
karma points in my book.

Some XEmacs people (including one who told me he's interested in
converting XEmacs to a Scheme base, but not with Guile) are developing
techniques for doing static analysis of Lisp code to determine when
dynamic bindings can be replaced with static bindings, to enable the
generated Scheme code to be more efficient.  Their goal is (possibly
expensive) conversion of Lisp libraries into Scheme libraries, which
could then be loaded at run-time as Scheme, or compiled, whatever.

> Or is the idea to have two languages side-by-side in Emacs?  I think
> this side-by-side feature is a strong point of Guile.  But I don't
> know much about Guile, so this could be totally wrong.

If we don't have full run-time translation, we can still use the
existing interpreter, modified to work with the new shared object
representation.  In fact, that's my first goal, having the modified
Lisp interpreter run Emacs, without even having any Scheme code
running at all.  Then get Scheme going, and at least minimal
interaction between the two languages.

I think it would be a mistake to stop there.  The Guile folks have a
decent run-time byte-code generator running, and machine code
generation may be a possibility down the road.  Converting Emacs Lisp
byte codes to C should be pretty easy too, but my impression is that
Scheme lends itself better to certain kinds of optimizations.  (For
example, lexical binding means you can use C local variables instead
of setting and restoring bindings of global variables, and the C
optimizer can probably do a better job with that.)

It'll certainly be worth looking at more closely.  Will it be a
worthwhile change?  That'll depend on a lot of factors.  We may find
that Guile needs a lot of work to bring its speed up to the level we
need.  We may find its memory use to be a problem.  These and other
parts of Guile are being worked on now; it's hard to predict where
they'll stand when we're ready to look at this question.

Re: emacs and guile (Re: ehelp woes, or why I hate a module that I love so much)

[Richard Stallman]

    (BTW, I've got a patch that adds configures option to turn on
    ENABLE_CHECKING, or just the use of a union for Lisp_Object.  Should I
    check it in?

Could you show us what it looks like?

Use of the union type caused the code to be much slower, with typical
Unix compilers in the 80s.  GCC might do a better job with it, but I think
it is an interesting question whether the compiled code for Emacs is as good
when using the union type as it is when using an integer type.  What have
you observed?

Re: emacs and guile (Re: ehelp woes, or why I hate a module that I love so much)

[Ken Raeburn]

>     (BTW, I've got a patch that adds configures option to turn on
>     ENABLE_CHECKING, or just the use of a union for Lisp_Object.  Should I
>     check it in?
>
> Could you show us what it looks like?


A patch file is appended with my current configure.in changes.  The
ENABLE_CHECKING and USE_LISP_UNION_TYPE macros are already tested in
lisp.h.

The CONFIG_NO_DUMP stuff probably isn't of interest, but when I start
playing with Guile again I don't want to have to assume from the start
that libguile is unexec-safe.  Those changes (and a related one in
src/Makefile.in) are just there so I can test the no-unexec case
separately from the Guile work.


> Use of the union type caused the code to be much slower, with typical
> Unix compilers in the 80s.  GCC might do a better job with it, but I think
> it is an interesting question whether the compiled code for Emacs is as good
> when using the union type as it is when using an integer type.  What have
> you observed?

It is still going to be slightly slower than using integer types
anywhere a word-sized union is passed or returned by reference.  I
wouldn't recommend it as a default, because we want all the speed we
can reasonably get, but it's not that painful to use.

I haven't tried any timing tests, but it would be an interesting
experiment to do with modern compilers on various platforms.

I'm also not using it in the Emacs binary I use daily, but I did have
to check to figure that out.  I'll try switching (it's about time I
update to current-CVS again anyways) and see if it's a problem.


Index: configure.in
===================================================================
RCS file: /cvsroot/emacs/emacs/configure.in,v
retrieving revision 1.300
diff -p -u -r1.300 configure.in
--- configure.in	21 Jun 2002 20:57:54 -0000	1.300
+++ configure.in	18 Jul 2002 19:01:15 -0000
@@ -132,6 +132,24 @@ AC_ARG_WITH(xim,
 AC_ARG_WITH(carbon,
 [  --without-carbon        don't use Carbon GUI on Mac OS X])
 
+AC_ARG_ENABLE(checking,
+[  --enable-checking       turn on expensive internal run-time consistency
+                             checks (not recommended unless debugging Emacs)])
+if test "${enable_checking}" = yes ; then
+  AC_DEFINE(ENABLE_CHECKING)
+fi
+AC_ARG_WITH(union-type,
+[  --with-union-type       use union type for lisp object representation
+                             (slows performance, better compile-time checks)])
+if test "${with_union_type}" = yes ; then
+  AC_DEFINE(USE_LISP_UNION_TYPE)
+fi
+AC_ARG_ENABLE(unexec,
+[  --disable-unexec        don't generate "dumped" Emacs with pre-loaded lisp])
+if test "${enable_unexec}" = no ; then
+  AC_DEFINE(CONFIG_NO_DUMP)
+fi
+
 #### Make srcdir absolute, if it isn't already.  It's important to
 #### avoid running the path through pwd unnecessarily, since pwd can
 #### give you automounter prefixes, which can go away.  We do all this
Index: src/config.in
===================================================================
RCS file: /cvsroot/emacs/emacs/src/config.in,v
retrieving revision 1.174
diff -p -u -r1.174 config.in
--- src/config.in	1 May 2002 04:30:59 -0000	1.174
+++ src/config.in	18 Jul 2002 19:01:17 -0000
@@ -916,6 +916,12 @@ extern char *getenv ();
 #define HAVE_X11R6_XIM
 #endif
 
+/* Force use of union type for Lisp_Object?  */
+#undef USE_LISP_UNION_TYPE
+
+/* Disable unexec code?  */
+#undef CONFIG_NO_DUMP
+
 /* Should we enable expensive run-time checking of data types?  */
 #undef ENABLE_CHECKING

Re: emacs and guile (Re: ehelp woes, or why I hate a module that I love so much)

[Andreas Schwab]

Ken Raeburn <raeburn@raeburn.org> writes:

|> Index: configure.in
|> ===================================================================
|> RCS file: /cvsroot/emacs/emacs/configure.in,v
|> retrieving revision 1.300
|> diff -p -u -r1.300 configure.in
|> --- configure.in	21 Jun 2002 20:57:54 -0000	1.300
|> +++ configure.in	18 Jul 2002 19:01:15 -0000
|> @@ -132,6 +132,24 @@ AC_ARG_WITH(xim,
|>  AC_ARG_WITH(carbon,
|>  [  --without-carbon        don't use Carbon GUI on Mac OS X])
|>  
|> +AC_ARG_ENABLE(checking,
|> +[  --enable-checking       turn on expensive internal run-time consistency
|> +                             checks (not recommended unless debugging Emacs)])
|> +if test "${enable_checking}" = yes ; then
|> +  AC_DEFINE(ENABLE_CHECKING)

Please use the optional third argument of AC_DEFINE (the description)
instead of adding entries to src/config.in, since the latter is generated
by autoheader, which copies the description string into the comment.

Andreas.

-- 
Andreas Schwab, SuSE Labs, schwab@suse.de
SuSE Linux AG, Deutschherrnstr. 15-19, D-90429 Nürnberg
Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: emacs and guile (Re: ehelp woes, or why I hate a module that I love so much)

[Ken Raeburn]

Andreas Schwab <schwab@suse.de> writes:
> Please use the optional third argument of AC_DEFINE (the description)
> instead of adding entries to src/config.in, since the latter is generated
> by autoheader, which copies the description string into the comment.

Yes, thanks for pointing that out.  I hadn't updated my patch since
the autoheader work was done.  I've added the description, and removed
ENABLE_CHECKING from the config.in fragment copied literally from
configure.in.

Ken

Re: emacs and guile (Re: ehelp woes, or why I hate a module that I love so much)

[Richard Stallman]

    +AC_ARG_WITH(union-type,
    +[  --with-union-type       use union type for lisp object representation
    +                             (slows performance, better compile-time checks)])

--with options are supposed to be used only for stating whether to use
specific other software packages.  So you can't use --with here.

It looks like there is no standard form of argument that you can use.
I think you should leave this out.

Re: emacs and guile (Re: ehelp woes, or why I hate a module that I love so much)

[Ken Raeburn]

Richard Stallman <rms@gnu.org> writes:
> It looks like there is no standard form of argument that you can use.

Yes, neither --enable nor --with seems appropriate for an option
relevant only to the build process, and invisible to the user.  Well,
there's always 'CC="gcc -DENABLE_CHECKING"'....

> I think you should leave this out.

Okay.

Re: emacs and guile (Re: ehelp woes, or why I hate a module that I love so much)

[Richard Stallman]

    My approach last time around was to add a SCM field to most of the
    "interesting" data types (symbol, misc, and vectorlike), fill it in
    when the object was created, and make XSETFOO just copy the value out.
    Rather crude, and wasteful of storage space, but it got the job done,
    and it minimized the intrusion in random Emacs code and kept it
    running with both object representations.

I don't think this method makes real sense, because if an object is
going to have the scheme object header at all, we may as well use that
object header universaly throughout Emacs as the way to reference the
object.  It would cost no extra storage, and it would be simpler.

The place where having this extra baggage is really unfortunate is for
the small data types--symbol and Lisp_Misc.  Perhaps Scheme needs
a new facility for small user data types.

Why can't Guile's symbols be used as Lisp symbols?

Re: emacs and guile (Re: ehelp woes, or why I hate a module that I love so much)

[Ken Raeburn]

Richard Stallman <rms@gnu.org> writes:
> I don't think this method makes real sense, because if an object is
> going to have the scheme object header at all, we may as well use that
> object header universaly throughout Emacs as the way to reference the
> object.  It would cost no extra storage, and it would be simpler.

It would certainly be simpler, just a little slower as the
object-to-pointer conversions are done multiple times, particularly
when a function called frequently is changed from taking a pointer
argument to taking a lisp object.  I've made a few changes along those
lines, and will almost certainly make more.

> Why can't Guile's symbols be used as Lisp symbols?

Actually, the issue is the symbol's value(s).  Scheme symbols don't
have separate function and value slots.  Emacs Lisp symbols have a few
other fields associated with them, like indirect_variable, that also
don't fit the basic Scheme model.  And if we separate the name from
the values, there's the question of what exactly a symbol is.

There are several ways to deal with this.

Some involve changing Guile's symbol representation to have the extra
fields.  IMNSHO this is tantamount to declaring Scheme not to be up to
the task of implementing Lisp, and I'm not prepared to believe that.
(That Guile isn't currently up to the task, maybe; we'll find out, and
Guile's Scheme implementation can be improved.  That Scheme can't do
it, no.)  It also sets a poor example for people wanting to put other
existing languages on top of Guile.

Another approach would be to use Scheme symbols, but store as their
values vectors containing the various fields of a Lisp symbol.
Special accessor functions would be needed for Scheme code to
manipulate Lisp variables, and while I don't have a problem with that,
the people who want transparent interaction between the two languages
probably won't like it.  I guess it depends whether you view it as
Lisp on top of Scheme, or the two languages working side by side.  I'm
inclined towards the former view, maybe only because it's easier for
an implementor or designer, but I understand the temptation of the
latter for such similar languages.

But in that case, how do you implement Lisp's "symbolp" when looking
at Scheme objects?  Is a Lisp symbol a Scheme symbol that has a
particular kind of object as its value?  Should other Scheme symbols
be treated as Lisp symbols?  If a Scheme symbol has a value, what
happens if you do an "fset" or "let" on it in Lisp?  (Maybe that's an
argument for separate namespaces for Scheme and Lisp symbols, outside
of explicitly requested sharing.)

Treating Lisp symbols as a new type is a third approach.  (Either an
opaque type, or a vector as above but also including the symbol's
name.)  It's what I did before, and it does sidestep a lot of these
issues by simply not allowing any sort of confusion between the two
symbol types, but it's not as appealing as using Scheme symbols.


A related issue is the symbol name -- a Lisp symbol name is a Lisp
string, with some of the associated fields ignored for matching
purposes.  The names themselves can have text properties:

    ELISP> (symbol-name 'booga-booga)
    #("booga-booga" 0 3 nil 3 6
      (some-prop some-val)
      6 11 nil)

(I got this by previously interning a string with the same contents
and some text properties.)

An Emacs Lisp symbol's name can also be changed after interning, such
that it won't be found when interning either the old name or
(probably) the new name.  I don't know what Scheme's specifications
might say about such cases.  (Much as I like Scheme, I'm hardly an
expert; on subtle Scheme issues I'll readily defer to others.)

I really hope no one is relying on such weird behavior, but if it
needs to be maintained, that's another complication.


Because I don't have all the answers, I think it's simplest to plan to
treat them as separate types initially, and then merge them if and
when we decide a particular approach will work well in all cases we
care about, perhaps even before the actual coding starts, but perhaps
not; worst case, we change a few Lisp macros to just invoke Guile
macros.  It's easier than starting coding on the assumption that
they'll be the same, and then finding that we have to separate them.

Ken

Re: emacs and guile (Re: ehelp woes, or why I hate a module that I love so much)

[Stefan Monnier]

> A related issue is the symbol name -- a Lisp symbol name is a Lisp
> string, with some of the associated fields ignored for matching
> purposes.  The names themselves can have text properties:
> 
>     ELISP> (symbol-name 'booga-booga)
>     #("booga-booga" 0 3 nil 3 6
>       (some-prop some-val)
>       6 11 nil)
> 
> (I got this by previously interning a string with the same contents
> and some text properties.)

Actually, this is a recent change, right ?  In Emacs-21.2, `symbol-name'
always returns a string with no properties.  In that case we should
probably also strip the properties when we create a new symbol, instead
of wasting those intervals.

> An Emacs Lisp symbol's name can also be changed after interning, such
> that it won't be found when interning either the old name or
> (probably) the new name.

This is undocumented and I don't know of any code that relies on it
(well, I know of some such code, but not in any elisp package).
I think it should be considered as a misfeature/bug (but I don't
think it's worth fixing unless the fix is to make the output of
`symbol-name' be read-only).

> I don't know what Scheme's specifications
> might say about such cases.  (Much as I like Scheme, I'm hardly an
> expert; on subtle Scheme issues I'll readily defer to others.)

I'm pretty sure that Scheme's symbol-name (if such a function exists)
returns a brand new string.


	Stefan

Re: emacs and guile (Re: ehelp woes, or why I hate a module that I love so much)

[Ken Raeburn]

"Stefan Monnier" <monnier+gnu/emacs@rum.cs.yale.edu> writes:
> Actually, this is a recent change, right ?  In Emacs-21.2, `symbol-name'
> always returns a string with no properties.  In that case we should
> probably also strip the properties when we create a new symbol, instead
> of wasting those intervals.

I don't know about 21.2, but it happens for me with:
 21.1    - at work
 21.1.50 - which I'm using daily at home (I *really* need to update...)
 21.3.50 - built from very recent CVS sources

Are you sure about 21.2?  It would be odd for it to have been fixed
there but not on the trunk.  Here's the whole thing I'm doing to
reproduce it in ielm:

    ELISP> (setq s "booga-booga")
    "booga-booga"
    ELISP> (put-text-property 3 6 'some-prop 'some-val s)
    nil
    ELISP> (setq s2 (intern s))
    booga-booga
    ELISP> (symbol-name 'booga-booga)
    #("booga-booga" 0 3 nil 3 6
      (some-prop some-val)
      6 11 nil)

I do think it makes sense for symbol-name's return value to have no
text properties.

>> An Emacs Lisp symbol's name can also be changed after interning, such
>> that it won't be found when interning either the old name or
>> (probably) the new name.
>
> This is undocumented and I don't know of any code that relies on it
> (well, I know of some such code, but not in any elisp package).
> I think it should be considered as a misfeature/bug (but I don't
> think it's worth fixing unless the fix is to make the output of
> `symbol-name' be read-only).

Maybe if we had read-only strings....

> I'm pretty sure that Scheme's symbol-name (if such a function exists)
> returns a brand new string.

Which would be another difference -- retrieving the name twice would
give two values not "eq?" while in Emacs Lisp they are "eq".  So is
*that* property something that anyone might rely on in their Lisp
code?

Re: emacs and guile (Re: ehelp woes, or why I hate a module that I love so much)

[Stefan Monnier]

> > Actually, this is a recent change, right ?  In Emacs-21.2, `symbol-name'
> > always returns a string with no properties.  In that case we should
> > probably also strip the properties when we create a new symbol, instead
> > of wasting those intervals.
> 
> I don't know about 21.2, but it happens for me with:
>  21.1    - at work

Looks like I was wrong then.  Sorry.

> Are you sure about 21.2?  It would be odd for it to have been fixed
> there but not on the trunk.  Here's the whole thing I'm doing to
> reproduce it in ielm:
> 
>     ELISP> (setq s "booga-booga")
>     "booga-booga"
>     ELISP> (put-text-property 3 6 'some-prop 'some-val s)
>     nil
>     ELISP> (setq s2 (intern s))
>     booga-booga
>     ELISP> (symbol-name 'booga-booga)
>     #("booga-booga" 0 3 nil 3 6
>       (some-prop some-val)
>       6 11 nil)
> 
> I do think it makes sense for symbol-name's return value to have no
> text properties.

I agree, but I'm not sure it matters enough to change `intern's behavior.

> >> An Emacs Lisp symbol's name can also be changed after interning, such
> >> that it won't be found when interning either the old name or
> >> (probably) the new name.
> >
> > This is undocumented and I don't know of any code that relies on it
> > (well, I know of some such code, but not in any elisp package).
> > I think it should be considered as a misfeature/bug (but I don't
> > think it's worth fixing unless the fix is to make the output of
> > `symbol-name' be read-only).
> 
> Maybe if we had read-only strings....

My Emacs' strings are (mostly) read-only (as are its `defconst' variables).
It does introduce some breakage, but the use of `aset' on strings is
surprisingly infrequent.

> > I'm pretty sure that Scheme's symbol-name (if such a function exists)
> > returns a brand new string.
> 
> Which would be another difference -- retrieving the name twice would
> give two values not "eq?" while in Emacs Lisp they are "eq".  So is
> *that* property something that anyone might rely on in their Lisp code?

I don't think I've ever seen code rely on it, but I like it from
a performance point of view.


	Stefan

Re: emacs and guile (Re: ehelp woes, or why I hate a module that I love so much)

[Andreas Schwab]

"Stefan Monnier" <monnier+gnu/emacs@rum.cs.yale.edu> writes:

|> > > I'm pretty sure that Scheme's symbol-name (if such a function exists)
|> > > returns a brand new string.
|> > 
|> > Which would be another difference -- retrieving the name twice would
|> > give two values not "eq?" while in Emacs Lisp they are "eq".  So is
|> > *that* property something that anyone might rely on in their Lisp code?
|> 
|> I don't think I've ever seen code rely on it, but I like it from
|> a performance point of view.

If you care about performance you should eq the symbols, not their names.

Andreas.

-- 
Andreas Schwab, SuSE Labs, schwab@suse.de
SuSE Linux AG, Deutschherrnstr. 15-19, D-90429 Nürnberg
Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: emacs and guile (Re: ehelp woes, or why I hate a module that I love so much)

[Stefan Monnier]

> |> I don't think I've ever seen code rely on it, but I like it from
> |> a performance point of view.
> If you care about performance you should eq the symbols, not their names.

I'm only talking about the performance of `symbol-name'.
And I tend to measure performance in MB or MB/s (where less-is-better)
rather than in Mops/s.


	Stefan

Re: emacs and guile (Re: ehelp woes, or why I hate a module that I love so much)

[Richard Stallman]

    Actually, the issue is the symbol's value(s).  Scheme symbols don't
    have separate function and value slots.  Emacs Lisp symbols have a few
    other fields associated with them, like indirect_variable, that also
    don't fit the basic Scheme model.  And if we separate the name from
    the values, there's the question of what exactly a symbol is.

    There are several ways to deal with this.

    Some involve changing Guile's symbol representation to have the extra
    fields.

If that gives good and clean results, we should use that method, for
efficiency's sake.

      IMNSHO this is tantamount to declaring Scheme not to be up to
    the task of implementing Lisp, and I'm not prepared to believe that.

Our goal is to support Scheme, but this doesn't mean we have to be
purists about how we do it.  We can bring Guile "up to the task" by
adding a facility to add extra slots to a symbol, for
application-specific purposes.  Rather than a kludge for Emacs, this
will be a general Guile feature with wider usefulness.

Re: emacs and guile (Re: ehelp woes, or why I hate a module that I love so much)

[Ken Raeburn]

Richard Stallman <rms@gnu.org> writes:
> Our goal is to support Scheme, but this doesn't mean we have to be
> purists about how we do it.

True, and I'm not saying that changing Guile in a Lisp-specific way is
the wrong approach.  I'm just not convinced it's necessary, and I'm a
bit more hesitant than some others to charge ahead and do it.  If we
can do it cleanly and efficiently in "real" Scheme, I think that
would be preferable.

>    We can bring Guile "up to the task" by
> adding a facility to add extra slots to a symbol, for
> application-specific purposes.  Rather than a kludge for Emacs, this
> will be a general Guile feature with wider usefulness.

I recall thinking once upon a time about using symbol properties.
Guile has object properties in general, implemented through a weak
hash table, but adding a property field to a symbol for performance
may be a reasonable change.  (Though I think a property-list field was
taken *out* of some object type a while back; I don't remember if it
was symbols.)  Then the extra Lisp-related data can be stored as one
or more properties.

Re: emacs and guile (Re: ehelp woes, or why I hate a module that I love so much)

[Marius Vollmer]

Richard Stallman <rms@gnu.org> writes:

>     Some involve changing Guile's symbol representation to have the extra
>     fields.
> 
> If that gives good and clean results, we should use that method, for
> efficiency's sake.

We already have two extra fields on symbols, one for a function and
one for a plist.  Access to these slots is very fast since they are in
symbol header itself.

For additional fields, there is 'make-object-property'.  This function
returns a procedure-with-setter so that can you do, for example,

    (define symbol-foo (make-object-property))

    (symbol-foo 'x)
    => #f
    (set! (symbol-foo 'x) 123)
    (symbol-foo 'x)
    => 123

make-object-property is implemented with a hash table of alists.  If
that is too slow, we could try to speed it up.  For example, the
getter of a property might secretly use the plist of a symbol instead
of the hash table.


But note that in Guile, a symbol does not really have a value.
Symbols are used to name variables relative to some module, and these
variables have values.  We could extend this by having more than one
symbol->variable mapping per module.  For Elisp, there could be two:
one for 'variable values' and one for 'function values'.  We could
then add some primitive syntax to Guile to select a particular
mapping, for example

    Elisp: (foo foo)

could be translated into

    Guile: ((#:func foo) foo)

The form (#:func foo) would use the function mapping when looking up
the variable for foo, while the argument position will use the default
mapping.

The lookup will be done when the code is executed for the first time
(or when it is loaded, should we have a compiler) and subsequent
executions will use the variable directly.

This scheme would allow us to have function values that are also
subject to the module system, just like the normal variable values.

(I have not thought about how well this all fits with dynamic scoping,
tho...)

You might also want to talk to Neil Jerram <neil@ossau.uklinux.net>,
his work on Elisp support has progressed quite far already.

Re: emacs and guile (Re: ehelp woes, or why I hate a module that I love so much)

[Richard Stallman]

    For additional fields, there is 'make-object-property'.  This function
    returns a procedure-with-setter so that can you do, for example,

	(define symbol-foo (make-object-property))

	(symbol-foo 'x)
	=> #f
	(set! (symbol-foo 'x) 123)
	(symbol-foo 'x)
	=> 123

    make-object-property is implemented with a hash table of alists.

If Emacs needs any more than the function and the plist, 
we should provide additional slots in the symbol itself,
or a facility to for an application that uses Guile to request
more of them.

Re: emacs and guile (Re: ehelp woes, or why I hate a module that I love so much)

[Marius Vollmer]

Richard Stallman <rms@gnu.org> writes:

>     For additional fields, there is 'make-object-property'.  This function
>     returns a procedure-with-setter so that can you do, for example,
> 
> 	(define symbol-foo (make-object-property))
> 
> 	(symbol-foo 'x)
> 	=> #f
> 	(set! (symbol-foo 'x) 123)
> 	(symbol-foo 'x)
> 	=> 123
> 
>     make-object-property is implemented with a hash table of alists.
> 
> If Emacs needs any more than the function and the plist, 
> we should provide additional slots in the symbol itself,
> or a facility to for an application that uses Guile to request
> more of them.

Ok.  However, we have only two sizes of Scheme cells: 2 words and 4
words.  (Non-immediate objects are implemented by pointers to such a
cell.)  We can not easily add more slots to the cell of a symbol.
Thus, we will lose some performance for additional slots, as compared
to the function slot, say.

Re: emacs and guile (Re: ehelp woes, or why I hate a module that I love so much)

[Richard Stallman]

    Ok.  However, we have only two sizes of Scheme cells: 2 words and 4
    words.  (Non-immediate objects are implemented by pointers to such a
    cell.)

Most of the Emacs misc objects are 4 words or less.  Can Scheme's
4-word types be used for them?

What part of that 4 words is used for specifying the data type more
precisely?

One very important type, the marker, needs 5 words as currently
defined.  More precisely, it contains a buffer, a chain of markers
pointer, two kinds of buffer positions (integers), and a bit flag.  We
could use a list to record the markers in a buffer instead of chaining
them together; that would cost one extra word per marker, which is ok.
Then the marker would use 3 words and one bit.  Could the 4-word object
be used for this?

An Emacs Lisp symbol has 3 slots of meaning: value, function, and
plist.  It also has its name and the mechanism of interning it; I
suppose Scheme symbols already have something equivalent to that.
Does this mean we don't need any more slots in a symbol?

Emacs Lisp allows multiple obarrays, but we might be able to dispense
with that feature, since (I think) only abbrevs use it.  (Does anyone
know of other uses?)  However, uninterned symbols are used in more
places I think.  Is there anything like an uninterned symbol in
Scheme?

A Lisp symbol also has a couple of flag bits, one for "variable alias"
(which says this variable is an alias for another variable) and one
for "constant" (it is an error to try to set this variable).  There
are a few data types that appear only in the value cell of a symbol,
and indicate special ways to get and set the value.  For instance, one
says "this is a buffer-local variable" and another says "the value is
stored in a certain address."

These flag bits and forwarding values have a lot in common, and we
could implement them with a single mechanism.  But what mechanism?

Re: emacs and guile (Re: ehelp woes, or why I hate a module that I love so much)

[Richard Stallman]

    A related issue is the symbol name -- a Lisp symbol name is a Lisp
    string, with some of the associated fields ignored for matching
    purposes.

I don't think this is a feature.  Perhaps we should make intern
discard text properties (if there are any) in the string it uses as
the symbol's name.

    An Emacs Lisp symbol's name can also be changed after interning, such
    that it won't be found when interning either the old name or
    (probably) the new name.

This is something you should never do; we can consider it as
an undefined case.

Re: ehelp woes, or why I hate a module that I love so much

[Henrik Enberg]

Juanma Barranquero <lektu@terra.es> writes:

> And, with your argument, what's the relative importance of integrating
> Guile, when Emacs already has a fine and tested extension language?
> Why replace something that works, instead of focusing on developing
> new features more oriented towards the *user*, as oposed as the
> developer (who's the only one that will, perhaps, get any improvement
> from Guile)? The hundred of thousands of users implied in your message
> aren't interested in programming Emacs, just using it and customizing
> it (through Customize).

One very important benefit for users if Guile is implemented is that it
is the official extension language for GNU.  A single extension
language would benefit not just Emacs users, but all GNU users.

IMO, one of the biggest weaknesses of the GNU system is the hodge-podge
of extension languages and config-file formats.  Other systems are no
better of course, but that doesn't mean that it isn't a worthwhile goal
for GNU.  And if Emacs started using it, I belive it would give more
momentum for Guile.  

-- 
Booting... /vmemacs.el

Re: ehelp woes, or why I hate a module that I love so much

[Juanma Barranquero]

> One very important benefit for users if Guile is implemented is that it
> is the official extension language for GNU.

And what GNU software is using it right now?

> A single extension
> language would benefit not just Emacs users, but all GNU users.

Right, but still...

> And if Emacs started using it, I belive it would give more
> momentum for Guile.  

Yeah. But the same is true for any other common extension language that
GNU were to use; elisp, for example :-)

(Or, dreaming out loud, Common Lisp).

-- 
Juanma Barranquero <lektu@terra.es>

Re: ehelp woes, or why I hate a module that I love so much

[Per Abrahamsen]

Juanma Barranquero <lektu@terra.es> writes:

> And what GNU software is using it right now?

< http://www.gnu.org/software/guile/gnu-guile-projects.html#Applications >

> Yeah. But the same is true for any other common extension language that
> GNU were to use; elisp, for example :-)

Yes, but they had to pick one.

Re: ehelp woes, or why I hate a module that I love so much

[Richard Stallman]

I agree it would not be fun to work on Emacs if every single step were
planned and controlled.  However, nobody proposed that.  What I am
doing is urging you to try to find a more useful and important task
that does interest you.

Right now most people working on Emacs development pay no attention to
the task list.  Items have been there for years with no one even
trying to work on them.  What it means is that most of the work people
do on Emacs is not going into the places that will help the users.

There must be room between this unfortunate state of affairs and your
extreme fantasy for us to pay more attention to doing tasks that will
make Emacs a better editor for lots of people.  I will continue
pointing out when people are proposing to do a lot of work in a narrow
corner that won't make Emacs much better.