Re: Next pretest, and branching plans

Re: Next pretest, and branching plans

[Jason Rumney]

On 22/02/2010 05:46, Chong Yidong wrote:

> Jason, do you think you could down write some instructions?  You could
> put it at the end of admin/make-tarball.txt, or as its own file in the
> admin/ directory.
>    

I don't have access to bzr at the moment, but here are the steps required:

1. Download the source tarball, check signature and unpack.
2. Make sure you have an up to date bzr source tree to use for the admin 
directory.
3. "make install" to create the bin subdirectory with the appropriate files.
4. Copy libxpm.dll from a previous tarball into the bin directory.
5. Check the file admin/nt/README.W32 (from bzr tree) for necessary changes.
6. Make sure 7z.exe is in your PATH, or modify admin/nt/makedist.bat to 
use your preferred zip utility (I found that InfoZip zip.exe does not 
work well, which is why I changed to use 7z.exe)
7. Run admin/nt/makedist.bat. With no args it will give you help on usage.
8. Test the resulting zip files.

Then the process of signing and uploading begins. This is documented in 
the GNU maintainers manual, and is probably the more difficult part of 
the process, particularly if your internet connection is not fast and 
stable and uploads get interrupted.

Re: Next pretest, and branching plans

[Eli Zaretskii]

> Date: Tue, 23 Feb 2010 13:31:25 +0800
> From: Jason Rumney <jasonr@gnu.org>
> CC: Christoph <cschol2112@googlemail.com>, eliz@gnu.org, 
>  emacs-devel@gnu.org
> 
> 6. Make sure 7z.exe is in your PATH, or modify admin/nt/makedist.bat to 
> use your preferred zip utility (I found that InfoZip zip.exe does not 
> work well, which is why I changed to use 7z.exe)

What didn't work well with zip.exe?

> 8. Test the resulting zip files.

What tests did you usually run?

> Then the process of signing and uploading begins. This is documented in 
> the GNU maintainers manual, and is probably the more difficult part of 
> the process, particularly if your internet connection is not fast and 
> stable and uploads get interrupted.

It also requires to register a GPG signature, doesn't it?

Re: Next pretest, and branching plans

[Jason Rumney]

On 24/02/2010 02:29, Eli Zaretskii wrote:
>> 6. Make sure 7z.exe is in your PATH, or modify admin/nt/makedist.bat to
>> use your preferred zip utility (I found that InfoZip zip.exe does not
>> work well, which is why I changed to use 7z.exe)
>>      
> What didn't work well with zip.exe
>    

rem Info-ZIP zip seems to be broken on Windows.
rem It always writes to zip.zip and treats the zipfile argument as one
rem of the files to go in it.
rem zip -9 -r %2-bin-i386 emacs-%1/BUGS emacs-%1/COPYING emacs-%1/README emacs-%1/README.W32 emacs-%1/INSTALL emacs-%1/bin emacs-%1/etc emacs-%1/info emacs-%1/lisp emacs-%1/leim -x emacs.mdp *.pdb *.opt *~ CVS


>> 8. Test the resulting zip files.
>>      
> What tests did you usually run?
>    
Test that the unpacked Emacs runs, along with the other exes in the bin 
directory, and check that the number of files in the zip file is in the 
same ballpark as the previous version.


> It also requires to register a GPG signature, doesn't it?
>    

Yes, your public key needs to be in the GNU keyring to upload to the FTP 
server.

Re: Next pretest, and branching plans

[Richard Stallman]

Have the manuals been updated?  In a release, the manuals should always
be up to date.

Re: Next pretest, and branching plans

[Sean Sieger]

[-- Attachment #1: Type: text/plain, Size: 231 bytes --]

Richard Stallman <rms@gnu.org> writes:

    Have the manuals been updated?  In a release, the manuals should always
    be up to date.

I did

emacs -Q --eval "(setq Info-default-directory-list '(\".\"))" \ -f
info-xref-check-all


[-- Attachment #2: manual xrefs --]
[-- Type: application/octet-stream, Size: 2217 bytes --]

In file /usr/local/texlive/2009/texmf/doc/info/dvips.info:
No such node: (kpathsea)Bugs
No such node: (kpathsea)unixtex
No such node: (kpathsea)Bugs
No such node: (kpathsea)Bugs
No such node: (kpathsea)Path specifications
No such node: (kpathsea)MakeTeX script arguments
No such node: (kpathsea)unixtex
No such node: (kpathsea)unixtex
No such node: (kpathsea)unixtex
No such node: (kpathsea)unixtex
In file /usr/local/texlive/2009/texmf/doc/info/fontname.info:
No such node: (dvips)psfonts
In file /usr/local/texlive/2009/texmf/doc/info/kpathsea.info:
Not available to check: (fontu)
Not available to check: (bash)
Not available to check: (autoconf)
Not available to check: (coreutils)
No such node: (web2c)inimf invocation
Not available to check: (gcc)
No such node: (web2c)dmp invocation
No such node: (web2c)DMP invocation
In file /usr/local/texlive/2009/texmf/doc/info/web2c.info:
No such node: (kpathsea)unixtex
No such node: (kpathsea)Copying
No such node: (kpathsea)unixtex
No such node: (kpathsea)Bugs
Not available to check: (autoconf)
Not available to check: (libc)
No such node: (kpathsea)unixtex
No such node: (dvips)psfonts
No such node: (kpathsea)unixtex
No such node: (kpathsea)unixtex
No such node: (kpathsea)unixtex
In file /home/sean/trunk/info/ccmode:
Not available to check: (indent)
In file /home/sean/trunk/info/eintr:
Not available to check: (texinfo)
In file /home/sean/trunk/info/elisp:
Not available to check: (coreutils)
Not available to check: (libc)
In file /home/sean/trunk/info/emacs:
Not available to check: (aspell)
Not available to check: (diff)
Not available to check: (make)
Not available to check: (gdb)
Not available to check: (mailutils)
In file /home/sean/trunk/info/epa:
Not available to check: (gnupg)
In file /home/sean/trunk/info/eudc:
Not available to check: (bbdb)
In file /home/sean/trunk/info/info:
Not available to check: (info-stnd)
Not available to check: (texinfo)
In file /home/sean/trunk/info/message:
Not available to check: (gnupg)
In file /home/sean/trunk/info/pgg:
Not available to check: (gnupg)
In file /home/sean/trunk/info/reftex:
Not available to check: (auctex)
In file /home/sean/trunk/info/tramp:
Not available to check: (bbdb)
done, 563 good, 23 bad

[-- Attachment #3: Type: text/plain, Size: 50 bytes --]


And I'm trying to figure out what to do next ...

Re: Next pretest, and branching plans

[Davis Herring]

> And I'm trying to figure out what to do next ...

The first 34 lines are unrelated to Emacs.  All the error messages that
are for Emacs are saying you're missing other Info files (like, say, those
for `make') and so it can't check that the cross-references in the Emacs
manual point to extant pages in those other manuals.  In short, you've not
found anything wrong, but there were lots of things you couldn't verify.

Davis

-- 
This product is sold by volume, not by mass.  If it appears too dense or
too sparse, it is because mass-energy conversion has occurred during
shipping.

Re: Next pretest, and branching plans

[Sean Sieger]

    The first 34 lines are unrelated to Emacs.  All the error messages that
    are for Emacs are saying you're missing other Info files (like, say, those
    for `make') and so it can't check that the cross-references in the Emacs
    manual point to extant pages in those other manuals.  In short, you've not
    found anything wrong, but there were lots of things you couldn't verify.

    Davis

Thank you for your time and thought, Davis.

Re: Next pretest, and branching plans

[Chong Yidong]

Richard Stallman <rms@gnu.org> writes:

> Have the manuals been updated?  In a release, the manuals should always
> be up to date.

There are still a few items in NEWS that have not yet been documented,
but we are mostly there.

Re: Next pretest, and branching plans

[Richard Stallman]

    There are still a few items in NEWS that have not yet been documented,
    but we are mostly there.

To properly update the manuals calls for more than just to document
the new features.  That is the first step.

The next step is to reread the whole text looking for statements that
have become outdated or incorrect, and fix them.

The next step is to ask other people to do likewise, looking for
double coverage.

manuals [was Re: Next pretest, and branching plans]

[Glenn Morris]

Richard Stallman wrote:

> The next step is to reread the whole text looking for statements that
> have become outdated or incorrect, and fix them.
>
> The next step is to ask other people to do likewise, looking for
> double coverage.

Emacs 23.2 is to 23.1 as 22.2 was to 22.1: largely a bug-fix release,
with a few stand-alone new features.

I seem to recall that for Emacs 22.x, the manuals were thoroughly
checked in the way you describe only before 22.1. After that, they
were only updated were needed.

The manuals were checked quite well before 23.1, so following the same
policy as for Emacs 22, and only updating them where needed, seems
fine to me.

The big changes in 23.2 are CEDET, which has its own manuals; and the
change from mail-mode to message-mode. I do think the latter will
affect quite a bit of documentation.

Re: manuals [was Re: Next pretest, and branching plans]

[Richard Stallman]

    The manuals were checked quite well before 23.1, so following the same
    policy as for Emacs 22, and only updating them where needed, seems
    fine to me.

If this was done thoroughly for 23.1, I agree there is no need to
repeat it now.

Re: Next pretest, and branching plans

[Johan Bockgård]

Chong Yidong <cyd@stupidchicken.com> writes:

> Richard Stallman <rms@gnu.org> writes:
>
>> Have the manuals been updated?  In a release, the manuals should always
>> be up to date.
>
> There are still a few items in NEWS that have not yet been documented,
> but we are mostly there.

The Emacs Lisp manual hasn't been updated with 30-bit integers (only the
Emacs manual has (max buffer size)).

2.3.1 Integer Type
3.1 Integer Basics
3.8 Bitwise Operations on Integers

Re: Next pretest, and branching plans

[Glenn Morris]

Johan Bockgård wrote:

> The Emacs Lisp manual hasn't been updated with 30-bit integers (only the
> Emacs manual has (max buffer size)).
>
> 2.3.1 Integer Type
> 3.1 Integer Basics
> 3.8 Bitwise Operations on Integers

Thanks, I tried to update this. It would be great if you could check it.

(Please make a bug report for anything else that is incorrectly marked
as having being updated.)

Re: Next pretest, and branching plans

[Johan Bockgård]

Glenn Morris <rgm@gnu.org> writes:

> Johan Bockgård wrote:
>
>> The Emacs Lisp manual hasn't been updated with 30-bit integers (only the
>> Emacs manual has (max buffer size)).
>>
>> 2.3.1 Integer Type
>> 3.1 Integer Basics
>> 3.8 Bitwise Operations on Integers
>
> Thanks, I tried to update this. It would be great if you could check it.

Looks fine. However, there is another small complication.

In 2.3.1 Integer Type:

> 1073741825       ; Also the integer 1 on a 30-bit implementation.

In 3.1 Integer Basics:

> 1073741825      ; Also the integer 1, due to overflow.

When in fact (type-of 1073741825) => float!

Is this even documented?

integer overflow [was Re: Next pretest, and branching plans]

[Glenn Morris]

Johan Bockgård wrote:

> In 2.3.1 Integer Type:
>
>> 1073741825       ; Also the integer 1 on a 30-bit implementation.
>
> In 3.1 Integer Basics:
>
>> 1073741825      ; Also the integer 1, due to overflow.
>
> When in fact (type-of 1073741825) => float!
>
> Is this even documented?

A good question. I can't see anything about this in NEWS.
Might it be the following change?

2009-08-27  Stefan Monnier  <monnier at iro.umontreal.ca>

    * lread.c (read_integer): Use doubles (and potentially
    return a float number) as we do in string-to-number.
    (read1): Use strtol to read integers, signal errors on
    strtol's overflow and use floats if strtol's output is too large for
    Elisp integers.

Re: integer overflow

[Stefan Monnier]

>> In 2.3.1 Integer Type:
>>> 1073741825       ; Also the integer 1 on a 30-bit implementation.
>> In 3.1 Integer Basics:
>>> 1073741825      ; Also the integer 1, due to overflow.
>> When in fact (type-of 1073741825) => float!
>> Is this even documented?
> A good question.  I can't see anything about this in NEWS.
> Might it be the following change?

> 2009-08-27  Stefan Monnier  <monnier at iro.umontreal.ca>

>     * lread.c (read_integer): Use doubles (and potentially
>     return a float number) as we do in string-to-number.
>     (read1): Use strtol to read integers, signal errors on
>     strtol's overflow and use floats if strtol's output is too large for
>     Elisp integers.

Indeed, I thought about it as a pure bug-fix, but it needs to be added
to the NEWS.  Can you do that and update the doc accordingly?


        Stefan

Re: integer overflow

[Glenn Morris]

Stefan Monnier wrote:

> Indeed, I thought about it as a pure bug-fix, but it needs to be added
> to the NEWS.  Can you do that and update the doc accordingly?

I don't really understand it. In a 32-bit Emacs:

(1+ 536870911) -536870912
(+ 2 536870911) -536870911  
(1+ 536870912)  536870913.0
(1+ 4294967295) overflow
(1+ 4294967295.0) 4294967296.0

Can you summarize how it's supposed to work?

Re: integer overflow

[Chong Yidong]

Glenn Morris <rgm@gnu.org> writes:

> I don't really understand it. In a 32-bit Emacs:
>
> (1+ 536870911) -536870912
> (+ 2 536870911) -536870911
> (1+ 536870912)  536870913.0
> (1+ 4294967295) overflow
> (1+ 4294967295.0) 4294967296.0
>
> Can you summarize how it's supposed to work?

The change affects the Lisp reader: reading "536870912" or any larger
number now gives you a float Lisp object, instead of an integer Lisp
object (since there is no way to represent 536870912 or larger as
integers).  536870911 and below are still read as integers.

The change does not prevent the possibility of overflowing integers, as
your example shows.

Re: integer overflow

[Helmut Eller]

* Chong Yidong [2010-03-06 04:11+0100] writes:

>> Can you summarize how it's supposed to work?
>
> The change affects the Lisp reader: reading "536870912" or any larger
> number now gives you a float Lisp object, instead of an integer Lisp
> object (since there is no way to represent 536870912 or larger as
> integers).  536870911 and below are still read as integers.

What's the point of doing that?  I would much prefer if read would just
raise and error.

Helmut

Re: integer overflow

[Stefan Monnier]

>>> Can you summarize how it's supposed to work?
>> The change affects the Lisp reader: reading "536870912" or any larger
>> number now gives you a float Lisp object, instead of an integer Lisp
>> object (since there is no way to represent 536870912 or larger as
>> integers).  536870911 and below are still read as integers.
> What's the point of doing that?  I would much prefer if read would just
> raise an error.

There are many cases where it's important to read an integer from some
other program, so it's important to be able to read integers larger the
Elisp's largest fixnum (tho less so for integers larger than the
host system's natural "int").
Examples are file sizes, process ids, user ids, ...


        Stefan

Re: integer overflow

[Davis Herring]

>> What's the point of doing that?  I would much prefer if read would just
>> raise an error.
>
> There are many cases where it's important to read an integer from some
> other program, so it's important to be able to read integers larger the
> Elisp's largest fixnum (tho less so for integers larger than the
> host system's natural "int").
> Examples are file sizes, process ids, user ids, ...

It seems to me that such cases are relevant to `string-to-*' more than to
`read'.   There is value in having them be consistent in their shared
domain, but there is also value in having the syntax of elisp not involve
magic thresholds where the type of a literal changes.

Davis

-- 
This product is sold by volume, not by mass.  If it appears too dense or
too sparse, it is because mass-energy conversion has occurred during
shipping.

RE: integer overflow

[Drew Adams]

> There are many cases where it's important to read an integer
> from some other program, so it's important to be able to read
> integers larger the Elisp's largest fixnum (tho less so for
> integers larger than the host system's natural "int").
> Examples are file sizes, process ids, user ids, ...

Lisp (MacLisp) invented bignums 40+ years ago. Maybe it's time...

http://en.wikipedia.org/wiki/Bignum

Re: integer overflow

[Richard Stallman]

I have heard that Guile now supports Emacs Lisp.  And it has bignums.
So if we make Emacs use Guile, that will automatically provide
bignums.

Re: integer overflow

[David Kastrup]

Richard Stallman <rms@gnu.org> writes:

> I have heard that Guile now supports Emacs Lisp.  And it has bignums.
> So if we make Emacs use Guile, that will automatically provide
> bignums.

My first reaction to that would be "Whoa, whoa, whoa!".

The second that "librep" might be an easier starting point for migrating
the Lisp engine.  After all, it was designed to be like Emacs Lisp.
From glancing over the wording of the manual, it appears to be a one-man
project.  Given his motivation of imitating Emacs Lisp, it is
conceivable that he would not be averse to copyright assignment.

Anyway: supplanting Elisp is not exactly a small-hearted task.  Even
making the lexbind branch the default would be quite some change (being
able to write lexical closured would be nice, though).

-- 
David Kastrup

Re: integer overflow

[Stephen J. Turnbull]

David Kastrup writes:
 > Richard Stallman <rms@gnu.org> writes:
 > 
 > > I have heard that Guile now supports Emacs Lisp.  And it has bignums.
 > > So if we make Emacs use Guile, that will automatically provide
 > > bignums.
 > 
 > My first reaction to that would be "Whoa, whoa, whoa!".
 > 
 > The second that "librep" might be an easier starting point for migrating
 > the Lisp engine.

For bignums, XEmacs or SXEmacs might be easier yet.  The work of
integrating bignums (GNU MP and BSD MP are both supported as
compile-time options) was done by Jerry James.  He is probably *not*
available any time soon (his day job just turned into a day-and-night
job) but the pedigree of the code is easily established and he and
other contributors probably would be willing to sign papers.

I believe SXEmacs supports quaternions and Gaussian integers, and
perhaps even more exotic number systems. :-)  However, I have no idea
whether the SXEmacs folks would be willing to assign their
contributions.

However, this is all kind of irrelevant.  Using bignums as pure
numbers is relatively easy.  The hard part is introducing them as
numerical components of internal Emacs structures (buffer size and
positions, markers, overlays, text properties), and integrating them
with niceties like (system-dependent) large file support, etc.  This
is in no way an automatic consequence of language support for bignums.

Re: integer overflow

[David Kastrup]

"Stephen J. Turnbull" <turnbull@sk.tsukuba.ac.jp> writes:

> David Kastrup writes:
>  > Richard Stallman <rms@gnu.org> writes:
>  > 
>  > > I have heard that Guile now supports Emacs Lisp.  And it has bignums.
>  > > So if we make Emacs use Guile, that will automatically provide
>  > > bignums.
>  > 
>  > My first reaction to that would be "Whoa, whoa, whoa!".
>  > 
>  > The second that "librep" might be an easier starting point for migrating
>  > the Lisp engine.
>
> For bignums, XEmacs or SXEmacs might be easier yet.

But the "Whoa" factor is lost.

> The work of integrating bignums (GNU MP and BSD MP are both supported
> as compile-time options) was done by Jerry James.  He is probably
> *not* available any time soon (his day job just turned into a
> day-and-night job) but the pedigree of the code is easily established
> and he and other contributors probably would be willing to sign
> papers.

[...]

> However, this is all kind of irrelevant.  Using bignums as pure
> numbers is relatively easy.  The hard part is introducing them as
> numerical components of internal Emacs structures (buffer size and
> positions, markers, overlays, text properties), and integrating them
> with niceties like (system-dependent) large file support, etc.  This
> is in no way an automatic consequence of language support for bignums.

IIRC, this has not been done in the bignum integration of XEmacs and its
ilk.  So there is not much code worth the taking.

Transparent bignums should indeed be easy to do: just check the overflow
of integer arithmetic operations and behave accordingly.  Arithmetic now
catering for float and integer would have to cater for bignums as well.

Anyway, I guess that Richard's point more or less was that Emacs' Lisp
engine was worth substituting, and that such a radical move might solve
unrelated problems as well.

bignum support alone indeed should be reasonably straightforward.  It
should certainly speed up Emacs calc once Emacs calc is cleaned up to
use it.

-- 
David Kastrup

Re: integer overflow

[Jay Belanger]

David Kastrup <dak@gnu.org> writes:
...
> bignum support alone indeed should be reasonably straightforward.  It
> should certainly speed up Emacs calc once Emacs calc is cleaned up to
> use it.

Yes, and if Emacs gets bignum support it shouldn't take long for Calc to
use it.  (Oddly enough, while he might have written Calc anyhow, being
able to work with big integers was one of Dave Gillespie's motivations
to write Calc.)

Jay

Re: integer overflow

[Richard Stallman]

    The second that "librep" might be an easier starting point for migrating
    the Lisp engine.  After all, it was designed to be like Emacs Lisp.

If it only supports something like Emacs Lisp then it is not much of
an advance.  Guile supports Scheme and Javascript as well as Emacs Lisp.

Guile in Emacs (was: integer overflow)

[Ted Zlatanov]

On Mon, 08 Mar 2010 03:04:40 -0500 Richard Stallman <rms@gnu.org> wrote: 

RS>     The second that "librep" might be an easier starting point for migrating
RS>     the Lisp engine.  After all, it was designed to be like Emacs Lisp.

RS> If it only supports something like Emacs Lisp then it is not much of
RS> an advance.  Guile supports Scheme and Javascript as well as Emacs Lisp.

It also supports multithreading, which IMHO is a big deal as well,
perhaps more important than bignums.

Thread subject changed in the hopes that the topic change won't be lost.

Ted

Re: Guile in Emacs

[Ted Zlatanov]

On Mon, 08 Mar 2010 08:41:53 -0600 Ted Zlatanov <tzz@lifelogs.com> wrote: 

TZ> On Mon, 08 Mar 2010 03:04:40 -0500 Richard Stallman <rms@gnu.org> wrote: 
RS> The second that "librep" might be an easier starting point for migrating
RS> the Lisp engine.  After all, it was designed to be like Emacs Lisp.

RS> If it only supports something like Emacs Lisp then it is not much of
RS> an advance.  Guile supports Scheme and Javascript as well as Emacs Lisp.

TZ> It also supports multithreading, which IMHO is a big deal as well,
TZ> perhaps more important than bignums.

By "it" I meant Guile, not librep.  librep does not (based on the
manual) support multithreading.

Ted

Re: Guile in Emacs (was: integer overflow)

[Chad Brown]

Ken Raeburn has done some work on Guile in Emacs, which might be helpful.  There's a (not super-current) status page here:

  http://www.mit.edu/~raeburn/guilemacs/

Summary: there is something runable, but it's certainly not finished.  There are some design questions that I think it's safe to say are still in an `exploratory' phase.  Ken has given some thought to threading (not a surprise if you know Ken), but as of the middle of last year (the date on the status update), I would characterize his progress as `thoughts and questions'.

Hope this helps,
*Chad

Re: Guile in Emacs (was: integer overflow)

[Ken Raeburn]

On Mar 8, 2010, at 13:42, Chad Brown wrote:
> Ken Raeburn has done some work on Guile in Emacs, which might be helpful.  There's a (not super-current) status page here:
> 
>  http://www.mit.edu/~raeburn/guilemacs/

Yeah, I haven't updated that in a while...

> Summary: there is something runable, but it's certainly not finished.  There are some design questions that I think it's safe to say are still in an `exploratory' phase.

Yes, though I think many of the open questions can be put off until what I might call "phase 2" ... where "phase 1" just deals with basic object representations, allocation and GC, and keeps the current single-threaded Lisp engine.  I want that to be on solid footing before pushing other stuff forward too much, if only so there's something to show off. :-)

So after "phase 1", I think it would be time to assess how well things work.  Is work in certain areas of Guile needed?  (If so, go hack on Guile a while and come back later.)  Does it do some things better than stock Emacs?  Is it time to discuss whether it would be suitable for integration into Emacs 24/25/26/whatever?  What else needs to happen first?

>  Ken has given some thought to threading (not a surprise if you know Ken), but as of the middle of last year (the date on the status update), I would characterize his progress as `thoughts and questions'.

Yeah, threads bring up lots of interesting issues, and there's enough stuff going on in this Guile project even without having to settle those issues right away.  (Even though my original impetus was wanting to do work in Emacs while Gnus was crunching away on newsgroup data in the same process!)  Other people are already working on related projects (Tom Tromey's work on the Emacs side, and Daniel Kraft and others working on a Lisp reader for Guile) where some of the issues may get hashed out.  Though, I ought to be paying more attention to them....


I haven't made a lot of direct progress on the project since then.  Some amount of my effort has gone into indirectly-helpful sub-projects, like making upstream Emacs work better without doing the unexec step, so that I don't have to make Guile and everything it might link against unexec-friendly, deal with shared libraries in unexec, make static versions of Guile and N other libraries, etc.  (Obviously we don't want Emacs to take forever to start up for end users, but if we don't get another solution for that from the Guile side, we can work on the unexec bits later, and sidestepping that problem for now lets me focus on other things.)  Another one that's appeared on the radar is digging up Dave Love's old work on making Emacs work with Boehm's GC library, which Guile uses, and Emacs-with-Guile would wind up using as a result, but I haven't even started with that one; currently I think I've just had Emacs leak memory since Guile switched to libgc.

(And then there's the new job, which is sucking up a lot of my time now.  Most of the work seems to get done in bursts every several years, when I'm between jobs; then I get a job, or there's some VCS breakage, and the work stops for a while....  I'm hoping to put more time in on it soon, especially since Andy Wingo surprised me with a proposed timeline for finishing this work in his Guile presentation at GHM '09, and I'd like to meet it, but not by going jobless again. :-)

Ken

Re: Guile in Emacs

[Ted Zlatanov]

On Tue, 9 Mar 2010 02:07:11 -0500 Ken Raeburn <raeburn@raeburn.org> wrote: 

KR> On Mar 8, 2010, at 13:42, Chad Brown wrote:

>> Ken has given some thought to threading (not a surprise if you know
>> Ken), but as of the middle of last year (the date on the status
>> update), I would characterize his progress as `thoughts and
>> questions'.

KR> Yeah, threads bring up lots of interesting issues, and there's
KR> enough stuff going on in this Guile project even without having to
KR> settle those issues right away.  (Even though my original impetus
KR> was wanting to do work in Emacs while Gnus was crunching away on
KR> newsgroup data in the same process!)  Other people are already
KR> working on related projects (Tom Tromey's work on the Emacs side,
KR> and Daniel Kraft and others working on a Lisp reader for Guile)
KR> where some of the issues may get hashed out.  Though, I ought to be
KR> paying more attention to them....

Make sure you check out the emacs-mt work too at
http://gitorious.org/emacs-mt as that's (AFAIK) the closest to getting
into Emacs.  Giuseppe Scrivano has done great work.

Have you considered continuations support?  I didn't see mention of them
in the docs.

Ted

Re: Guile in Emacs

[Ken Raeburn]

On Mar 9, 2010, at 11:22, Ted Zlatanov wrote:
> Make sure you check out the emacs-mt work too at
> http://gitorious.org/emacs-mt as that's (AFAIK) the closest to getting
> into Emacs.  Giuseppe Scrivano has done great work.

Ah, yes, I thought I might be forgetting someone.  (Sorry!)
So many to keep track of... that's a *good* thing! :-)

> Have you considered continuations support?  I didn't see mention of them
> in the docs.

Not even remotely, no. :-)  I'm not particularly familiar with continuations in Scheme, let alone how you'd map that into elisp, or if the Lisp world has a similar construct....

Ken

Re: Guile in Emacs

[David Kastrup]

Ken Raeburn <raeburn@raeburn.org> writes:

> On Mar 9, 2010, at 11:22, Ted Zlatanov wrote:
>> Make sure you check out the emacs-mt work too at
>> http://gitorious.org/emacs-mt as that's (AFAIK) the closest to getting
>> into Emacs.  Giuseppe Scrivano has done great work.
>
> Ah, yes, I thought I might be forgetting someone.  (Sorry!)
> So many to keep track of... that's a *good* thing! :-)
>
>> Have you considered continuations support?  I didn't see mention of them
>> in the docs.
>
> Not even remotely, no. :-) I'm not particularly familiar with
> continuations in Scheme, let alone how you'd map that into elisp, or
> if the Lisp world has a similar construct....

Basically the idea (Chicken even implements it like that) is that you
never clean up your return stack, but treat it like a garbage-collected
heap.

Procedures are called implicitly with a pointer to the return stack.
When a procedure returns, it basically "calls" this pointer.  Now
call/cc does the same, but passes this pointer also _explicitly_ to the
procedure in question, which may store it someplace.  If at some later
point of time somebody calls this from wherever it may have been stored,
Scheme in effect returns "again" (if it did previously return) or for
the first time, abandoning the control flow it happened to be in at that
time (unless you stored its continuation somewhere else, of course).

-- 
David Kastrup

Re: Guile in Emacs (was: integer overflow)

[Richard Stallman]

    RS> If it only supports something like Emacs Lisp then it is not much of
    RS> an advance.  Guile supports Scheme and Javascript as well as Emacs Lisp.

    It also supports multithreading, which IMHO is a big deal as well,
    perhaps more important than bignums.

I think the support for multiple languages is the biggest advance.
I hope support for other popular extension languages will be implemented
eventually -- that was the original goal of Guile.

Re: Guile in Emacs (was: integer overflow)

[Thomas Lord]

I'd like to correct some accounts of history.

On Mon, 2010-03-08 at 22:19 -0500, Richard Stallman wrote:
> RS> If it only supports something like Emacs Lisp then it is not much of
>     RS> an advance.  Guile supports Scheme and Javascript as well as Emacs Lisp.
> 
>     It also supports multithreading, which IMHO is a big deal as well,
>     perhaps more important than bignums.
> 
> I think the support for multiple languages is the biggest advance.
> I hope support for other popular extension languages will be
> implemented
> eventually -- that was the original goal of Guile.

Your recollection there seems to me to be mistaken.  It
was a long time ago but I distinctly remember things
differently.  As I recall:

The original GNU Manifesto described a unix-like foundation
with a heavy emphasis on a Lisp-centric user-space.

In the early 1990s, when I worked at the FSF, several
of the hackers there (including me) understood the GNU
vision to imply that many interactive programs - not just
Emacs - would be extensible in a dialect of Lisp.  We
mostly shared the opinion that Scheme was an appropriate
dialect.   

Before I got to the FSF people had already started on a 
GNU Extension Language Library - an embeddable Scheme 
interpreter.   As the (bad) "joke" goes: they code-named the 
project "revoc" but upon reflection that's just a "cover".  
(Get it?  Spooky, ain't it?)


Revoc was (at least back then) going nowhere fast.  Later,
while working on a spreadsheet program, I attempted to start
a new attempt at a scheme-based extension language library.
You were (rightly, I suppose) upset that there were higher
priorities.  You also pointed out that I was probably wasting
time by starting from scratch and that I ought to have started
from some existing Scheme implementation.  I don't recall
if you specifically suggested SCM but I think you might have.
So there were two attempts at a Scheme-based GNU extension
library down - and one to go.

A year or two later I went to work for Cygnus and we, there,
initially agreed to invest in making a scheme-based extension
language library for GNU programs.  This was originally named
"GEL" (for GNU Extension Language) until the company lawyers 
warned that "GEL" was a trademark for another program.  It
was renamed Guile (a cognitive relative of "scheme" and a 
pun for "Guy L.").   You were informed of this work and were
encouraging.   I forget at which stage of this process you 
declared Guile to be a "GNU project" but I'm sure the extant
public record has it somewhere.

Around that time, Sun was beginning to announce and promote 
Java.  They also hired John Ousterhout and began declaring 
"Tcl is to be the scripting language for the Internet!"  Back
then, Sun was a particularly influential company.   On a 
technical level, Tcl was a horribly regressive language - 
a giant step backwards in many subtle but important ways.
Its license was a free software license but it was uncomforable
at the time to have Sun pushing it so heavily because of 
the technical problems and because of doubts about Sun's
motives and control over the software.

A faction arose within Cygnus that argued that Cygnus should
divest from the Guile project and adopt Tcl instead.  I made
you aware of that and of Sun's swagger, regarding Tcl.

Around the same time, two of Prof. Ousterhout's graduate 
students, John Blow and Adam Sah, were proposing that
the TCL language be *altered* in some small but significant
ways that would facilitate its translation into reasonable
efficient Scheme code.  Their altered version of Tcl was
largely but not completely upward compatible with standard
Tcl.  By applying a Scheme->C compiler to the generated 
Scheme code, they were able to benchmark far better than 
standard Tcl in many cases.

And around that same time I was independently proposing
similar things.   Mssrs. Blow, Sah, and I compared notes
and so forth.

Around that time, also, Python was beginning to gain 
recognition in the US (it already had a foothold in Europe).
Blow, Sah, and I had dinner with Von Rossum and tried to 
persuade him to add LAMBDA and proper tail calls and perhaps
continuations to the language so that it could be usefully
translated to Scheme.  He did not agree.  Back at Cygnus,
the oppositional factions had fun lampooning lisp syntax
and pointing to examples like Python of what was better.

Some of my notes to you mysteriously transmogrified into
a USENET post which the archives show as having you in the
"From:" line and with my signature at the bottom of the
message.  This sparked what people, to this day,
still call "The TCL war".

So:

The "original goal" of Guile was most definitely *not*
to support multiple languages.   Rather, that was a goal
that was established as a tactic in response to early
competition with Guile.

Moreover, it was *never*, in the early days, a goal
that Guile support other popular extension languages.
That was something that you added and that I (regrettably)
agreed to in response to the perceived threat of Tcl
and to a lesser extent Python.

And: it was *never* in those years a goal to support
any of those other languages exactly.  It was *always*
a goal to have a Tcl-like syntax, a Python-like syntax,
and a C-like syntax for Guile Scheme.  But tight 
compatibility with those languages was *not* the goal.
The notion wasn't so much to "support Tcl" (or any other
language) as to have a hyper-flexible syntax and to 
support, via libraries, environments with all of the 
convenience features of a shell-like language like Tcl.

Early on after we adopted that tactic you and I and
a few others had some lengthy discussions about adding
Emacs Lisp to the list of languages that Guile could
run.  We ran into some severe issues such as the 
difference between 'NIL in Emacs lisp and '() and Scheme.
Perhaps not to you but to several other of us it became
fairly clear, back then, that a strictly compatible 
Emacs lisp would never sit comfortably alongside a 
proper Scheme environment.  Just as we would need to make
a "Tcl-like" language that would break some Tcl code,
we would need to break some Emacs lisp code, if indeed
we ultimately wanted to bother trying to support any.
(That is part of why, at Cygnus, I built a from-scratch
multi-buffer, self-documenting, extensible text editor
in Scheme with multi-font and proportionally-spaced
font capabilities that, at the time, GNU Emacs wasn't
close to achieving.  The notion was to see how far I could
get just leapfrogging over GNU Emacs instead of trying 
to retrofit it.)

Now, years have passed.  In recent years, I gather, 
the implementation of Guile has been fairly radically
altered so that it is now more of a bytecode VM with
primitive LISP-ish types.   In some sense, it has begun
to drift away from being primarily a Scheme to being
more in the category of JVM or the Mono VM.   It is easier
to target multiple languages to such a VM but no less
easy to make them interoperate cleanly in a way one would 
want to live with for the long run.

So Guile's goals have shifted.  It was once (originally)
to be a tight, clean, fun Scheme environment with some
alternative syntaxes and specialized environments -- and
only later did it become the kind of heterogenous language
environment we see it moving towards today.

So, again:

> I hope support for other popular extension languages will be
> implemented eventually -- that was the original goal of Guile.

Nah.  The heck it was.  That's just not true.

Now, there is a separate question:  is that goal to support
other popular extension languages a better idea or a worse
idea (or not importantly different) than the original goal
of a nice, tight, fun Scheme-based system?

I have my opinion ("far worse!") but it's not my intent
to argue for that opinion here.  Just to set history straight.

That multi-lingual stuff was *not* the original goal of Guile.
On the contrary, the multi-lingual goals only came up at all
because of a perceived crisis sparked by Sun's announcement
that Tcl was to become the ubiquitous scripting language of
the Internet.

-t

Re: Guile in Emacs (was: integer overflow)

[Richard Stallman]

When I read about Sun's plan to make TCL the universal scripting
language, I decided to oppose that.  The plan I chose was to implement
Scheme and support other scripting languages by translation into Scheme.
Guile is the program I chose to adopt to do this with.  That plan
is what I am talking about.

Whatever history that code had before its adoption for this plan
is not what I am talking about.

Re: Guile in Emacs (was: integer overflow)

[Thomas Lord]

On Mon, 2010-04-12 at 08:30 -0400, Richard Stallman wrote:
> When I read about Sun's plan to make TCL the universal scripting
> language, I decided to oppose that.  The plan I chose was to implement
> Scheme and support other scripting languages by translation into Scheme.
> Guile is the program I chose to adopt to do this with.  That plan
> is what I am talking about.
> 
> Whatever history that code had before its adoption for this plan
> is not what I am talking about.

Sure.  In one sense it was just your use of the word "original"
as in "original goal" that I was objecting too.  

Yet, there is another aspect of this which I think is
relevant to "Guile in Emacs" and to your notion of 
supporting other scripting languages -- otherwise I 
wouldn't harp on it:

In those early days of Guile, after your decision, those
of us closer to the project discussed at considerable
length how exactly to support Tcl, Emacs lisp, and 
other languages.  Not just how to be able to run programs
in those languages but how to integrate them into a 
cohesive environment.

In each and every case we discovered devils in the details
and realized "Well, we can't."   We could make a TCL-like
language that could run many simple TCL programs but that
would not be upward compatible - and have that TCL-like
language nicely integrated with the larger environment.
We could make an Emacs Lisp style of environment that 
could run some Emacs Lisp code directly but that would
not be upwards compatible - and have that alternative Emacs
Lisp nicely integrated with the larger environment.  
But we absolutely could not, for fundamental reasons,
directly support Tcl and Emacs Lisp with fidelity and
wind up with a sane programming environment.

We realized that pretty quickly and tried (but failed) to 
convey to you this notion that we could not promise to
seamlessly integrate those other languages - but that we
could offer a reasonable compromise.  It was always 
an oversimplifying exaggeration to say that Guile would
support all of those other languages in any strong sense
of the word "support".  We could offer alternative 
syntaxes.  We could offer environments with simplified
evaluation models and more flexible types.  We could
give people the *feel* of Tcl or Python or Emacs Lisp
but there was no point in trying to faithfully reimplement
those languages in detail.   We failed miserably at 
communicating that distinction, apparently.

There was some momentary political convenience, back
then, around the burning question of which scripting
language would "win" and take over the world.  Would
Tcl become the ubiquitous scripting language?  Python?
It was an easy story to tell that Guile somehow transcended
the question for it could be both Tcl *and* Python (*and* 
Scheme) depending solely on user preference.   But it was
clear at the technical level that really Guile could only be
Scheme, pure and simple, although perhaps offering a
Tcl-*like* environment and a Python-*like* environment.
We - meaning you, me, and several others - were sloppy
back then about making that distinction clear.

If anything remains of the shared *technical* vision
of a complete GNU system that is lisp-centric with 
many extensible, self-documenting programs -- and if 
sentiment remains that Scheme is a fine choice for 
extension language -- then I mainly hope that people
pursuing that vision today won't go down the same rat-hole
that caught us up back then.  "I, alone, survive to tell
the tale...".

If you want a half-decent Scheme as your core 
extension language then *make that work first* and don't
worry so much about compatibility with legacy code
in those other languages.   There are no good answers
about how to cleanly integrate Emacs Lisp and other
languages with Scheme at that level.  People have 
thought about for, what, something over 15 years now
and the ones thinking about it today are getting 
stuck going over the very same questions people got
stuck on 15 years ago.

Meanwhile, with all the (often interesting and skilled - 
admirable) work that has gone down that rat-hole in 
those years, a thoroughly Scheme-based but not upwards
compatible Emacs could have been casually produced by,
say, 10 or 12 years ago.   Coulda' Shoulda' Woulda' but
Didn't, as the saying goes.

I just hope that the next 10 years of GNU generally
and Emacs specifically isn't going to be tied down
to the historic baggage of the "Tcl Wars" and the 
misunderstandings it provoked.

Someone -- that is to say "someone" -- should just 
rip Emacs Lisp out of the C part of GNU Emacs, bind 
the best parts of that stuff to Guile, and start *there*.
It's not a small job but it's also not a "more than 
a decade" job.  It could have been started much more than
a decade ago.  It'd be, in my view, a giant leap back
towards the vision of a GNU system that was shared 
among several key GNU hackers back in the early 1990s
and earlier.  And it'd be, in my view, technically sane
in comparison to some more popular alternatives like 
trying to support Emacs Lisp in detail.

I'm done.  I've said my piece.

-t

Re: Guile in Emacs

[Bruce Stephens]

Thomas Lord <lord@emf.net> writes:

[...]

> Someone -- that is to say "someone" -- should just rip Emacs Lisp out
> of the C part of GNU Emacs, bind the best parts of that stuff to
> Guile, and start *there*.  It's not a small job but it's also not a
> "more than a decade" job.  It could have been started much more than a
> decade ago.  It'd be, in my view, a giant leap back towards the vision
> of a GNU system that was shared among several key GNU hackers back in
> the early 1990s and earlier.

I wonder.  Looking at the list of Guile applications doesn't give me the
impression of a guile-using GNU ecosystem just waiting for Emacs to
join.

Wouldn't it be about as good (and probably less work), to give up on the
guile idea and evolve emacs lisp (with Miles's lexical-bind changes, and
(one hopes) multithreading, and maybe other things)?

[...]

Re: Guile in Emacs

[Thomas Lord]

We risk drifting into the topic of "general strategies
for GNU" and away from "what could / should(?) Emacs
do" so let's not make a long discussion of it but:

On Tue, 2010-04-13 at 13:44 +0100, Bruce Stephens wrote:

> I wonder.  Looking at the list of Guile applications doesn't give me the
> impression of a guile-using GNU ecosystem just waiting for Emacs to
> join.

Sure.  The optimist's version of that observation
is that, hey, it's a blank slate: pure opportunity,
no legacy "baggage".  The sky's the limit.


> Wouldn't it be about as good (and probably less work), to give up on the
> guile idea and evolve emacs lisp (with Miles's lexical-bind changes, and
> (one hopes) multithreading, and maybe other things)?

I dunno.  Maybe.  I'd guess that, no, that's not a 
good strategy.   Four reasons come quickly to mind:

1) There are many more materials out there that teach
Scheme than teach a novel spin-off of Emacs Lisp.

2) There are many and will continue to be many
implementations of Scheme.  Some of these implementations
are the product of pretty intense R&D.   No, they aren't
all 100% compatible with one another - far from it - but
it is easier to start with any one of those and make it
more like the others than to, say, build better and alternative
implementations of Emacs Lisp.

3) Scheme has features that we know make it a pretty
good "multi-paradigm" language.   Features like Scheme's
macro systems, proper safe-for-space tail recursion,
etc. are pretty good to have.   It would be easier to just
use those existing features than to try to retrofit them
into Emacs Lisp.

4) Using Scheme, you can can leverage third party 
Scheme code.  There isn't a huge amount worth using
but there is a decent amount.  There are the SRFIs.
There is SCSH (the Scheme shell).

For a long-term investment, I'd put my money on Scheme
rather than an ad hoc offspring of Emacs Lisp.

Re: Guile in Emacs

[Stefan Monnier]

> I dunno.  Maybe.  I'd guess that, no, that's not a 
> good strategy.   Four reasons come quickly to mind:

I agree with your conclusion but for a very different reason.
In my opinion the actual language used is not very important because
most of the code that will be used with Emacs will be written
specifically for Emacs.
The availability of alternative implementations is also of no use
because changing the underlying implementation is the part that's
difficult (at least with Emacs's current structure).

What matters is that we reuse some existing implementation and benefit
from all the work done on it, so we don't have to spend time working on
the Elisp byte-compiler.

I like the idea of Guile not because it's using a "standard preexisting
language with libraries and experienced coders", but because it'll give
us a bunch of hackers working on efficient implementation,
multithreading, ...


        Stefan


PS: The same holds for the redisplay engine; I really hope/wish we will
be able to switch to some other project's redisplay engine at some point.

Re: Guile in Emacs

[Thomas Lord]

On Tue, 2010-04-13 at 12:07 -0400, Stefan Monnier wrote:

> I agree with your conclusion but for a very different reason.
> In my opinion the actual language used is not very important because
> most of the code that will be used with Emacs will be written
> specifically for Emacs.

Well, I'm not so sure that that's entirely true
and I'm also not so sure that that's the only 
consideration.

What you say is less than obviously true in this
way, at least:  Consider how modern browsers are
often written.  On the one hand there is some lower-level
library for rendering HTML and doing various HTTP magic.
On the other hand, there are higher levels that make
the complete application.  That lower level part gets re-used
in other applications.   A text editor isn't obviously
much (or perhaps at all) a different case.  It might
be desirable to start thinking of Emacs in two layers: an
embeddable library and, sure, a default application as its
basic front-end.   Emacs lisp library code would then 
be "emacs specific" in some sense (relative to the 
library) but not really application specific.

Beyond that:  While it might be the case that "most"
Emacs lisp library code is Emacs-specific in some sense,
some important code need not be.  One example is SCSH
(the Scheme "shell" for managing sub-processes).  
A possible future example is Unicode-oriented string
libraries.  "Most" of the code might be Emacs-specific 
but some important parts of the code need not be.

And, anyway, it's not the only consideration.  If 
Scheme were used by many GNU programs as an extension
language, then inter-process communication by writing
and reading Scheme s-exps would be more common.  Two
standard Scheme environments can exchange s-exps with
relatively little problem.   But toss in a very different
lisp - like Emacs lisp - and you start to have problems.
E.g., what does "nil" read as?  What's the syntax for
strings?  Are symbols case sensitive?




> The availability of alternative implementations is also of no use
> because changing the underlying implementation is the part that's
> difficult (at least with Emacs's current structure).

Sure.  But if we're thinking of a long term strategy
rather than quick hacks, we should also keep in mind
that much of what is today in C in Emacs could well be
usefully replaced by Scheme code.

That's part of why I suggest (you know, given the infinite
amount of spare time we all have) the radical option of 
stripping Emacs lisp out and re-binding to Scheme:  because
that exercise would lead to some careful re-evaluation of
what needs to be in the C part and how best to make a 
clean interface to it.




> What matters is that we reuse some existing implementation and benefit
> from all the work done on it, so we don't have to spend time working on
> the Elisp byte-compiler.

I fully agree that that matters.


> I like the idea of Guile not because it's using a "standard preexisting
> language with libraries and experienced coders", but because it'll give
> us a bunch of hackers working on efficient implementation,
> multithreading, ...


Yup.  A big "gotcha" trap to watch out for on that path
is Guile's C API.   The less intertwingled Guile's (or any
other implementations) assumptions about the C client can
be, the better and more flexible the result.   I.e., it would
be pretty much the same mistake as being too Emacs lisp centric
to be too Guile centric.   It would ideally be plausible to
swap out Guile for some other implementation, even if the 
initial investment and commitment is to use Guile.

-t



> 
>         Stefan
> 
> 
> PS: The same holds for the redisplay engine; I really hope/wish we will
> be able to switch to some other project's redisplay engine at some point.
> 
> 

Re: Guile in Emacs

[Christian Lynbech]

>>>>> "Thomas" == Thomas Lord <lord@emf.net> writes:

>> Wouldn't it be about as good (and probably less work), to give up on the
>> guile idea and evolve emacs lisp (with Miles's lexical-bind changes, and
>> (one hopes) multithreading, and maybe other things)?

Thomas> I dunno.  Maybe.  I'd guess that, no, that's not a 
Thomas> good strategy.   Four reasons come quickly to mind: ...

Wouldn't a reasonable alternative to using a scheme implementation be to
use a Common Lisp ditto? And wouldn't that be a much closer fit
semantically to the current Emacs Lisp dialect?

I kind of hear you suggesting ditching all of the existing Emacs Lisp in
favour of starting over from scratch with scheme. 

While it will be easy to list examples of existing libraries few will
miss, emacs killer features such as gnus or org-mode still represent
significant investments that are not easily reproduced from scratch.


------------------------+-----------------------------------------------------
Christian Lynbech       | christian #\@ defun #\. dk
------------------------+-----------------------------------------------------
Hit the philistines three times over the head with the Elisp reference manual.
                                        - petonic@hal.com (Michael A. Petonic)

Re: Guile in Emacs

[Bruce Stephens]

Stefan Monnier <monnier@IRO.UMontreal.CA> writes:

[...]

> What matters is that we reuse some existing implementation and benefit
> from all the work done on it, so we don't have to spend time working on
> the Elisp byte-compiler.
>
> I like the idea of Guile not because it's using a "standard preexisting
> language with libraries and experienced coders", but because it'll give
> us a bunch of hackers working on efficient implementation,
> multithreading, ...

That seems like a stronger argument to me.  Back in 1995 (or
thereabouts) using scheme for better interoperability with other GNU
applications seemed a plausible argument, but I don't think that's how
things have worked out.  I doubt that replacing emacs lisp with scheme
would have much impact for interoperability, though removing one
extension language won't hurt.

[...]

Re: Guile in Emacs

[joakim]

Stefan Monnier <monnier@IRO.UMontreal.CA> writes:

>> I dunno.  Maybe.  I'd guess that, no, that's not a 
>> good strategy.   Four reasons come quickly to mind:
>
> I agree with your conclusion but for a very different reason.
> In my opinion the actual language used is not very important because
> most of the code that will be used with Emacs will be written
> specifically for Emacs.
> The availability of alternative implementations is also of no use
> because changing the underlying implementation is the part that's
> difficult (at least with Emacs's current structure).
>
> What matters is that we reuse some existing implementation and benefit
> from all the work done on it, so we don't have to spend time working on
> the Elisp byte-compiler.
>
> I like the idea of Guile not because it's using a "standard preexisting
> language with libraries and experienced coders", but because it'll give
> us a bunch of hackers working on efficient implementation,
> multithreading, ...
>
>
>         Stefan
>
>
> PS: The same holds for the redisplay engine; I really hope/wish we will
> be able to switch to some other project's redisplay engine at some point.
>

IMHO it would be interesting to have Emacs as a set of components which
could be plugged together, which also happened to include an elisp
interpreter. This could work sort of like JNI in Java, or some other
component model(Like one of the lightweight corba implementations)

A start would be a plug-in model that allowed C plugins to be jacked
into Emacs. That could allow other runtimes, like Guile, to run in Emacs
and affect the Emacs internal state.

-- 
Joakim Verona

Re: Guile in Emacs

[Christian Lynbech]

>>>>> "Thomas" == Thomas Lord <lord@emf.net> writes:

>> Wouldn't it be about as good (and probably less work), to give up on the
>> guile idea and evolve emacs lisp (with Miles's lexical-bind changes, and
>> (one hopes) multithreading, and maybe other things)?

Thomas> I dunno.  Maybe.  I'd guess that, no, that's not a 
Thomas> good strategy.   Four reasons come quickly to mind: ...

Wouldn't a reasonable alternative to using a scheme implementation be to
use a Common Lisp ditto? And wouldn't that be a much closer fit
semantically to the current Emacs Lisp dialect?

I kind of hear you suggesting ditching all of the existing Emacs Lisp in
favour of starting over from scratch with scheme. 

While it will be easy to list examples of existing libraries few will
miss, emacs killer features such as gnus or org-mode still represent
significant investments that are not easily reproduced from scratch.


------------------------+-----------------------------------------------------
Christian Lynbech       | christian #\@ defun #\. dk
------------------------+-----------------------------------------------------
Hit the philistines three times over the head with the Elisp reference manual.
                                        - petonic@hal.com (Michael A. Petonic)

Re: Guile in Emacs

[Thomas Lord]

On Tue, 2010-04-13 at 21:13 +0200, Christian Lynbech wrote:
> >>>>> "Thomas" == Thomas Lord <lord@emf.net> writes:
> 
> >> Wouldn't it be about as good (and probably less work), to give up on the
> >> guile idea and evolve emacs lisp (with Miles's lexical-bind changes, and
> >> (one hopes) multithreading, and maybe other things)?
> 
> Thomas> I dunno.  Maybe.  I'd guess that, no, that's not a 
> Thomas> good strategy.   Four reasons come quickly to mind: ...
> 
> Wouldn't a reasonable alternative to using a scheme implementation be to
> use a Common Lisp ditto?

I would have a hard time arguing conclusively that that
would be unreasonable.

Scheme is tidier and more compact and yields more
interesting subsets.  I think that Scheme currently
enjoys way more R&D than CL.  So I'd be on Scheme over
CL for the general GNU extension language (not just
for Emacs).   And I don't suppose that legacy Emacs
lisp code should in any way be allowed to hold back or
mess up the design of a GNU extension language.

But there are are good arguments both ways and
I think you are right that a CL "ditto" might be
a good alternative.


>  And wouldn't that be a much closer fit
> semantically to the current Emacs Lisp dialect?

Yes, I think it would.   I'm free enough in my current
position in life that I can afford to be cavalier and
say "Eh, who cares about all that tonnage of extant
Emacs lisp code.  What's the Right Thing if we discount
that code?"   I can't sell you or anyone on that attitude
and it might just be flat out wrong.  It's just, at least,
a position worth considering in the "big picture".



> I kind of hear you suggesting ditching all of the existing Emacs Lisp in
> favour of starting over from scratch with scheme. 


Yeah, I am, but I don't see anyone who is an obvious 
candidate to take on such a huge job with no guaranteed
success at the end.   As I said (and the scare-quotes are
significant):  it's something that "someone" should do.



> While it will be easy to list examples of existing libraries few will
> miss, emacs killer features such as gnus or org-mode still represent
> significant investments that are not easily reproduced from scratch.

Sure.  But don't misunderestimate the rapidity with
which a programmer fluent in both Emacs lisp and Scheme
can sit down and port those programs from one language to
the other.   (The one that really scares *me* is calc.el!)

-t




> 
> 
> ------------------------+-----------------------------------------------------
> Christian Lynbech       | christian #\@ defun #\. dk
> ------------------------+-----------------------------------------------------
> Hit the philistines three times over the head with the Elisp reference manual.
>                                         - petonic@hal.com (Michael A. Petonic)

Re: Guile in Emacs

[Christian Lynbech]

>>>>> "Thomas" == Thomas Lord <lord@emf.net> writes:

>> While it will be easy to list examples of existing libraries few will
>> miss, emacs killer features such as gnus or org-mode still represent
>> significant investments that are not easily reproduced from scratch.

Thomas> Sure.  But don't misunderestimate the rapidity with
Thomas> which a programmer fluent in both Emacs lisp and Scheme
Thomas> can sit down and port those programs from one language to
Thomas> the other.

Possibly, but what scares me in that is the amount of dedication from
*both* library authors and users needed at the same point in time to make
such a switch happen. 

Users will not switch unless the packages they use are not ready in the
new form and authors will not switch until the user base is ready to
follow. It can very easily end up in a catch 22 situation. 

It is not that there haven't been several attempts already to implement
a new direction for emacs.

                             -- Chrisitian

Re: Guile in Emacs

[Richard Stallman]

I think it would be a tremendous waste of effort to rewrite Emacs
libraries in any language other than Emacs Lisp.  Thus, what interests
me is the idea of supporting other languages in addition to Emacs
Lisp.

Re: Guile in Emacs

[Richard Stallman]

Common Lisp is extremely complicated and ugly.  When I wrote GNU Emacs
I had just finished implementing Common Lisp, and I did not like it
much.  It would bloat Emacs terribly, and documenting it would be hard
too.

Scheme is elegant, and it is a better direction to move in.

Since we have our own Scheme implementation, we should use that one.
If it has a serious disadvantage, we should do something about that.
There are various things that might be right to do, but simply
disregarding it in the case of Emacs cannot be right.

Re: Guile in Emacs

[christian.lynbech]

>>>>> "RMS" == Richard Stallman <rms@gnu.org> writes:

RMS> Common Lisp is extremely complicated and ugly.  When I wrote GNU Emacs
RMS> I had just finished implementing Common Lisp, and I did not like it
RMS> much.  It would bloat Emacs terribly, and documenting it would be hard
RMS> too.

Fair enough, I happen to disagree but I respect that you have real-life
experiennce in the matter.

I do not understand the point about documentation though. Why would it
be harder to document a common lisp based emacs than a scheme based
emacs? I do not expect that we would like to make a brand new common
lisp from scratch but rather to start out from an exisiting
implementation, just as we would with scheme.

RMS> Scheme is elegant, and it is a better direction to move in.

I will challenge the claim that emacs-on-scheme would be any less
bloated than an emacs-on-cl. Sure, core scheme is smaller than core
common lisp, but once you have finished adding all of the stuff you need
to get a working application, I believe you will have reached pretty
much the same level of bloat.

RMS> Since we have our own Scheme implementation, we should use that one.
RMS> If it has a serious disadvantage, we should do something about that.
RMS> There are various things that might be right to do, but simply
RMS> disregarding it in the case of Emacs cannot be right.

I am personally mostly worried about Thomas' points about getting scheme
and emacs lisp to coexist. I just cannot see any evolution of emacs fly
in the real world if it involves a clean cut away from the existing base
of emacs lisp libraries. How we would ever get all developers and all
users to back up such a move is beyond me ("all" used here in the sense
of "enough to form a critical mass").

(Incidently, there is still something called GNU Common Lisp, even if
not updated since 2005)


------------------------+-----------------------------------------------------
Christian Lynbech       | christian #\@ defun #\. dk
------------------------+-----------------------------------------------------
Hit the philistines three times over the head with the Elisp reference manual.
                                        - petonic@hal.com (Michael A. Petonic)

Re: Guile in Emacs

[Thomas Lord]

I really didn't mean to provoke a 20 message 
thread about what was, to me, just an idle 
thought and an interesting idea I wanted to share.

So, below I'll reply to Christian and I'll also 
indirectly reply to some of the other messages - 
but unless there is someone or are some people 
who, right away, want to start working on a Scheme-based
Emacs - there is probably not a lot to discuss.

If there *is* a non-empty set of people who want 
to work on it, they should speak up and we can try
to organize such an effort - the first step of which
would be to convene elsewhere besides this mailing 
list.

Otherwise, the whole thing is just a (to me) pretty
thought that I wanted to share with others who might
like to put in the back of their minds and contemplate
it a bit, until perhaps someday when there is an
obvious course of action to take.

On Wed, 2010-04-14 at 08:45 +0200, christian.lynbech@tieto.com wrote:
> I am personally mostly worried about Thomas' points about 
> getting scheme and emacs lisp to coexist. I just cannot
> see any evolution of emacs fly in the real world if it 
> involves a clean cut away from the existing base
> of emacs lisp libraries. How we would ever get 
> all developers and all users to back up such a move
> is beyond me ("all" used here in the sense
> of "enough to form a critical mass").


That's a legitimate concern but there are viable
solutions to the problem:

a) I'm *not* suggesting the idea of abandoning work
on the main development line of GNU Emacs.  I would
expect that work to continue and to continue to use
Emacs lisp.   There are so many GNU Emacs users and 
the program generates such a large amount of good
will for GNU that it seems to me very desirable to 
not "fix" what isn't broken.

b) I don't imagine that a Scheme-centric Emacs would
find itself with especially huge numbers of users in
its first few years of life.   I would *guess* that what
would happen is that many people would try out the 
first release just to see what the fuss was about, that 
a much smaller number would stick with it, and that a 
slightly smaller number of those initial users would become
contributing developers.   My guess is based on the assumption
that the first release is any good.

If things went well in the first few years, then 
the Scheme-based Emacs might start becoming seriously
popular.

c) Someone - I think it was Tom Tromey - observed
that Emacs Lisp is hardly something that is holding
Emacs back (so why change at all?, he asks).

I pretty much agree with that, at least in this sense:
If the only serious difference between GNU Emacs and
a new Scheme-based GNU Emacs were a change in extension
language - then there would be little point in bothering.

To succeed, a Scheme-based Emacs project would have to
produce an Emacs with features that users really enjoy
but that would be hard if not impossible to do in 
GNU Emacs with Emacs lisp.  (In contrast, just replacing 
the Emacs lisp interpreter with a version of Guile that
faithfully  implements Emacs Lisp can succeed
without needing to produce radically new features.)

What kind of features might a Scheme-based Emacs offer
that would be that important?  Some speculation:

Because Emacs Lisp support would be an explicit
non-goal, everyone's favorite lisp package would have
to be re-written (though, yes, often cribbing off
of the Emacs Lisp version).   Thus there would be 
opportunity (and a forcing function) to reconsider
countless "little annoying details" of those packages
and produce new versions with many improved details.

Because a Scheme engine *should* wind up being 
significantly faster, and because such radical 
changes to the C code would be needed anyway, there
is an opportunity to write more of the editor in
the extension language, and less in C.   In and 
of itself that doesn't help users.  In practice it 
can help users by making more of Emacs much easier
to modify and improve.   For example (and, yes, this
is a bit hand-wavy): if more of the display code
were in Scheme, there is a good chance that there 
would be more and fancier features and "nice touches"
in the display.

I think that most people would agree that Scheme
is a significantly more powerful language than Emacs
lisp in the sense that it is easier to write more
sophisticated programs in Scheme (with its closures, 
its fancy macro systems, and so forth).   This, again,
does not directly benefit users.   In practice it could
benefit users because in the same number of lines of
code (Scheme vs. Emacs Lisp) an extension package could
often provide more ambitious, fancier features.  I've
a few times mentioned the example SCSH (the Scheme shell)
and it's a fine example to use here:  a SCSH is not 
especially large or complicated.   SCSH presents a 
subprocess management interface that is light years ahead
of Emacs' and that would be rather painful to write in 
Emacs Lisp.  The interface to sub-processes is again
of no direct importance to users but what would benefit 
users are creative new applications enabled by the new 
interface.   That is, Scheme's more powerful environment
leads to subsystems like SCSH which leads to extension package
developers using sub-processes more often and in fancier
ways which - one would hope - leads to fancier, user-facing
extension language packages.

You get the idea, I hope.

Scheme isn't some magic bullet that, by its mere presence,
would advance Emacs into a bright new future.  It is a 
nicer, more comfortable extension language, more standard
than Emacs lisp, and affording more efficient implementations
than Emacs lisp could ever hope to afford - all details that
don't help users directly.

The stubborn decision to start fresh and  cannibalize
GNU Emacs to make a Scheme-based Emacs *could* be, in addition
to possibly being fun, a catalyst.  By forcing the rewrite or 
significant modification of large subsystems, it would force
a careful design review and updating - without the pressure
of being upwards compatible with Emacs lisp.   In that 
adapt / rewrite process, there is opportunity not just to go
after one or two new "killer features" - but rather to go after
many, many small but noticeable improvements all across the 
board.  And if more of the editor is written in the extension
language and the extension language is a more powerful language
with a faster implementation - then it *should* (no guarantees)
work out that within a few years the Scheme based version 
will have user-facing features that make people say "Wow.  The
old Emacs couldn't do that!"

-t

Re: Guile in Emacs

[Bruce Stephens]

Thomas Lord <lord@emf.net> writes:

[...]

> If there *is* a non-empty set of people who want 
> to work on it, they should speak up and we can try
> to organize such an effort - the first step of which
> would be to convene elsewhere besides this mailing 
> list.

Judging by
<http://wingolog.org/archives/2010/04/02/recent-developments-in-guile>,
it is being worked on.  The claim is that there's an emacs lisp compiler
to guile that gives (or will give RSN) a faster implementation than that
in Emacs.

I presume the language will be equivalent to current emacs lisp (so that
wouldn't force the rewrite and perhaps spur the improvements you'd like
to see), on the other hand not having to rewrite the lisp libraries
would make it an easier sell.  Presumably it could easily support
lexical binding (and closures) and other features.

Nothing wrong (IMHO) in proceeding to improve emacs lisp by integrating
the lexical-binding branch, perhaps adding hygienic (or other) macros,
etc.

[...]

Re: Guile in Emacs

[Jose A. Ortega Ruiz]

Thomas Lord <lord@emf.net> writes:

> If there *is* a non-empty set of people who want 
> to work on it, they should speak up and we can try
> to organize such an effort - the first step of which
> would be to convene elsewhere besides this mailing 
> list.

i am interested.

cheers,
jao

Re: Guile in Emacs

[christian.lynbech]

>>>>> "Thomas" == Thomas Lord <lord@emf.net> writes:

Thomas> If the only serious difference between GNU Emacs and
Thomas> a new Scheme-based GNU Emacs were a change in extension
Thomas> language - then there would be little point in bothering.

Theses points have been made more or less in several places already but
just to reiterate, changing the extension language to a more widely used
language variant, even if everything else stays the same, would have
several benefits:

  - faster execution of also emacs lisp code
  - shared burden of maintenance for the language platform
  - ability to load up (external) libaries written in the new language
  - a platform to evolve also the emacs lisp parts into safer/better/faster
  - a platform to evolve C parts back into the extension language realm

What kind of future this would lead Emacs into is hard to say but it
would certainly facilitate many great things. I agree with you that, as
on example, being able to move functionality from back into scheme or
elisp or whatever will be a very exiticing thing.


------------------------+-----------------------------------------------------
Christian Lynbech       | christian #\@ defun #\. dk
------------------------+-----------------------------------------------------
Hit the philistines three times over the head with the Elisp reference manual.
                                        - petonic@hal.com (Michael A. Petonic)

Re: Guile in Emacs

[Richard Stallman]

    I do not understand the point about documentation though. Why would it
    be harder to document a common lisp based emacs than a scheme based
    emacs?

Common Lisp has sooo much to document.  The Emacs Lisp Manual would
get a lot bigger if it had to include the specs of so many functions.

RE: Guile in Emacs

[Drew Adams]

> Common Lisp has sooo much to document.  The Emacs Lisp Manual would
> get a lot bigger if it had to include the specs of so many functions.

The Elisp manual would not need to document things that are vanilla Common Lisp
(i.e. respect the CL standard). It could simply cross-reference the relevant CL
doc. The Elisp manual would need to document only extensions or departures from
the standard. It does not need to reproduce all of the existing CL
documentation.

Re: Guile in Emacs

[David Kastrup]

"Drew Adams" <drew.adams@oracle.com> writes:

>> Common Lisp has sooo much to document.  The Emacs Lisp Manual would
>> get a lot bigger if it had to include the specs of so many functions.
>
> The Elisp manual would not need to document things that are vanilla
> Common Lisp (i.e. respect the CL standard). It could simply
> cross-reference the relevant CL doc. The Elisp manual would need to
> document only extensions or departures from the standard. It does not
> need to reproduce all of the existing CL documentation.

The manuals of an editor with a user-accessible extension language (with
Emacs, this is the case for Elisp but not for C) need to document this
extension language.

Otherwise, only programmers can be expected to be able to use it.  One
reason for that is that non-Emacs specific Lisp manuals will not focus
about how to get things done with Emacs.  Applying a manual utterly
without editing focus to editing tasks is quite a large intellectual
feat.

-- 
David Kastrup

RE: Guile in Emacs

[Drew Adams]

> >> Common Lisp has sooo much to document.  The Emacs Lisp Manual would
> >> get a lot bigger if it had to include the specs of so many 
> >> functions.
> >
> > The Elisp manual would not need to document things that are vanilla
> > Common Lisp (i.e. respect the CL standard). It could simply
> > cross-reference the relevant CL doc. The Elisp manual would need to
> > document only extensions or departures from the standard. 
> > It does not need to reproduce all of the existing CL documentation.
> 
> The manuals of an editor with a user-accessible extension 
> language (with Emacs, this is the case for Elisp but not
> for C) need to document this extension language.

Emacs needs to document its extension language, yes of course. That means that
it needs to enable users to obtain complete documentation about the language.
That does not mean that the Emacs manuals (e.g. Elisp manual) themselves need to
specify everything about CL functions Emacs might support. It simply means that
Emacs needs to provide users access to that information.

A cross reference can be focused directly to an appropriate section of existing
Common-Lisp documentation. We need not just dump a user into the TOC of a large
manual.

It is a choice whether we want clicking a cross-reference in an Emacs manual to
take you (a) to a section of the same manual or another manual available locally
or (b) to a section of a manual that might be on the Web. A cross reference
might well be to a non-local or a non-GNU manual or specification. What is
important is that we give users access to the specific info they need.

> Otherwise, only programmers can be expected to be able to use it.  One
> reason for that is that non-Emacs specific Lisp manuals will not focus
> about how to get things done with Emacs.  Applying a manual utterly
> without editing focus to editing tasks is quite a large intellectual
> feat.

The Emacs manual(s) will of course focus on how to get things done with Emacs.
If there is something Emacs-specific to be said about a function such as `some',
then yes, the Emacs manuals are an appropriate place to say that. That does not
imply that everything possible to be said about every Common-Lisp construct
needs to be covered in the Emacs manual. That would be an absurd approach.

Re: Guile in Emacs

[Jeff Clough]

"Drew Adams" <drew.adams@oracle.com> writes:

> It is a choice whether we want clicking a cross-reference in an Emacs manual to
> take you (a) to a section of the same manual or another manual available locally
> or (b) to a section of a manual that might be on the Web. A cross reference
> might well be to a non-local or a non-GNU manual or specification. What is
> important is that we give users access to the specific info they need.

It is also important for that information to be available in one place.
When the information you need is split between two (or more) manuals, a
wiki, a HOW-TO and Wonder Tommy's blog, it sucks, no matter how well
cross-referenced or searchable it is.  There's a better way and Emacs
has it (for the most part).

I've never subscribed to the theory that just because a feature is
described on some web page out there, and not in the official manual,
that it's "documented".  Why people want to see this as the success of
the internet rather than as a failure of the documentation is beyond me.

Keep the information itself in one place that I can put on a tablet as a
PDF, or print out and take with me on an overnight.  Not everyone reads
their docs from within Emacs all of the time.

For what it's worth.

Jeff

Re: Guile in Emacs

[christian.lynbech]

>>>>> "Jeff" == Jeff Clough <jeff@chaosphere.com> writes:

Jeff> It is also important for that information to be available in one place.

I agree that Emacs should come with all relevant documentation bundled
up as part of the package, but I can not see that this is the same as
everything necessarily has to sit inside (the moral equivalent of) one
single gigantic texinfo file.

There will need to be easy acces to the reference just as we get today
easily can get from the documentation of a function to its source
definition and there should probably be some top-level entry points with
proper reading guidance just as the editing library manual should of
course link to the language manual where relevant.


------------------------+-----------------------------------------------------
Christian Lynbech       | christian #\@ defun #\. dk
------------------------+-----------------------------------------------------
Hit the philistines three times over the head with the Elisp reference manual.
                                        - petonic@hal.com (Michael A. Petonic)

Re: Guile in Emacs

[Jeff Clough]

<christian.lynbech@tieto.com> writes:

> Jeff> It is also important for that information to be available in one place.
>
> I agree that Emacs should come with all relevant documentation bundled
> up as part of the package, but I can not see that this is the same as
> everything necessarily has to sit inside (the moral equivalent of) one
> single gigantic texinfo file.

Neither do I, but what we have today works.  That's what I'm for.  Don't
fix what isn't broken.  Or, more appropriately, don't fix what Emacs
seems to be the only application to do *right*.

For Emacs itself, there are exactly four places, organized by context,
someone has to look to find the information they need.

1.  The Emacs Manual - How to use Emacs as an editor.  The only Lisp
found here is for basic, simple configuration.  This is for people that
don't want to learn Lisp, they just want to use Emacs as an editor, run
macros, dired, etc.

2.  The Emacs Lisp Intro - Teaches the basics of Emacs Lisp.  How to use
defadvice, creating a simple major mode, etc.  This is for people just
getting started, or who want to know only how to write a simple function
to accomplish some task this afternoon.

3.  The Emacs Lisp Reference - Describes every function, macro and
language construct with a few explainations.  There may be more
information to be had about a particular thing, but it at least lists
all things with *some* amount of description.  This is for people
seriously interested in the language who want to browse for
ideas/knowledge, read something away from their computer or copy a quick
"toy" example (if available) and make it work for them right now.

4.  The online documentation/what you get from describe-function and
it's kin - The definitive reference to every symbol you can throw at
Emacs.  Assumed to be up-to-date and accurate.  This is for people
knee-deep in code right now that can't remember what the third argument
of the function at point needs to be, or can't remember if what they
want is insert-file-contents or insert-file-contents-literally.

All four of these sources have their proper use, and none of them could
be used to duplicate exactly the purpose of any other.  And it covers
all of the contexts you'd want documentation (IMHO) well, in a minimum
number of places.

> There will need to be easy acces to the reference just as we get today
> easily can get from the documentation of a function to its source
> definition and there should probably be some top-level entry points with
> proper reading guidance just as the editing library manual should of
> course link to the language manual where relevant.

I think having cross references between the four sources described above
is a great thing.  There should be more of that today.

Jeff

Re: Guile in Emacs

[Lennart Borgman]

On Thu, Apr 15, 2010 at 3:54 PM, Jeff Clough <jeff@chaosphere.com> wrote:
>
> 4.  The online documentation/what you get from describe-function and
> it's kin - The definitive reference to every symbol you can throw at
> Emacs.  Assumed to be up-to-date and accurate.  This is for people
> knee-deep in code right now that can't remember what the third argument
> of the function at point needs to be, or can't remember if what they
> want is insert-file-contents or insert-file-contents-literally.


Maybe it should be mentioned here that the online documentation can't
handle defstruct etc in the cl packages. To me this means that the
package is not up to Emacs normal standards.

RE: Guile in Emacs

[Drew Adams]

> I think having cross references between the four sources 
> described above is a great thing.  There should be more
> of that today.

According to *your* argument: For any topic, you should be able to find an
explanation of *everything* about that topic (including all possible relations
and ramifications) in *one* place.

According to your argument, cross references are anathema - not even a necessary
evil, but something that can and should be dispensed with.

Do you get it now? Your argument is in fact an argument against *any* doc
organization. It is an argument against cross references and the need for or the
utility of cross references.

Re: Guile in Emacs

[Jeff Clough]

"Drew Adams" <drew.adams@oracle.com> writes:

>> I think having cross references between the four sources 
>> described above is a great thing.  There should be more
>> of that today.
>
> According to *your* argument: For any topic, you should be able to find an
> explanation of *everything* about that topic (including all possible relations
> and ramifications) in *one* place.
>
> According to your argument, cross references are anathema - not even a necessary
> evil, but something that can and should be dispensed with.

Did you read the rest of that referenced post?  I clarified that what I
want from a future Emacs, in terms of documentation, is what is provided
by today's Emacs.

> Do you get it now? Your argument is in fact an argument against *any* doc
> organization. It is an argument against cross references and the need for or the
> utility of cross references.

*I* got what I wrote, but I'm pretty sure that *you* didn't.

It is perfectly fine for an "Intro Manual" included with Emacs to link
to a "Reference Manual" included with Emacs.  It is not okay for
critical documentation to be hosted anywhere but a user's hard drive,
and the number of files this documentation is contained in should be
kept to a minimum.

I am not asking for a monolithic document that would be the end all, be
all Emacs Bible.  I am stating that the four manuals +
describe-function, et al, are a great arrangment and the example I would
point to when someone asked me for an example of good documentation.

If a change was made to this system, I have a hard time believing it
would be anything other than a mistake, if only because I cannot imagine
any change improving it and if no improvement is made, effort is wasted.

Jeff

Re: Guile in Emacs

[christian.lynbech]

>>>>> "Jeff" == Jeff Clough <jeff@chaosphere.com> writes:

Jeff> I am stating that the four manuals + describe-function, et al, are
Jeff> a great arrangment

I fail to see why 5 manuals are significantly worse than 4 manuals as
long as there is a clear separation between them.



------------------------+-----------------------------------------------------
Christian Lynbech       | christian #\@ defun #\. dk
------------------------+-----------------------------------------------------
Hit the philistines three times over the head with the Elisp reference manual.
                                        - petonic@hal.com (Michael A. Petonic)

Re: Guile in Emacs

[Jeff Clough]

<christian.lynbech@tieto.com> writes:

>>>>>> "Jeff" == Jeff Clough <jeff@chaosphere.com> writes:
>
> Jeff> I am stating that the four manuals + describe-function, et al, are
> Jeff> a great arrangment
>
> I fail to see why 5 manuals are significantly worse than 4 manuals as
> long as there is a clear separation between them.

At this point I'm just going to assume that I'm either being
unreasonable and not knowing it or that I'm not able to properly explain
my reasoning concerning this.

I believe that Emacs works perfectly with the four manuals it has
today.  I fail to see how a fifth manual would improve it.  That a fifth
manual would not make things "significantly worse" isn't a glowing
endorsement.

Regardless, I think I've explained myself here as best as I am able to.
When/if the change is made, I'm sure I'll be able to see for myself how
much of an effect it has and I may even be pleasantly surprised.

Jeff

RE: Guile in Emacs

[Drew Adams]

> It is not okay for critical documentation to be hosted
> anywhere but a user's hard drive,

1. Not every bit of description of every single Common-Lisp construct is
"critical" documentation. Far from it.

You've added "critical" now, but your previous argument was against all, not
just "critical", non-local reference.


2. Depending on what is meant by such a watered-down meaning of "critical", it
would be absurd in 2010 and beyond to limit, by design, the doc set to what is
or can be hosted on a user's hard drive.

Which user? What size hard drive? Will you tailor the available doc set by
design to Emacs running on a 1 gig flash drive or other lowest common
denominator?

Nowadays, the number of people who actually download PDF docs for a large
software product, in order to have them locally, is miniscule, compared to the
number who consult the same doc set on demand, on line (Web) as HTML.

It is good for users to be _able_ to have a certain number of core docs locally.
And they should be able to control just how much they want to keep locally and
how much they are willing to access remotely.

But that doesn't mean that the entire doc set for Emacs (or anything else) needs
to be or should be kept to a minimal size that every user could store on a
typical hard disk.


Yes, there are good reasons to avoid documentation bloat - always. Just as there
are good reasons to avoid software bloat (a more important argument concerning
CL, perhaps). The question in both cases is whether users _need_ all that doc or
all that software. _If_ we decide the answer is yes, then, well, it's needed,
regardless of how big typical hard disks might be this year.

A hard-and-simple hard-disk criterion such as you express is not appropriate.
Doc that is not needed by users is wasted and should be pruned. But if some
particular doc is needed to explain something, then it's needed, period.

Re: Guile in Emacs

[David Kastrup]

"Drew Adams" <drew.adams@oracle.com> writes:

>> It is not okay for critical documentation to be hosted
>> anywhere but a user's hard drive,
>
> 1. Not every bit of description of every single Common-Lisp construct is
> "critical" documentation. Far from it.

It is critical if it is being used in a package you are trying to
understand.

-- 
David Kastrup

Re: Guile in Emacs

[Jeff Clough]

"Drew Adams" <drew.adams@oracle.com> writes:

>> It is not okay for critical documentation to be hosted
>> anywhere but a user's hard drive,
> 
> You've added "critical" now, but your previous argument was against all, not
> just "critical", non-local reference.

Sigh.  I'm not adding requirements, *you* are being pedantic.

> Which user? What size hard drive? Will you tailor the available doc set by
> design to Emacs running on a 1 gig flash drive or other lowest common
> denominator?

I'll just point at what Emacs is doing today and say "Pretty much that".

> Nowadays, the number of people who actually download PDF docs for a large
> software product, in order to have them locally, is miniscule, compared to the
> number who consult the same doc set on demand, on line (Web) as HTML.

And a growing number of people are using netbooks with dodgy wi-fi
connections.

There is a general philosophical debate showing up in various places as
to whether it is okay to assume the presence of the network when the
user is not doing non network-specific things.  Apparently you feel the
answer is yes.

> But that doesn't mean that the entire doc set for Emacs (or anything else) needs
> to be or should be kept to a minimal size that every user could store on a
> typical hard disk.

My info directory is 14 megabytes.  Even if the new version required
this to increase by a full order of magnitude in order to be "complete",
I fail to see how this is in any way a limitation.

> A hard-and-simple hard-disk criterion such as you express is not appropriate.
> Doc that is not needed by users is wasted and should be pruned. But if some
> particular doc is needed to explain something, then it's needed, period.

Again, I'm going to point to what Emacs has today and say "See, that?
We should do that."

You disagree, fair enough.

Jeff

Re: Guile in Emacs

[christian.lynbech]

>>>>> "Jeff" == Jeff Clough <jeff@chaosphere.com> writes:

Jeff> Neither do I, but what we have today works.  That's what I'm for.  Don't
Jeff> fix what isn't broken.  Or, more appropriately, don't fix what Emacs
Jeff> seems to be the only application to do *right*.

Well, something needs to be done if emacs changes extension language.

Either we accept keeping the language documentation in a separate
document such we can use what the language platform already provides
*or* we duplicate the work done by the language platform in order to
integrate the language documentation into the emacs specific reference
manual.

The latter option both incurs an initial significant cost plus a
continued burden of maintenance to keep the emacs manual in synch with
the language platform.

It does not strike me as a good way of spending volunteer resources.


------------------------+-----------------------------------------------------
Christian Lynbech       | christian #\@ defun #\. dk
------------------------+-----------------------------------------------------
Hit the philistines three times over the head with the Elisp reference manual.
                                        - petonic@hal.com (Michael A. Petonic)

Re: Guile in Emacs

[Richard Stallman]

    Well, something needs to be done if emacs changes extension language.

There is no plan to change the extension language of Emacs,
only to support others in addition.

Re: Guile in Emacs

[christian.lynbech]

>>>>> "Richard" == Richard Stallman <rms@gnu.org> writes:

>     Well, something needs to be done if emacs changes extension language.

Richard> There is no plan to change the extension language of Emacs,
Richard> only to support others in addition.

Ok, I see.

If emacs becomes multi-lingual (or perhaps even if it just changes the
base implementation), I still think it would make sense to split out the
language specific documentation from the editing library reference
manual to increase the maintainability of both.

This is of course not a terribly important thing to discuss at the
moment.


------------------------+-----------------------------------------------------
Christian Lynbech       | christian #\@ defun #\. dk
------------------------+-----------------------------------------------------
Hit the philistines three times over the head with the Elisp reference manual.
                                        - petonic@hal.com (Michael A. Petonic)

Emacs Info manuals (was: Guile in Emacs)

[Juri Linkov]

> 1.  The Emacs Manual
[...]
> 2.  The Emacs Lisp Intro
[...]
> 3.  The Emacs Lisp Reference
[...]
> 4.  The online documentation/what you get from describe-function and
> it's kin - The definitive reference to every symbol you can throw at
> Emacs.  Assumed to be up-to-date and accurate.  This is for people
> knee-deep in code right now that can't remember what the third argument
> of the function at point needs to be, or can't remember if what they
> want is insert-file-contents or insert-file-contents-literally.

I'd like to also clarify that the purpose of the fourth is not only to
provide actual information about available functions and their arguments
but also to add dynamical information like whether a function is advised,
what keybindings it has, etc.  IOW, this information can't be in
a static Info file.  So unlike 3 static Info manuals, the online help
manual should be dynamically created at run time.  This is possible to do
using virtual Info manuals.

> I think having cross references between the four sources described above
> is a great thing.  There should be more of that today.

Yes, this would be a great thing, but unfortunately currently
cross references between the online documentation and Info manuals
are largely dysfunctional.  However, with the online documentation
in a virtual Info manual, cross references between these four sources
will use the existing Info cross references, thus providing more
consistent UI.

-- 
Juri Linkov
http://www.jurta.org/emacs/

RE: Guile in Emacs

[Drew Adams]

> > It is a choice whether we want clicking a cross-reference 
> > in an Emacs manual to take you (a) to a section of the
> > same manual or another manual available locally
> > or (b) to a section of a manual that might be on the Web. A 
> > cross reference might well be to a non-local or a non-GNU
> > manual or specification. What is important is that we give
> > users access to the specific info they need.
> 
> It is also important for that information to be available in 
> one place. When the information you need is split between two
> (or more) manuals, a wiki, a HOW-TO and Wonder Tommy's blog,
> it sucks, no matter how well cross-referenced or searchable it is.
> There's a better way and Emacs has it (for the most part).

It is a classic documentation dilemma that you cannot have everything you want
to know about something in a single place. What you want to know about something
is different from what person X wants/needs to know. And what you need to know
today is different from what you need tomorrow.

This is still a problem even if you are dealing with only a single, local manual
that is self-contained as a whole (contains all of the info on its subject). You
might want to see one organization into manual sections and person X might want
to see a different organization. And for you alone, the optimal organization
might be different today (looking for info on some topic) from tomorrow (looking
with a different eye).

IOW, there are *trade-offs*. People dealing with documentation professsionally
deal with such trade-offs everyday - it's a significant part of what they do.

Of course, there are trade-offs that many users might agree are reasonably good
ones, and there are trade-offs that many would agree are poor ones. Picking an
extremely poor documention organization as your example does not invalidate the
fact that trade-offs are always required.

> I've never subscribed to the theory that just because a feature is
> described on some web page out there, and not in the official manual,
> that it's "documented".  Why people want to see this as the success of
> the internet rather than as a failure of the documentation is 
> beyond me.

No one mentioned "some web page out there". You are simply arguing about a straw
man. The point is that Common Lisp has a standard definition (at least one!),
and the language is documented.

To the extent that Emacs supports some or all of that standard to various
degrees, the Emacs documentation needs to:

1. Be clear about that support, specifying which CL standard is supported and
specifying exactly any divergences from that standard.

2. Give clear guidance about any Emacs-specific uses of particular features from
that standard.

That does not imply that the *Emacs* doc needs to reproduce, in addition, a
complete documentation of the standard language that it (more or less) supports.
That's the point of having a standard that can be referenced.

Nothing prevents the Emacs doc from giving examples and explanation of how to
use particular Common Lisp constructs in an Emacs environment. Quite the
contrary.

> Keep the information itself in one place

It's a wonderfully naive idea. Get involved with documenting something
(non-trivial) and you will soon discover the trade-offs. Keeping X "in one
place" begs the question of what X is, how much X encompasses, and what point of
view X is regarded from (audience background, audience quest/need).

> that I can put on a tablet as a PDF, or print out and take with
> me on an overnight.  Not everyone reads their docs from within
> Emacs all of the time.
> 
> For what it's worth.

Re: Guile in Emacs

[christian.lynbech]

>>>>> "David" == David Kastrup <dak@gnu.org> writes:

David> The manuals of an editor with a user-accessible extension language (with
David> Emacs, this is the case for Elisp but not for C) need to document this
David> extension language.

Of course, but there is no reason why this information cannot be split
across several manuals and having the CL parts be documented by
respectively the CL standard (eg. the hyperspec) and whatever other
documentation exists for the CL implementation chosen to host emacs.

The documentation must be accessible but this is not the same as the
emacs maintainers need to (re)write all of it nor that it necessarily
must be put into one particular (already somewhat unwieldy) document.


------------------------+-----------------------------------------------------
Christian Lynbech       | christian #\@ defun #\. dk
------------------------+-----------------------------------------------------
Hit the philistines three times over the head with the Elisp reference manual.
                                        - petonic@hal.com (Michael A. Petonic)

Re: Guile in Emacs

[David Kastrup]

<christian.lynbech@tieto.com> writes:

>>>>>> "David" == David Kastrup <dak@gnu.org> writes:
>
> David> The manuals of an editor with a user-accessible extension
> David> language (with Emacs, this is the case for Elisp but not for C)
> David> need to document this extension language.
>
> Of course, but there is no reason why this information cannot be split
> across several manuals and having the CL parts be documented by
> respectively the CL standard (eg. the hyperspec) and whatever other
> documentation exists for the CL implementation chosen to host emacs.

Did you bother to read my posting to its end before replying to the
first sentence?  I should think that I already addressed this.

-- 
David Kastrup

Re: Guile in Emacs

[christian.lynbech]

>>>>> "David" == David Kastrup <dak@gnu.org> writes:

David> Did you bother to read my posting to its end before replying to the
David> first sentence?  I should think that I already addressed this.

My apologies if I have misunderstood something; I did read all of the
mail.

You wrote in the second paragraph:

> Otherwise, only programmers can be expected to be able to use it.  One
> reason for that is that non-Emacs specific Lisp manuals will not focus
> about how to get things done with Emacs.  Applying a manual utterly
> without editing focus to editing tasks is quite a large intellectual
> feat.

I would imagine that the system came with several manuals, a generic one
for common lisp (for instance the hyperspec) and a more emacs specific
one for the editing libraries (such as the buffer related functions).

Why would it make a big difference whether (say) `loop' and
`current-buffer' appear in different manuals as long as both are
available in the system and there are some appropriate top-level
index/search/contents/reading-guide support for the user?

It is even possible in some distant future that one would like to
unbundle some of the core emacs functionality (again take the buffer
functions as somewhat poor example) into standalone libraries that could
be made available to other applications supporting the same extension
language. In that case one would actually want to split up the current
emacs lisp manual into separate manuals that could accompany the
respective libraries.

------------------------+-----------------------------------------------------
Christian Lynbech       | christian #\@ defun #\. dk
------------------------+-----------------------------------------------------
Hit the philistines three times over the head with the Elisp reference manual.
                                        - petonic@hal.com (Michael A. Petonic)

Re: Guile in Emacs

[David Kastrup]

<christian.lynbech@tieto.com> writes:

>>>>>> "David" == David Kastrup <dak@gnu.org> writes:
>
> David> Did you bother to read my posting to its end before replying to the
> David> first sentence?  I should think that I already addressed this.
>
> My apologies if I have misunderstood something; I did read all of the
> mail.
>
> You wrote in the second paragraph:
>
>> Otherwise, only programmers can be expected to be able to use it.  One
>> reason for that is that non-Emacs specific Lisp manuals will not focus
>> about how to get things done with Emacs.  Applying a manual utterly
>> without editing focus to editing tasks is quite a large intellectual
>> feat.
>
> I would imagine that the system came with several manuals, a generic
> one for common lisp (for instance the hyperspec) and a more emacs
> specific one for the editing libraries (such as the buffer related
> functions).
>
> Why would it make a big difference whether (say) `loop' and
> `current-buffer' appear in different manuals as long as both are
> available in the system and there are some appropriate top-level
> index/search/contents/reading-guide support for the user?

Because the language specific manual would have no constructs and no
examples using any editor-specific data structures or any
editor-specific tasks.

But that's actually more or less a paraphrase of the "Otherwise, [...]"
paragraph above again.  I don't see anything in your reply that would
address that.

-- 
David Kastrup

Re: Guile in Emacs

[Helmut Eller]

* David Kastrup [2010-04-15 11:06+0200] writes:

> <christian.lynbech@tieto.com> writes:
>
>>>>>>> "David" == David Kastrup <dak@gnu.org> writes:
>>
>> David> Did you bother to read my posting to its end before replying to the
>> David> first sentence?  I should think that I already addressed this.
>>
>> My apologies if I have misunderstood something; I did read all of the
>> mail.
>>
>> You wrote in the second paragraph:
>>
>>> Otherwise, only programmers can be expected to be able to use it.  One
>>> reason for that is that non-Emacs specific Lisp manuals will not focus
>>> about how to get things done with Emacs.  Applying a manual utterly
>>> without editing focus to editing tasks is quite a large intellectual
>>> feat.
>>
>> I would imagine that the system came with several manuals, a generic
>> one for common lisp (for instance the hyperspec) and a more emacs
>> specific one for the editing libraries (such as the buffer related
>> functions).
>>
>> Why would it make a big difference whether (say) `loop' and
>> `current-buffer' appear in different manuals as long as both are
>> available in the system and there are some appropriate top-level
>> index/search/contents/reading-guide support for the user?
>
> Because the language specific manual would have no constructs and no
> examples using any editor-specific data structures or any
> editor-specific tasks.
>
> But that's actually more or less a paraphrase of the "Otherwise, [...]"
> paragraph above again.  I don't see anything in your reply that would
> address that.

Guile supposedly supports multiple languages, like Scheme, Python and
Javascript.  So who writes the manual for Python and Javascript?

Helmut

Re: Guile in Emacs

[christian.lynbech]

>>>>> "David" == David Kastrup <dak@gnu.org> writes:

David> Because the language specific manual would have no constructs and
David> no examples using any editor-specific data structures or any
David> editor-specific tasks.

Ah, now I begin to see more clearly what you are talking about (I think).

However, I am not really sure I understand why this is a problem. In the
editing library part of the manual, one could include the necessary
examples of how to express one self in the language related to that
feature. I do not see why, say, the description of `defun' or `loop'
necessarily need to use editing related examples.

The description of the core language will, IMHO, only really make sense
to programmers anyway. Non-programmers will (even today) find some
relevant code snippet, search for the relevant editing function and
apply some patterne matching between the examples there and the code
snippet. I doubt they will be anymore compelled to learn the language
just because examples have an editing focus.

I am also not entirely sure just how closely coupled the current manuals
are to the editing context. If one flips through the first dozen pages
of the emacs lisp introduction, there is nothing related to
editing. Flipping through the control structures chapter in the emacs
lisp reference manual shows a similar pattern. Most of those examples
uses `print'/`princ' rather than `insert'. This is obviously not a
thorough review of all of the manuals, but I am unconvinced that the
current manual could not be separated into a language part and an
editing library part with good effects if need be.

We will anyway have the same problem with Guile, I think. One of the
advantages of Guile is precisely that it is an independent piece of
software on which a future emacs will be based, among other applications
which also uses Guile.

I would expect Guile to have, and to continue to have, its own
manual. Whether this will include (in an integrated way) contents
equivalent to the relevant RxRS standards or just point to them and
focus on documenting the differences/extensions (as some CL
implementation do with the hyperspec) I do not know, but surely it will
be prudent for the Guile manual to be selfcontained and not littered
with examples that only makes sense in an emacs context.

Even though scheme is smaller than CL, and it could very well be that
even Guile is significantly smaller than some CL implementations, I hope
that the emacs maintainers would not opt to *duplicate* the Guile
documentation simply for the sake of being able to provide a version
with a more editing focus.


------------------------+-----------------------------------------------------
Christian Lynbech       | christian #\@ defun #\. dk
------------------------+-----------------------------------------------------
Hit the philistines three times over the head with the Elisp reference manual.
                                        - petonic@hal.com (Michael A. Petonic)

Re: Guile in Emacs

[Richard Stallman]

    The Elisp manual would not need to document things that are vanilla Common Lisp
    (i.e. respect the CL standard). It could simply cross-reference the relevant CL
    doc.

Do we have a good quality free Common Lisp manual?

Re: Guile in Emacs

[Harald Hanche-Olsen]

+ Richard Stallman <rms@gnu.org>:

> Do we have a good quality free Common Lisp manual?

Unfortunately, assuming the GNU definition of "free", I am afraid the
answer is no. You can read the copyright notice here:

http://www.lispworks.com/documentation/HyperSpec/Front/Help.htm#Legal

- Harald

Re: Guile in Emacs

[Harald Hanche-Olsen]

+ Harald Hanche-Olsen <hanche@math.ntnu.no>:

> You can read the copyright notice here:
> 
> http://www.lispworks.com/documentation/HyperSpec/Front/Help.htm#Legal

Of course, I meant to say the copyright notice of the HyperSpec.
But as far as I know, it is the only high quality reference that is
available for free (as in free beer), apart from Common Lisp the
Language 2nd ed (http://www.cs.cmu.edu/Groups/AI/html/cltl/cltl2.html),
whose license is not explicitly stated anywhere I can see. Besides,
it predates the CL standard, so it may be partially outdated.

- Harald

Re: Guile in Emacs

[Richard Stallman]

    apart from Common Lisp the
    Language 2nd ed (http://www.cs.cmu.edu/Groups/AI/html/cltl/cltl2.html),
    whose license is not explicitly stated anywhere I can see.

If it has no language, that means copying and modification are
prohibited by copyright law.

If you can find the authors, maybe we can convince them to free it.
 
    it predates the CL standard, so it may be partially outdated.

Updating it is surely much easier than starting from scratch.

Re: Guile in Emacs

[Helmut Eller]

* Richard Stallman [2010-04-15 20:47+0200] writes:

>     The Elisp manual would not need to document things that are
>     vanilla Common Lisp (i.e. respect the CL standard). It could
>     simply cross-reference the relevant CL doc.
>
> Do we have a good quality free Common Lisp manual?

The Tex sources of the "draft" of the ANSI standard are available on the
net[1].  I'm pretty sure that's the same text as the actual standard.
Bill Schelter used that to create an info version for GCL[2] and that is
also in the Debian package gclcvs-doc.  Independently, Jesper Harder
wrote some ELisp code[3] to convert the Tex code to texinfo which tries
to preserve more of the font information and cross references.

The legal status of the Tex sources seems a bit messy, though.
Apparently the intention of the involved parties was to put them into
Public Domain but somehow failed to make that formally correct[4].

Helmut

[1] http://www.cs.cmu.edu/afs/cs/project/ai-repository/ai/lang/lisp/doc/standard/ansi/0.html
[2] ftp://ftp.ma.utexas.edu/pub/gcl/gcl-info+texi.tgz
[3] http://www.phys.au.dk/~harder/dpans.html
[4] http://lists.debian.org/debian-legal/2004/06/msg00492.html

Re: Guile in Emacs

[Richard Stallman]

    The Tex sources of the "draft" of the ANSI standard are available on the
    net[1].

What license do they carry?

    The legal status of the Tex sources seems a bit messy, though.
    Apparently the intention of the involved parties was to put them into
    Public Domain but somehow failed to make that formally correct[4].

Can you boil down the facts they found and state them concisely?

Re: Guile in Emacs

[Helmut Eller]

* Richard Stallman [2010-04-17 06:40+0200] writes:

>     The Tex sources of the "draft" of the ANSI standard are available on the
>     net[1].
>
> What license do they carry?

Apparently none.  The only paragraph containing the word "copyright" is
this:

  Special thanks to Guy L. Steele Jr. and Digital Press for producing {\CLtL},
  and for relaxing copyright restrictions enough to make it possible for that
  document's text to provide an early basis of this work.

>
>     The legal status of the Tex sources seems a bit messy, though.
>     Apparently the intention of the involved parties was to put them into
>     Public Domain but somehow failed to make that formally correct[4].
>
> Can you boil down the facts they found and state them concisely?

At least in this thread, the Debian people didn't find any facts and
only stated that some lawyer would be needed to clarify the issues.

On another message board[1] Peter Seibel summarized the situation:

  So here's some info that I got from Steve Haflich of Franz and  
  onetime chair of XJ13, the committee that brought us the ANSI  
  Standard. (This is from a conversation we had standing in the lobby  
  of the Franz office building; I wasn't taking notes. Caveat Lector.)
  
    - At some point in the standardization process it became apparent  
  that there was a bunch of editorial work to be done and no one to do  
  it and no funding to pay someone to do it.
  
    - Various organizations involved in the standardization such as  
  Franz, Symbolics, Harlequin, Apple, and others decided that they  
  would each contract with Symbolics to pay for a Symbolic's employee,  
  Kent Pitman, to produce a draft standard which he would then "give"  
  to ANSI to do with whatever they wished.
  
    - All these organizations agreed that they would place the work (to  
  which they held copyright since they were paying Symbolics to do it  
  as a work for hire) into the public domain. Except some lawyer  
  pointed out that you can't really affirmatively put something in the  
  public domain. So they did something--not clear what--to assert their  
  copyright but to allow anyone to use the draft they were paying to  
  have produced for any purpose whatsoever.
  
    - That draft is the so-called dpANS2.
  
    - ANSI took the dpANS2 and made a few minor copy edits, slapped on  
  their logo and some front matter, and published it as the ANSI standard.
  
    - Kent Pitman, then at Harlequin then used the dpANS2 as the basis  
  for the HyperSpec. Franz similarly used it to make their HTML  
  version. Pitman also fought with ANSI to get permission to do  
  something (not clear exactly what) beyond what he would have been  
  allowed to do with dpANS2.
  
    - I'm pretty sure Harlequin (or Xanalys or Lispworks) owns the  
  copyright to the HyperSpec.
  
    - The issue of copyright on dpANS2 is muddied by the fact that it  
  includes big chunks of text that were written by Guy Steele for CLTL.  
  He, according to Haflich, donated that text to ANSI to use in the  
  standard but it's not clear that the folks (i.e. those companies)  
  that produced the dpANS actually had the right to use it. Obviously,  
  from a practical point of view, he and Digital Press, publishers of  
  CLTL2, haven't been bothered by the fact that their text is in the  
  HyperSpec and the dpANS, etc. but technically they could probably  
  make a stink. (Though maybe Pitman actually cleared that with them-- 
  he seems to make a point of being pretty scrupulous about  
  intellectual property issues.)
  
    - The issue of copyright on dpANS2 is also muddied by the many  
  small contributions of text by other people who participated in the  
  standardization process.
  
  So, to answer Don's question, probably not. If one wanted to take the  
  text of dpANS2 and use it for the basis of a derived work (say an  
  annotated version), and you wanted to be incredibly scrupulous about  
  making sure you weren't stepping on anyone's copyrights, you'd  
  probably need to track down the contracts wherein the companies that  
  funded the dpANS2 "licensed" it for use by anyone for any purpose.  
  Then you'd probably want to talk to Guy Steele and/or Digital Press.  
  And for good measure the known authors of any of the sections of the  
  dpANS2 that were written by someone else (e.g. Dick Waters, I  
  believe, wrote large chunks of the section on the pretty printer  
  since he invented it.) Then, if you really wanted to nail things  
  down, you'd probably need to contact the 100 or so folks who  
  participated in the standardization and who may have contributed text.
  
  But probably the right and most efficient thing to do is to find a  
  good IP lawyer and tell them what, specifically you want to do, and  
  ask them to help you figure out what you need to do to make sure  
  you're not exposing yourself to excessive liability by doing it.
  
  -Peter

I don't know how Pitman "asked" ANSI for permission he only writes this
in the HyperSpec:

  Parts of this work incorporate material taken from American National
  Standard X3.226, copyright 1994, and is used with permission of the X3
  Secretariat, ITI, 1250 Eye St., NW., Suite 200, Washington, DC 20005 and
  of the copyright holder, American National Standards
  Institute. ANSI/X3.226 was developed by Technical Committee X3J13,
  Common Lisp.

Helmut

[1] http://www.mail-archive.com/gardeners@lispniks.com/msg00189.html

Re: Guile in Emacs

[Richard Stallman]

    >     The Tex sources of the "draft" of the ANSI standard are available on the
    >     net[1].
    >
    > What license do they carry?

    Apparently none.  The only paragraph containing the word "copyright" is
    this:

      Special thanks to Guy L. Steele Jr. and Digital Press for producing {\CLtL},
      and for relaxing copyright restrictions enough to make it possible for that
      document's text to provide an early basis of this work.

Unless they state a license elsewhere, they are not free.

	- All these organizations agreed that they would place the work (to  
      which they held copyright since they were paying Symbolics to do it  
      as a work for hire) into the public domain. Except some lawyer  
      pointed out that you can't really affirmatively put something in the  
      public domain.

You can in the US, according to my lawyers.

		     So they did something--not clear what--to assert their  
      copyright but to allow anyone to use the draft they were paying to  
      have produced for any purpose whatsoever.

That would be tantamout to granting a license.
If we can find the statement they made, it might be adequate.

I will ask GLS if he can do anything about this.  However, unless you
hear about something that changes the situation, you may as well give
up on that text.

Re: Guile in Emacs

[John Wiegley]

On Apr 14, 2010, at 1:02 AM, Richard Stallman wrote:

> Common Lisp is extremely complicated and ugly.  When I wrote GNU Emacs
> I had just finished implementing Common Lisp, and I did not like it
> much.  It would bloat Emacs terribly, and documenting it would be hard
> too.

Do you mean the libraries are?  Because the core syntax of CL is pretty simple.  Also, with Scheme there are several core semantic differences from Emacs Lisp: a) It's a Lisp-1, and b) the difference between nil, () and #f.  Those two right there can make porting any significant Emacs application extremely difficult.

As for Common Lisp -- or let's say, a functional subset for the purpose of making it an Emacs extension language -- if, as some suggest, it's worthwhile to ignore existing code and start afresh, why isn't a project like CLmacs more interesting?  It's very Emacs like, but lacks all the useful libraries which make Emacs what it is.

Personally I think adopting a new internal language for Emacs, just so GNU can have a common scripting language, is somewhat ludicrous.  Emacs Lisp has proven itself to be incredibly useful, as shown by the enormous number of contributors and extensive library of code -- some of which has seen extensive usage for a decade or more now.  It may have its warts, but so will any system by the time its use\x01rs are done with it.

Lastly, I'm not at all convinced Emacs Lisp needs multithreading.  I would prefer a coarser model of process separation to allow things like Gnus to run in tandem with other tasks.  It would be nice if a wholly separate process could connect to the frames/windows/buffers (but not symbol space) of a master Emacs process.  I was just considering yesterday the truly scary-to-debug scenarios that come up when you realize dynamic variables can change mid-form, or that hooks might run while other hooks are executing.  There is so much code that depends on sequence of execution that I think multithreading could cause us stability headaches for years and years.

John

Re: Guile in Emacs

[Helmut Eller]

* John Wiegley [2010-04-14 15:30+0200] writes:

> Lastly, I'm not at all convinced Emacs Lisp needs multithreading.  I
> would prefer a coarser model of process separation to allow things
> like Gnus to run in tandem with other tasks.  It would be nice if a
> wholly separate process could connect to the frames/windows/buffers
> (but not symbol space) of a master Emacs process.  I was just
> considering yesterday the truly scary-to-debug scenarios that come up
> when you realize dynamic variables can change mid-form, or that hooks
> might run while other hooks are executing.  There is so much code that
> depends on sequence of execution that I think multithreading could
> cause us stability headaches for years and years.

Amen.  Also consider that all the web browsers move away from
multithreading and use processes these days.  Good that Emacs is so slow
at adapting new techologies; maybe we can take a shortcut this time and
skip the multithreading dead-end :-)

Helmut

Re: Guile in Emacs

[Tom Tromey]

John> Lastly, I'm not at all convinced Emacs Lisp needs multithreading.
John> I would prefer a coarser model of process separation to allow
John> things like Gnus to run in tandem with other tasks.

I did consider this model.  I even floated the idea on this list, though
as I recall nobody replied.

One thing worth noting is that you can experiment with this approach
right now.  I haven't tried but I think you can do it purely in elisp
with no emacs changes required.

Give it a try.  If it is a better model, well, great, let's use it.

John> I was just considering yesterday the truly scary-to-debug
John> scenarios that come up when you realize dynamic variables can
John> change mid-form

That is true, but due to the nature of actually existing Emacs code I
don't think it will be a problem in practice.

Also: (1) this cannot happen with let-bound variables, only globals; and
(2) this can already happen today via process filters (in limited
scenarios, to be sure -- but the threading is cooperative, so the same
applies).

Tom

Re: Guile in Emacs

[David Kastrup]

Christian Lynbech <christian@defun.dk> writes:

>>>>>> "Thomas" == Thomas Lord <lord@emf.net> writes:
>
>>> Wouldn't it be about as good (and probably less work), to give up on the
>>> guile idea and evolve emacs lisp (with Miles's lexical-bind changes, and
>>> (one hopes) multithreading, and maybe other things)?
>
> Thomas> I dunno.  Maybe.  I'd guess that, no, that's not a 
> Thomas> good strategy.   Four reasons come quickly to mind: ...
>
> Wouldn't a reasonable alternative to using a scheme implementation be to
> use a Common Lisp ditto? And wouldn't that be a much closer fit
> semantically to the current Emacs Lisp dialect?

Emacs Lisp is streamlined for editing.  Common Lisp has its own focus.
For an extension language, it is preferable to have a system where you
can read through the manual in one day and basically understand it.

Scheme is a smaller starting point than Common Lisp.

-- 
David Kastrup

Re: Guile in Emacs

[christian.lynbech]

>>>>> "David" == David Kastrup <dak@gnu.org> writes:

David> Emacs Lisp is streamlined for editing.  Common Lisp has its own focus.

You could say the same of Scheme.

I am also not quite sure I understand how Emacs Lisp as such is
streamlined for editing. 

Sure there are a bunch of libraries, for instance for handling buffers,
but I think of that as a library and not really an integral part of the
language. Whatever the solution, we will always have an `current-buffer'
function.

David> For an extension language, it is preferable to have a system where you
David> can read through the manual in one day and basically understand it.

Well, that is not exactly wrong, but I would maintain that you can write
a Common Lisp tutorial that you could read through and understand in a
day as easily as you can for Scheme or the current Emacs Lisp. 

Much of what is in the Common Lisp standard has the character of
libraries; you need not understand pathnames or CLOS to get started
writing functions.  

Wrt. libraries you will anyway be in for a lifelong journey, adding a
handfull on top of what is already in Emacs will not drastically change
that.


------------------------+-----------------------------------------------------
Christian Lynbech       | christian #\@ defun #\. dk
------------------------+-----------------------------------------------------
Hit the philistines three times over the head with the Elisp reference manual.
                                        - petonic@hal.com (Michael A. Petonic)

Re: Guile in Emacs

[immanuel litzroth]

> Emacs Lisp is streamlined for editing.  Common Lisp has its own focus.
Common Lisp doesn't have a focus that I am aware of. It is the language
that is most close to Emacs Lisp syntactically, and most emacs code that
doesn't have editor specific stuff will run in Common Lisp & vice versa.

> For an extension language, it is preferable to have a system where you
> can read through the manual in one day and basically understand it
The effort to "basically" understand CL is the same as the for scheme. Moreover
scheme has some exotic stuff like hygienic macros and continuations which are
not stuff you "basically understand in a day".

> Scheme is a smaller starting point than Common Lisp.
So with a common lisp system you get:
1) compilation to machine code
2) standardized implementation of classes
3) structures, hashes
4) Exceptions
With a scheme system you get
1) call-with-current-continuation

Moreover if you really like emacs based on scheme why not go for edwin?
Immanuel

Re: Guile in Emacs

[christian.lynbech]

>>>>> immanuel litzroth <ilitzroth@gmail.com> writes:

>> Scheme is a smaller starting point than Common Lisp.
> So with a common lisp system you get:
> 1) compilation to machine code
> 2) standardized implementation of classes
> 3) structures, hashes
> 4) Exceptions
> With a scheme system you get
> 1) call-with-current-continuation

This is probably a little unfair to Scheme. 

I do not know where the latest scheme standard is wrt. OO and exceptions
but many concrete implementations will have solutions for most of
this. Again, in practice, there just is a bunch of stuff you need and
whether you put it into the standard or bolt it onto your concrete
implementation is not all that important.

There are also scheme implementation that compile to native code and
although Guile isn't among them, it has a reasonable enough performance
on modern hardware that you for instance can successfully run a window
manager implemented in Guile (which I did for a couple of years).

------------------------+-----------------------------------------------------
Christian Lynbech       | christian #\@ defun #\. dk
------------------------+-----------------------------------------------------
Hit the philistines three times over the head with the Elisp reference manual.
                                        - petonic@hal.com (Michael A. Petonic)

Re: Guile in Emacs

[Stephen J. Turnbull]

christian.lynbech@tieto.com writes:

 > There are also scheme implementation that compile to native code and
 > although Guile isn't among them, it has a reasonable enough performance
 > on modern hardware that you for instance can successfully run a window
 > manager implemented in Guile (which I did for a couple of years).

That's hardly a recommendation -- window managers are not very
demanding of performance.  Heck, you can run a window manager (xwem)
implemented in Emacs Lisp (which in turn is based on X protocol
implemented in Emacs Lisp -- xlib).  Several SXEmacs guys do.  (Not to
mention anybody running sawfish -- librepl was basically Emacs Lisp on
Ritalin.)

Re: Guile in Emacs

[christian.lynbech]

>>>>> "Stephen" == Stephen J Turnbull <stephen@xemacs.org> writes:

Stephen> That's hardly a recommendation -- window managers are not very
Stephen> demanding of performance.

True, though one could argue that anything that sits between you and
your windows will be very visible if there is even the slightest lag.

Anyway, I just wanted to say that the performance issues of Guile is not
the most overwhelming problem we face (and I haven't touched Guile in
years so I do not actually know how it competes these days). 

I am certainly prepared to believe that Guile is faster than the current
Emacs Lisp engine but I am also absolutely convinced that unless it has
improved radically it will fall noticeably behind systems that compiles
to native code, whether directly or via C.


------------------------+-----------------------------------------------------
Christian Lynbech       | christian #\@ defun #\. dk
------------------------+-----------------------------------------------------
Hit the philistines three times over the head with the Elisp reference manual.
                                        - petonic@hal.com (Michael A. Petonic)

Re: Guile in Emacs

[Ken Raeburn]

On Apr 14, 2010, at 08:26, <christian.lynbech@tieto.com> <christian.lynbech@tieto.com> wrote:
> I am certainly prepared to believe that Guile is faster than the current
> Emacs Lisp engine

I have no idea if that's true, at the moment.

> but I am also absolutely convinced that unless it has
> improved radically

If you haven't used it in a while, it may have -- the current development version uses a byte code engine for execution, which I believe has sped most stuff up quite a bit.

> it will fall noticeably behind systems that compiles
> to native code, whether directly or via C.

Both approaches are getting serious consideration from the current Guile developers.

Ken

Re: Guile in Emacs

[Tom Tromey]

>>>>> "Ken" == Ken Raeburn <raeburn@raeburn.org> writes:

>> it will fall noticeably behind systems that compiles
>> to native code, whether directly or via C.

Ken> Both approaches are getting serious consideration from the current
Ken> Guile developers.

FWIW, there was once an elisp JIT compiler.

http://www.mundell.ukfsn.org/native/

It was rejected:

http://lists.gnu.org/archive/html/emacs-devel/2004-06/msg00103.html

Tom

Re: Guile in Emacs

[Ken Raeburn]

On Apr 14, 2010, at 14:29, Tom Tromey wrote:
> FWIW, there was once an elisp JIT compiler.

> It was rejected:

Interesting!  I'd wondered if there'd be anything to gain from doing that, but stopped short of actually trying it myself.  I'm glad to see someone else has tried it, even if the conclusion from the experiment was that it wasn't worth folding in.  Actually, I'd been thinking about converting byte-code to C, changing stack slots to register variables, that sort of thing, and turning GCC's optimizer loose on it, but I doubt it'd perform all that much better than the JIT version if the primitives were dominating.

Thanks for the links!

Ken

Re: Guile in Emacs

[Jeff Clough]

It seems I've lost track of this conversation.  Is this a purely
hypothetical discussion, or is it to be expected that some relatively
near future version of Emacs will be Guile-based?

For what it's worth, as someone who's used Emacs for years but only got
into Lisp a couple of months ago...

I can see this as a good thing for people who'd like a more "modern"
Lisp with their Emacs (and a consistent, "core" extension language
across GNU), but a bad thing for people relying on a large, complex
application maintained by a single busy guy.

Emacs Lisp supports such a grow-it-as-you-need-it development approach
that a lot of stuff out there was built a few lines at a time over
months or years.  Expecting the author of an application like that to
carve out a chunk of time for a complete re-write might be asking too
much.

Jeff

Re: Guile in Emacs

[Stephen J. Turnbull]

Jeff Clough writes:

 > Expecting the author of an application like that to carve out a
 > chunk of time for a complete re-write might be asking too much.

Mike Sperber supervised a student at U. Tuebingen who wrote a
proof-of-concept Emacs Lisp system in Scheme48, with quite reasonable
performance characteristics for a student project.  His estimate was
that performance equivalent to existing bytecode of the time was
achievable with reasonable effort.

Re: Guile in Emacs

[Stephen Eilert]

On Wed, Apr 14, 2010 at 6:55 AM, immanuel litzroth <ilitzroth@gmail.com> wrote:
>> Emacs Lisp is streamlined for editing.  Common Lisp has its own focus.
> Common Lisp doesn't have a focus that I am aware of. It is the language
> that is most close to Emacs Lisp syntactically, and most emacs code that
> doesn't have editor specific stuff will run in Common Lisp & vice versa.
>
>> For an extension language, it is preferable to have a system where you
>> can read through the manual in one day and basically understand it
> The effort to "basically" understand CL is the same as the for scheme. Moreover
> scheme has some exotic stuff like hygienic macros and continuations which are
> not stuff you "basically understand in a day".
>
>> Scheme is a smaller starting point than Common Lisp.
> So with a common lisp system you get:
> 1) compilation to machine code
> 2) standardized implementation of classes
> 3) structures, hashes
> 4) Exceptions
> With a scheme system you get
> 1) call-with-current-continuation
>

And who said you won't get native code, classes, structures, hashes
and exceptions with scheme? They are just not documented in R5RS
(let's forget R6RS ever existed). Most scheme implementations have
these features, they are not portable however.

Which is hardly a concern for Guile and Emacs Guile.


--Stephen

programmer, n:
        A red eyed, mumbling mammal capable of conversing with
inanimate monsters.

Re: Guile in Emacs

[Helmut Eller]

* Stephen Eilert [2010-04-14 15:11+0200] writes:

> On Wed, Apr 14, 2010 at 6:55 AM, immanuel litzroth <ilitzroth@gmail.com> wrote:
>>> Emacs Lisp is streamlined for editing.  Common Lisp has its own focus.
>> Common Lisp doesn't have a focus that I am aware of. It is the language
>> that is most close to Emacs Lisp syntactically, and most emacs code that
>> doesn't have editor specific stuff will run in Common Lisp & vice versa.
>>
>>> For an extension language, it is preferable to have a system where you
>>> can read through the manual in one day and basically understand it
>> The effort to "basically" understand CL is the same as the for scheme. Moreover
>> scheme has some exotic stuff like hygienic macros and continuations which are
>> not stuff you "basically understand in a day".
>>
>>> Scheme is a smaller starting point than Common Lisp.
>> So with a common lisp system you get:
>> 1) compilation to machine code
>> 2) standardized implementation of classes
>> 3) structures, hashes
>> 4) Exceptions
>> With a scheme system you get
>> 1) call-with-current-continuation
>>
>
> And who said you won't get native code, classes, structures, hashes
> and exceptions with scheme? 

Those who say that Scheme is small and elegant.  If Scheme has all the
features of CL it's no longer smaller than CL.

> They are just not documented in R5RS (let's forget R6RS ever
> existed).  Most scheme implementations have these features, they are
> not portable however.

Comparing implementations of one language with the specification of
another language makes as much sense as comparing apples with oranges.

Helmut

Re: Guile in Emacs

[Tom Tromey]

>>>>> "David" == David Kastrup <dak@gnu.org> writes:

David> Scheme is a smaller starting point than Common Lisp.

But we aren't talking about "Scheme", we're talking about Guile, which
is significantly larger.  It has modules and OO and all kinds of other
stuff.

While I rather like CL, and much prefer it to Scheme, I don't see why we
need to consider either.  I don't see much wrong with elisp as it is.
In my view it simply is not among the major limiting factors of Emacs.

Tom

RE: Guile in Emacs

[Drew Adams]

> While I rather like CL, and much prefer it to Scheme, I don't 
> see why we need to consider either.  I don't see much wrong
> with elisp as it is. In my view it simply is not among the
> major limiting factors of Emacs.

FWIW, I agree. I don't see a need to change things fundamentally.

If some particular aspects of either CL or Scheme are considered desirable
individually, then we could move toward accommodating them (adopting/adapting).

Incremental evolution that way, in the normal Emacs base code (as opposed to
creating a parallel Scheme-based or CL-based Emacs), would help ensure that
non-core libraries, including 3rd-party libraries, kept pace. Emacs is not just
someone's term project.

The main and most important difference from Elisp, for both CL and Scheme, lies
in more robust support for lexical scoping. CL's approach to mixing lexical and
dynamic scoping would be a reasonable one for Emacs Lisp. We currently have
`lexical-let' and that's about all. Common Lisp goes much further toward taking
advantage of lexical scoping than we currently do in Emacs.

I might add that a better time to have considered this would have been 1985 or
1990, not 2010. ;-) When both CL and GNU Emacs were young would have been the
perfect time. Richard is not crazy about CL, and my guess is that that's the
reason we aren't as close to it now as we might be.

I do, BTW, agree with Richard that there is a lot in CL that Emacs does not
necessarily need. But the basic design of CL would not be a bad one for Emacs,
IMO.

I also suspect that if there were really much interest in moving in this
direction, then we would already have moved Emacs Lisp a little closer to CL. It
just hasn't happened.

It is also the case that Emacs would already have much more CL-like and
CL-somewhat-compatible code now if there were not so much push-back to using CL
functions at runtime. Don't get me wrong - I support the push-back.

My point is only that if you look at 3rd-party Emacs-Lisp libraries (not mine
;-)), especially by younger developers, you'll see widespread use of cl.el
functions. That's an indication that people would use CL constructs if they were
more solidly part of Emacs Lisp. The evolution toward more CL-ness would thus
partly take care of itself. The problem is that the foundation is shaky.

Re: Guile in Emacs

[David De La Harpe Golden]

Drew Adams wrote:
> The main and most important difference from Elisp, for both CL and Scheme, lies
> in more robust support for lexical scoping. CL's approach to mixing lexical and
> dynamic scoping would be a reasonable one for Emacs Lisp.

islisp's [1] approach is a little bit better IIRC.  Islisp impls are few 
and far between of course.  Emacs Lisp and CL or Islisp are far, far 
closer than emacs lisp and scheme.

[1] http://islisp.info/

Re: Guile in Emacs

[Leo]

On 2010-04-14 16:47 +0100, Tom Tromey wrote:
> While I rather like CL, and much prefer it to Scheme, I don't see why
> we need to consider either. I don't see much wrong with elisp as it
> is. In my view it simply is not among the major limiting factors of
> Emacs.

Keep elisp but extend it with some goodies from common lisp or scheme
sounds good to me.

Leo

Re: integer overflow

[Helmut Eller]

* Stefan Monnier [2010-03-06 16:45+0100] writes:

>>>> Can you summarize how it's supposed to work?
>>> The change affects the Lisp reader: reading "536870912" or any larger
>>> number now gives you a float Lisp object, instead of an integer Lisp
>>> object (since there is no way to represent 536870912 or larger as
>>> integers).  536870911 and below are still read as integers.
>> What's the point of doing that?  I would much prefer if read would just
>> raise an error.
>
> There are many cases where it's important to read an integer from some
> other program, so it's important to be able to read integers larger the
> Elisp's largest fixnum (tho less so for integers larger than the
> host system's natural "int").
> Examples are file sizes, process ids, user ids, ...

read doesn't seem the right tool for this job; read is for parsing Lisp
syntax.  Those few programs which produce Lisp syntax should probably
just write floats if there is a danger of overflow and precision doesn't
matter.  For those cases were precision does matter (eg. a 32bit Emacs
reading a file produced by a 64bit Emacs) I'd prefer an error instead of
automatic coercions.

Helmut

Re: integer overflow

[Stefan Monnier]

> read doesn't seem the right tool for this job; read is for parsing Lisp
> syntax.

I can partly agree with the feeling, but since writing a lexer&parser in
Elisp results in fairly poor performance, it's common to (ab)use the
Lisp reader when possible.

The use of bignums would probably be a generally better option than
floats, but noone has tried to solve this yet (there have been some
patches that add bignums, but they still require a fair bit of work).


        Stefan

Re: integer overflow

[Richard Stallman]

    There are many cases where it's important to read an integer from some
    other program, so it's important to be able to read integers larger the
    Elisp's largest fixnum (tho less so for integers larger than the
    host system's natural "int").
    Examples are file sizes, process ids, user ids, ...

Would it make sense to set up a variable to control this?
You could bind it to t for those special cases.

Re: integer overflow

[Johan Bockgård]

Chong Yidong <cyd@stupidchicken.com> writes:

> Glenn Morris <rgm@gnu.org> writes:
>
>> I don't really understand it. In a 32-bit Emacs:
>>
>> (1+ 536870911) -536870912
>> (+ 2 536870911) -536870911
>> (1+ 536870912)  536870913.0
>> (1+ 4294967295) overflow
>> (1+ 4294967295.0) 4294967296.0
>>
>> Can you summarize how it's supposed to work?
>
> The change affects the Lisp reader: reading "536870912" or any larger
> number now gives you a float Lisp object

or an error if the integer is larger than 32 bits, as the fourth example
shows.

Re: integer overflow

[Chong Yidong]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> Indeed, I thought about it as a pure bug-fix, but it needs to be added
> to the NEWS.  Can you do that and update the doc accordingly?

I've just done so.

Re: Next pretest, and branching plans

[Christoph]

On 2/22/2010 10:31 PM, Jason Rumney wrote:
> On 22/02/2010 05:46, Chong Yidong wrote:
>
>> Jason, do you think you could down write some instructions?  You could
>> put it at the end of admin/make-tarball.txt, or as its own file in the
>> admin/ directory.
>
> I don't have access to bzr at the moment, but here are the steps 
> required:
>
> 3. "make install" to create the bin subdirectory with the appropriate 
> files.

I am trying to package the Windows binaries per Jason's instructions and 
I was wondering about the make install command on Windows: without 
specifying a prefix it installs in the current directory but it also 
does other stuff, for example adding a shortcut to the start menu (by 
calling addpm.exe).

Would it make sense to create a 'make package-install' target that omits 
these things (if there is other stuff besides the shortcut, that is more 
intended for a real installation rather than packaging)? When I am 
packaging I don't want the shortcut created.

Or maybe I am just overlooking an obvious way to do this?

Thanks,
Christoph

Re: Next pretest, and branching plans

[Eli Zaretskii]

> Date: Fri, 12 Mar 2010 20:10:09 -0700
> From: Christoph <cschol2112@googlemail.com>
> 
> I am trying to package the Windows binaries per Jason's instructions and 
> I was wondering about the make install command on Windows: without 
> specifying a prefix it installs in the current directory but it also 
> does other stuff, for example adding a shortcut to the start menu (by 
> calling addpm.exe).

I think "make install" does that even if you do specify the directory
to install in.

> Would it make sense to create a 'make package-install' target that omits 
> these things (if there is other stuff besides the shortcut, that is more 
> intended for a real installation rather than packaging)? When I am 
> packaging I don't want the shortcut created.

Patches are welcome.  While at that, perhaps have that package-install
target run the admin/nt/makedist.bat automatically.

Re: Next pretest, and branching plans

[Christoph]

On 3/13/2010 12:42 AM, Eli Zaretskii wrote:
>> Would it make sense to create a 'make package-install' target that omits
>> these things (if there is other stuff besides the shortcut, that is more
>> intended for a real installation rather than packaging)? When I am
>> packaging I don't want the shortcut created.
>>      
> Patches are welcome.  While at that, perhaps have that package-install
> target run the admin/nt/makedist.bat automatically.
>    
Sounds good, I will take a stab at that.

Thanks,
Christoph

Re: Next pretest, and branching plans

[Stefan Monnier]

> Would it make sense to create a 'make package-install' target that omits
> these things (if there is other stuff besides the shortcut, that is more
> intended for a real installation rather than packaging)? When I am packaging
> I don't want the shortcut created.

To me at least, the name "package-install" would not be helpful.
Something like "install_files_only" would sound more meaningful (or
"install_for_packaging", or ...).


        Stefan

Re: Next pretest, and branching plans

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Date: Sat, 13 Mar 2010 13:30:28 -0500
> Cc: emacs-devel@gnu.org
> 
> > Would it make sense to create a 'make package-install' target that omits
> > these things (if there is other stuff besides the shortcut, that is more
> > intended for a real installation rather than packaging)? When I am packaging
> > I don't want the shortcut created.
> 
> To me at least, the name "package-install" would not be helpful.
> Something like "install_files_only" would sound more meaningful (or
> "install_for_packaging", or ...).

I suggest "dist", like in the top-level Makefile.in.  There's no need
for this target without actually producing the binary zip file, so it
should run the makedist.bat script as part of its job.

Re: Next pretest, and branching plans

[Jason Rumney]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> Would it make sense to create a 'make package-install' target that omits
>> these things (if there is other stuff besides the shortcut, that is more
>> intended for a real installation rather than packaging)? When I am packaging
>> I don't want the shortcut created.
>
> To me at least, the name "package-install" would not be helpful.
> Something like "install_files_only" would sound more meaningful (or
> "install_for_packaging", or ...).

My preference would be for make install to install the files only, and a
new rule for making shortcuts (install_icons).

Most people who build from source on Windows are probably building in
the same location all the time, so they don't always need to replace the
shortcut. And if they have multiple versions installed, they will want
to maintain the shortcut icons manually to avoid having all the versions
overwrite each other.

Re: Next pretest, and branching plans

[Christoph]

On 3/13/2010 7:38 PM, Jason Rumney wrote:
> Stefan Monnier<monnier@iro.umontreal.ca>  writes:
>    
>>> Would it make sense to create a 'make package-install' target that omits
>>> these things (if there is other stuff besides the shortcut, that is more
>>> intended for a real installation rather than packaging)? When I am packaging
>>> I don't want the shortcut created.
>>>        
>> To me at least, the name "package-install" would not be helpful.
>> Something like "install_files_only" would sound more meaningful (or
>> "install_for_packaging", or ...).
>>      
> My preference would be for make install to install the files only, and a
> new rule for making shortcuts (install_icons).
>
> Most people who build from source on Windows are probably building in
> the same location all the time, so they don't always need to replace the
> shortcut. And if they have multiple versions installed, they will want
> to maintain the shortcut icons manually to avoid having all the versions
> overwrite each other.
>    
For packaging, a make dist rule would also need to copy the dist files 
(libXpm.dll for example) before invoking makedist.bat to really automate 
the entire process.

I agree with you, Jason, that installing shortcuts should be a separate 
step and not the included in the normal make install (for pretty much 
the same reasons you pointed out). But that to me, is a separate issue 
from packaging.

I went ahead and implemented the following so far in my local branch:

- added an option "--distfiles [path to file, for example libXpm.dll]" 
to configure.bat.
Adding those external binaries was the only manual step left in the 
process and can also be automated now.

- added build target 'dist' to makefile.w32-in.
It basically does a 'make install' without creating shortcuts, copies 
the distfiles, i.e. the libXpm.dll to the bin directory and then calls 
'makedist.bat' to create the zip file for distribution.

One problem is that calling makedist.bat means a dependency on the 
trunk, since it is not available in the source tarball. Can we add the 
/admin/nt directory and its contents to the source tarball? Or move the 
files to ../nt? Then, the 'make dist' target would be able to create a 
zip from just the tarball, without having to have the trunk available. 
But since the directory structure is the same, running 'make dist' would 
also work in the trunk itself to easily create binary snapshots of the 
trunk.

The makedist.bat needs to be changed a little because it expects the 
files to be in a folder emacs-xx.x.xx as they are in the tarball, but 
that is a trivial change change to make it more generic and automated.

Also, is there any way to get the version number from a file contained 
in the source tar ball? Then make dist would always output a zip file 
properly named according to the current version.

Any thoughts?

Christoph

Re: Next pretest, and branching plans

[Jason Rumney]

Christoph <cschol2112@googlemail.com> writes:

> One problem is that calling makedist.bat means a dependency on the
> trunk, since it is not available in the source tarball. Can we add the
> /admin/nt directory and its contents to the source tarball? Or move
> the files to ../nt? Then, the 'make dist' target would be able to
> create a zip from just the tarball, without having to have the trunk
> available. But since the directory structure is the same, running
> make dist' would also work in the trunk itself to easily create binary
> snapshots of the trunk.

Moving the files to the top level nt subdirectory would be OK.

> Also, is there any way to get the version number from a file contained
> in the source tar ball? Then make dist would always output a zip file
> properly named according to the current version.

The lib-src makefile contains the version number on Windows.  You could
update admin/admin.el if you want to move this to nt/makefile.

Re: Next pretest, and branching plans

[Eli Zaretskii]

> Date: Sat, 13 Mar 2010 19:50:30 -0700
> From: Christoph <cschol2112@googlemail.com>
> Cc: Stefan Monnier <monnier@iro.umontreal.ca>, emacs-devel@gnu.org
> 
> - added an option "--distfiles [path to file, for example libXpm.dll]" 
> to configure.bat.

That's gratuitous, I think: modern Windows shells are powerful enough
to let you write a FOR loop looking for libXpm.dll along PATH.

> Also, is there any way to get the version number from a file contained 
> in the source tar ball? Then make dist would always output a zip file 
> properly named according to the current version.

Again, one of the variants of the FOR command should do the trick.

Re: Next pretest, and branching plans

[Christoph]

On 3/14/2010 11:55 AM, Eli Zaretskii wrote:
>> Date: Sat, 13 Mar 2010 19:50:30 -0700
>> From: Christoph<cschol2112@googlemail.com>
>> Cc: Stefan Monnier<monnier@iro.umontreal.ca>, emacs-devel@gnu.org
>>
>> - added an option "--distfiles [path to file, for example libXpm.dll]"
>> to configure.bat.
>>      
> That's gratuitous, I think: modern Windows shells are powerful enough
> to let you write a FOR loop looking for libXpm.dll along PATH.
>    
This assumes that libXpm.dll is actually somwhere on the PATH, which it 
might or might not be (and I might not put it on the path). I think  the 
command line option for configure.bat is way more flexible for one's 
individual build environment, plus who knows, maybe I want to also 
package libSvg.dll or libWhatever.dll. Now I can do this without having 
to change the code.

>> Also, is there any way to get the version number from a file contained
>> in the source tar ball? Then make dist would always output a zip file
>> properly named according to the current version.
>>      
> Again, one of the variants of the FOR command should do the trick.
>    
Could you elaborate on this solution? Thanks!

Christoph

PS: As for the "powerful Windows shells"...not before Windows 7 (or 
Vista?) with Powershell did Windows ever have a powerful shell... ;)

Re: Next pretest, and branching plans

[Eli Zaretskii]

> Date: Tue, 16 Mar 2010 18:29:17 -0600
> From: Christoph <cschol2112@googlemail.com>
> Cc: emacs-devel@gnu.org, monnier@iro.umontreal.ca, jasonr@gnu.org
> 
> >> - added an option "--distfiles [path to file, for example libXpm.dll]"
> >> to configure.bat.
> >>      
> > That's gratuitous, I think: modern Windows shells are powerful enough
> > to let you write a FOR loop looking for libXpm.dll along PATH.
> >    
> This assumes that libXpm.dll is actually somwhere on the PATH, which it 
> might or might not be (and I might not put it on the path).

If it's not on PATH, it's in the directory where you have the Emacs
binary.

> I think  the 
> command line option for configure.bat is way more flexible for one's 
> individual build environment

Yes, but flexibility comes at a price: one needs _always_ to type that
argument.  Why impose this inconvenience on the user?

> >> Also, is there any way to get the version number from a file contained
> >> in the source tar ball? Then make dist would always output a zip file
> >> properly named according to the current version.
> >>      
> > Again, one of the variants of the FOR command should do the trick.
> >    
> Could you elaborate on this solution? Thanks!

Type "for /?" from the cmd prompt, and read there, especially about
"for /f".  It's too long to post that info here.

> PS: As for the "powerful Windows shells"...not before Windows 7 (or 
> Vista?) with Powershell did Windows ever have a powerful shell... ;)

I was talking about cmd, not about Powershell.

Re: Next pretest, and branching plans

[Jason Rumney]

Eli Zaretskii <eliz@gnu.org> writes:

> If it's not on PATH, it's in the directory where you have the Emacs
> binary.

Well, no. The directory where he has the Emacs binary has just been
created via make install, so it doesn't contain any files that are not
part of Emacs.

> Yes, but flexibility comes at a price: one needs _always_ to type that
> argument.  Why impose this inconvenience on the user?

The make target Christoph is creating is for packaging Emacs. Most users
will have no need to use it.