CL package serious deficiencies

CL package serious deficiencies

[egnarts-ms]

Hello to everyone ! In this message I would like to share my (mostly
negative) experience with CL package of Elisp.  I'm sure, some Emacs/Elisp
gurus are already familiar with many of these, but I still believe I have to
say something new.  Let's start.

1. The most striking thing is the way symbol macros are implemented: authors
of CL merely put cons cells ("symbol-name" <expansion>) onto
`cl-macro-environment' variable, and when the time of possible
macroexpansion comes, this alist is searched with assq -- that it, strings
are compared with `eq'.
  This obviously is not correct. Look at this example:

(defmacro test-macro ()
  (let* ((symbol-1 'sym)
	 (symbol-2 (make-symbol (symbol-name symbol-1))))
    `(let ((v1 0)
	   (v2 0))
       (symbol-macrolet ((,symbol-1 (incf v1))
			          (,symbol-2 (incf v2)))
	 ,symbol-1
	 ,symbol-2
	 (list v1 v2)))))

(princ (test-macro))

This code prints (0 2), when it should print (1 1): that's because the same
string object serves the purpose of being the symbol name for 2 distinct
symbols. This situation is only possible if we deal with uninterned symbols
(and in more-or-less serious Lisp programming this practice is quite
common).

I tested this exampe (as well as all the following ones) in CLISP 2.41.1; it
prints (1 1).


2. Generalized variables.

`get-setf-method' works by calling `macroexpand' when it cannot find a
setf-method for a form.  `macroexpand' honestly expands this form the whole
way down, without regarding to all the intermediate forms; 
http://www.lispworks.com/documentation/HyperSpec/Body/05_abg.htm Common Lisp
uses  `macroexpand-1' for the same purpose.  This means that in case of any
intermediate macroexpansion result having a setf-method defined, it will be
ignored.  Only the end non-macro-call form is considered for setf-method.

I cannot give right away a good example of a harm this may cause. 
Nevertheless, this behavior seems fishy.. for instance, defining
setf-methods for macro names may lead to counterintuitive things: imagine
there is some other macro named B that expands into the call of this macro
(named, say, A), and A expands into a non-macro form X (for instance, just
car of smth).  Given that B has no setf-method defined, when
`get-setf-method' searches for the setf-method for B, the setf-method for A
doesn't count at all: it is just skipped.  X is what is searched in this
case.

`macroexpand-1' is absent in Emacs Lisp.


3. The way `get-setf-method', `incf', `decf', `push', `pushnew' and `pop'
deal with symbol macros.
Look at this:
(defun get-setf-method (place &optional env)
  (if (symbolp place)
      (let ((temp (make-symbol "--cl-setf--")))
	(list nil nil (list temp) (list 'setq place temp) place))
    ...

`get-setf-method' is absolutely ignorant of symbol macros: the fact that
"place" satisfies "symbolp" doesn't in any way mean it is a simple variable,
since we have a concept of symbol macro in the language.  The same thing is
true about all 5 functions listed above. 

To see a real example, consider the following:

(cl-macroexpand-all
 '(symbol-macrolet ((s (cadr (get-some-list))))
    (incf s)))

which results in

(let* ((--cl-x-- (get-some-list)))
  (setcar (cdr --cl-x--)
             (1+ (cadr (get-some-list)))))

this calls "get-some-list" 2 times, which is apparently erroneous.  Again,
Common Lisp makes only 1 call.

4. `cl-macroexpand-all'. This should be blamed for 2 things.

First, it doesn't macroexpand all, in contrast to what its name states.  Try
out this:

(cl-macroexpand-all
 '(let (((car x) 10))
    (body-1)))

You get the following result:

(letf (((car x) 10)) (body-1)),

and this is not the result of the macroexpansion ! Yes, I agree that this
case is unlikely: `let' is not supposed to bind places, this feature was
supposed to be internal to CL implementation.. but wait, this function is
nevertheless cheating.  It doesn't do what it must; it reckons upon someone
else to finish the job.  In some advanced, sophisticated code this hidden
flaw may suddenly show itself.

Second, look at the following code:

((eq (car form) 'setq)
 (let* ((args (cl-macroexpand-body (cdr form) env)) (p args))
   (while (and p (symbolp (car p))) (setq p (cddr p)))
   (if p (cl-macroexpand-all (cons 'setf args)) (cons 'setq args))))

This is actually a cond clause from `cl-macroexpand-all'.  See, it looks
whether any of "setq" targets in not a simple symbol, and if it's not, we
end up with a "setf" form being processed instead of original "setq" form. 
We have already discussed that problem: full expansion of "setq" arguments. 
No intermediate setf-methods have a chance to come up and be considered.

5. Loop facility, if/when/unless clause.  This is the most violent point of
all above.

To put it short, in the loop clause that follows "if" we can access the
result of evaluation of the condition with the name "it".  But have a look
at how they have implemented that "it":

(setq form (list* 'if (list 'setq temp cond)
		       (subst temp 'it form)))

Just subst, and that's all.  There's no surprise that when I put my
anaphoric "awhen" construct (inspired by Paul Graham) inside the clause
following "when" in a loop, I got something uncompilable.


Conclusion
I can draw the only conclusion from all above: Elisp has not been used much
to create true Lispy programs.  Features like symbol macros and generalized
variables were introduced to be quite close to those respective concepts of
Common Lisp, but they are still not that mature.

I've managed to fix all above (except `pop', `push', `incf'.. -- they proved
hard to be changed and recompiled correctly), and I'm still using Emacs, and
I'm still programming in Elisp.  I'm sure Elisp does his job of being an
excellent training ground for learning Lisp concepts.
-- 
View this message in context: http://old.nabble.com/CL-package-serious-deficiencies-tp33271707p33271707.html
Sent from the Emacs - Dev mailing list archive at Nabble.com.

Re: CL package serious deficiencies

[Stefan Monnier]

> Hello to everyone ! In this message I would like to share my (mostly
> negative) experience with CL package of Elisp.  I'm sure, some Emacs/Elisp
> gurus are already familiar with many of these, but I still believe I have to
> say something new.  Let's start.

Elisp is not Common-Lisp.  CL does try to provide some CL-style
functionality, but indeed it has some rough edges in this regard.  It is
much better to look at it as a handy toolbox, whose design was inspired
by the experience of Common-Lisp, than to look at it as a "Common-Lisp
compatibility layer".

The points you raise sound like bugs indeed.  We welcome patches to fix
them, but I personally won't spend much time tracking those bugs down,
because my experience with this part of CL is that it's not always easy
to dig into it (it's *very* lightly commented for one, and parts of it
are fundamentally broken).


        Stefan

RE: CL package serious deficiencies

[Drew Adams]

> Elisp is not Common-Lisp.  CL does try to provide some CL-style
> functionality, but indeed it has some rough edges in this 
> regard.  It is much better to look at it as a handy toolbox,
> whose design was inspired by the experience of Common-Lisp,
> than to look at it as a "Common-Lisp compatibility layer".

I have to agree that that is the case.  And that is the right way to look at
things, in the current state.

> The points you raise sound like bugs indeed.  We welcome 
> patches to fix them, but I personally won't spend much time
> tracking those bugs down, because my experience with this
> part of CL is that it's not always easy to dig into it
> (it's *very* lightly commented for one, and parts of it
> are fundamentally broken).

Some comments.

1. I don't blame you or others for not wanting to spend much time on such
things.  Everyone who contributes is a volunteer.

2. There are lots of people who use cl.el.  There are even lots of people who
systematically write libraries that depend on cl.el functions at runtime (not
just macros at compile time).  Unfortunately.  My impression is that this might
be increasingly the case, and that perhaps younger or newer Emacs users are more
likely to do so.

3. I think that it would be good to make fixing more of such bugs and lacunae a
higher Emacs Dev priority.  Such development does not implement a sexy new
feature, but it is worthwhile, I think.  

4. And yes, it's non-trivial work to get things right.  If you, Stefan, find the
existing code too lightly commented and less than clear, then it should be
evident that others will too.  For progress to be made here there would, IMO,
need to be some commitment from people like you.  But maybe I'm wrong and all it
will take is some new, bright volunteer lurking on the list and waiting for an
opportunity to make a difference. ;-)

5. IMO it makes sense to proceed gradually, and to start by including more of
the simpler things that do work well, even if in a limited way, into regular
Emacs Lisp.

6. We've been through this (#5) before.  Candidate functions have been nominated
and discussed.  Some people have longer wishlists of functions than others.  But
little to nothing comes of it.  Dommage, IMO.

Just one opinion.  And I probably will not follow up with more discussion, which
is likely to go nowhere again.

Re: CL package serious deficiencies

[Stefan Monnier]

> 4. And yes, it's non-trivial work to get things right.  If you,
> Stefan, find the existing code too lightly commented and less than
> clear, then it should be evident that others will too.  For progress
> to be made here there would, IMO, need to be some commitment from
> people like you.  But maybe I'm wrong and all it will take is some
> new, bright volunteer lurking on the list and waiting for an
> opportunity to make a difference. ;-)

I'd start by declaring symbol-macrolet and defsubst* as deprecated
(they both suffer from serious bugs, the corresponding features aren't
very important and noone is willing/able to fix them).
Of course lexical-let will be marked obsolete soon as well.

Fixing the setf-method handling shouldn't be that hard (basically
adding macroexpand-1 and making it use it).


        Stefan

Re: CL package serious deficiencies

[egnarts-ms]

Stefan Monnier wrote:
> 
> I'd start by declaring symbol-macrolet and defsubst* as deprecated
> (they both suffer from serious bugs, the corresponding features aren't
> very important and no one is willing/able to fix them).
> Of course lexical-let will be marked obsolete soon as well.
> 

1. Please explain what are those bugs "defsubst*" is suffering from ? I'm
intrigued.
2. What the hell does making "lexical-let" obsolete mean ? I guess,
introducing true lexical scope, like in Common Lisp and Scheme ? IMHO,
"lexical-let" (barring any potential bugs) makes its job quite well for now.



> Fixing the setf-method handling shouldn't be that hard (basically
> adding macroexpand-1 and making it use it).
> 

I have actually added `macroexpand-1' as a C subroutine, and
`cl-macroexpand-1' function.  It seems to work well for 2 weeks now :)
-- 
View this message in context: http://old.nabble.com/CL-package-serious-deficiencies-tp33271707p33278115.html
Sent from the Emacs - Dev mailing list archive at Nabble.com.

Re: CL package serious deficiencies

[Stefan Monnier]

>> I'd start by declaring symbol-macrolet and defsubst* as deprecated
>> (they both suffer from serious bugs, the corresponding features aren't
>> very important and no one is willing/able to fix them).
>> Of course lexical-let will be marked obsolete soon as well.
> 1. Please explain what are those bugs "defsubst*" is suffering from ? I'm
> intrigued.

defubst* performs substitution in a completely naive way without paying
attention to Lisp syntax and scoping rules.

E.g. try (defsubst* sm-foo (a b) (cons a '(a))) and then
(byte-compile '(lambda () (sm-foo 2 3))) and you'll see that both
occurrences of the `a' symbol have been replaced by 2, even though the
second is not a reference to the variable `a'.

> 2. What the hell does making "lexical-let" obsolete mean ? I guess,
> introducing true lexical scope, like in Common Lisp and Scheme?

Yup, it's in Emacs-24.

>> Fixing the setf-method handling shouldn't be that hard (basically
>> adding macroexpand-1 and making it use it).
> I have actually added `macroexpand-1' as a C subroutine, and
> `cl-macroexpand-1' function.  It seems to work well for 2 weeks now :)

Care to submit a patch for review?


        Stefan

Re: CL package serious deficiencies

[egnarts-ms]

>>> Fixing the setf-method handling shouldn't be that hard (basically
>>> adding macroexpand-1 and making it use it).
>> I have actually added `macroexpand-1' as a C subroutine, and
>> `cl-macroexpand-1' function.  It seems to work well for 2 weeks now :)
> 
> Care to submit a patch for review?
> 

Surely I would be very glad to make a contribution to Emacs and Free
Software in general !
But I have never done so before.. Please tell me what to start with ? How
can I submit my patch ?

-- 
View this message in context: http://old.nabble.com/CL-package-serious-deficiencies-tp33271707p33285840.html
Sent from the Emacs - Dev mailing list archive at Nabble.com.

Re: CL package serious deficiencies

[Stefan Monnier]

> But I have never done so before.. Please tell me what to start with ?
> How can I submit my patch ?

Start by sending a patch here (preferably smallish, addressing one
particular bug),


        Stefan

RE: CL package serious deficiencies

[egnarts-ms]

Drew Adams wrote:
> 
> 2. There are lots of people who use cl.el.  There are even lots of people
> who
> systematically write libraries that depend on cl.el functions at runtime
> (not
> just macros at compile time).  Unfortunately.  My impression is that this
> might
> be increasingly the case, and that perhaps younger or newer Emacs users
> are more
> likely to do so.
> 

As I have pointed out in the original post, I have managed to fix all the
points described there.  (Except for `incf', `decf' etc. -- I just
introduced new, correct variants like `incf*', `decf*').  So I am eager to
share my experience.. although I realize that my experience with Emacs is
relatively poor.  And what is the most annoying thing, to my mind, is an
absence of understanding of interdependencies between different
parts/libs/files of Emacs during the build process.  I just don't know what
piece of functionality it is possible to use when implementing another piece
of functionality.



> 4. And yes, it's non-trivial work to get things right.  If you, Stefan,
> find the
> existing code too lightly commented and less than clear, then it should be
> evident that others will too.
> 

You know, in my opinion the code of CL package is quite understandable
(especially in comparison with commercial C++ outsource code I'm working
with) :)

And what kills me the most, is that I cannot manage to find a true Lispy
code.  Yes, I'm inexperienced lisper and maybe I just don't understand
something.  But what I saw in CL and some other places of Emacs sources
(lisp-mode.el for example) is not true Lispy code in the sense that it
doesn't attempt to make use of the main Lisp advantage -- ability to create
DSLs.

To be more specific, now that I read about anaphoric macros in "On Lisp" by
Paul Graham, I find it awkward not to use that functionality in complicated
code containing lots of conditioning.  Moreover, I created something that is
called "aif-op-1" -- a macro that works like this:

(aif-op-1 (eq (func-1)
                  'symbol)
    ;; here we have (func-1) result accessible with "it" name
    (func-2 arg1 it)
  ;;here we also have "it" bound to what previous (func-1) returned
  (func-3 it)
  ...)

Such things are not difficult to implement in Elisp.  But I haven't seen
anything like these neat language tricks in Emacs sources, especially in CL
package.  Instead, much code is just written manually rather than
automatically (by macros).



> 5. IMO it makes sense to proceed gradually, and to start by including more
> of
> the simpler things that do work well, even if in a limited way, into
> regular
> Emacs Lisp.
> 

This is a good idea.  We must try to make basic Lisp ideas be viable and
working, and have them evolve and develop in the modern world.  Elisp is a
good language for what it is used.  We must make it excellent.
-- 
View this message in context: http://old.nabble.com/CL-package-serious-deficiencies-tp33271707p33278223.html
Sent from the Emacs - Dev mailing list archive at Nabble.com.

Re: CL package serious deficiencies

[Nix]

On 6 Feb 2012, Drew Adams outgrape:
> 2. There are lots of people who use cl.el.  There are even lots of people who
> systematically write libraries that depend on cl.el functions at runtime (not
> just macros at compile time).  Unfortunately.  My impression is that this might
> be increasingly the case, and that perhaps younger or newer Emacs users are more
> likely to do so.

I've never understood what's wrong with including cl.el, nor why the
byte-compiler should warn specially about it, any more than it warns
about any other package. XEmacs has had cl.el in the dumped set for
absolutely ages and it has caused zero problems as far as I know.

So what's wrong with using it? It's there, it's always available, and if
you steer clear of the buggy bits it provides some useful facilities.
But Emacs packages persist in reimplementing them over and over in
package after package rather than just requiring 'cl and being done with
it.

-- 
NULL && (void)

Re: CL package serious deficiencies

[Lennart Borgman]

On Tue, Feb 7, 2012 at 19:43, Nix <nix@esperi.org.uk> wrote:
> On 6 Feb 2012, Drew Adams outgrape:
>> 2. There are lots of people who use cl.el.  There are even lots of people who
>> systematically write libraries that depend on cl.el functions at runtime (not
>> just macros at compile time).  Unfortunately.  My impression is that this might
>> be increasingly the case, and that perhaps younger or newer Emacs users are more
>> likely to do so.
>
> I've never understood what's wrong with including cl.el, nor why the
> byte-compiler should warn specially about it, any more than it warns
> about any other package. XEmacs has had cl.el in the dumped set for
> absolutely ages and it has caused zero problems as far as I know.
>
> So what's wrong with using it? It's there, it's always available, and if
> you steer clear of the buggy bits it provides some useful facilities.
> But Emacs packages persist in reimplementing them over and over in
> package after package rather than just requiring 'cl and being done with
> it.

There are things that do not work if you include cl.el at run time.
You may not notice the problems and tracking it down has been quite
difficult some times.

Re: CL package serious deficiencies

[Juanma Barranquero]

On Tue, Feb 7, 2012 at 20:11, Lennart Borgman <lennart.borgman@gmail.com> wrote:

> There are things that do not work if you include cl.el at run time.
> You may not notice the problems and tracking it down has been quite
> difficult some times.

I do (require 'cl) as one of the first things in my .emacs and I've
never seen a problem that was related to CL (I don't mean that CL has
no bugs, only that it hasn't triggered bugs in other code yet).

Do you have more specific info?

    Juanma

Re: CL package serious deficiencies

[Lennart Borgman]

On Tue, Feb 7, 2012 at 20:15, Juanma Barranquero <lekktu@gmail.com> wrote:
> On Tue, Feb 7, 2012 at 20:11, Lennart Borgman <lennart.borgman@gmail.com> wrote:
>
>> There are things that do not work if you include cl.el at run time.
>> You may not notice the problems and tracking it down has been quite
>> difficult some times.
>
> I do (require 'cl) as one of the first things in my .emacs and I've
> never seen a problem that was related to CL (I don't mean that CL has
> no bugs, only that it hasn't triggered bugs in other code yet).
>
> Do you have more specific info?

Yes. I wonder if I have not posted it here too? But it was long ago
and I do not have time to dig it up now. If I stumble on it I will
post it here.

Re: CL package serious deficiencies

[John Wiegley]

>>>>> Lennart Borgman <lennart.borgman@gmail.com> writes:

> There are things that do not work if you include cl.el at run time.  You may
> not notice the problems and tracking it down has been quite difficult some
> times.

Can we move those breaking definitions into another file, say cl-ext.el, and
then allow library authors to require 'cl at runtime?  There are a lot of
useful routines in cl.el which are not harmful to other code.

John

Re: CL package serious deficiencies

[Nix]

On 7 Feb 2012, Lennart Borgman verbalised:

> There are things that do not work if you include cl.el at run time.
> You may not notice the problems and tracking it down has been quite
> difficult some times.

If so, these are either bugs in cl.el, or bugs in the conflicting
packages, and should surely be fixed, rather than simply trying to avoid
using cl.

-- 
NULL && (void)

Re: CL package serious deficiencies

[Lennart Borgman]

On Tue, Feb 7, 2012 at 22:06, Nix <nix@esperi.org.uk> wrote:
> On 7 Feb 2012, Lennart Borgman verbalised:
>
>> There are things that do not work if you include cl.el at run time.
>> You may not notice the problems and tracking it down has been quite
>> difficult some times.
>
> If so, these are either bugs in cl.el, or bugs in the conflicting
> packages, and should surely be fixed, rather than simply trying to avoid
> using cl.

Fixing the bugs would be better yes. Anyone interested can do that.

But no one has said that cl.el should not be used. I have not seen any
problems when the instructions for its use are followed.

Re: CL package serious deficiencies

[Nix]

On 7 Feb 2012, Lennart Borgman told this:

> On Tue, Feb 7, 2012 at 22:06, Nix <nix@esperi.org.uk> wrote:
>> On 7 Feb 2012, Lennart Borgman verbalised:
>>
>>> There are things that do not work if you include cl.el at run time.
>>> You may not notice the problems and tracking it down has been quite
>>> difficult some times.
>>
>> If so, these are either bugs in cl.el, or bugs in the conflicting
>> packages, and should surely be fixed, rather than simply trying to avoid
>> using cl.
>
> Fixing the bugs would be better yes. Anyone interested can do that.

Well, if you find out what the problems are, I'll see about fixing them.

> But no one has said that cl.el should not be used. I have not seen any
> problems when the instructions for its use are followed.

Since those instructions amount to 'don't use it except at compile-
time', this is not incredibly surprising.

-- 
NULL && (void)

Re: CL package serious deficiencies

[Lennart Borgman]

On Tue, Feb 7, 2012 at 22:10, Nix <nix@esperi.org.uk> wrote:
> On 7 Feb 2012, Lennart Borgman told this:
>
>> On Tue, Feb 7, 2012 at 22:06, Nix <nix@esperi.org.uk> wrote:
>>> On 7 Feb 2012, Lennart Borgman verbalised:
>>>
>>>> There are things that do not work if you include cl.el at run time.
>>>> You may not notice the problems and tracking it down has been quite
>>>> difficult some times.
>>>
>>> If so, these are either bugs in cl.el, or bugs in the conflicting
>>> packages, and should surely be fixed, rather than simply trying to avoid
>>> using cl.
>>
>> Fixing the bugs would be better yes. Anyone interested can do that.
>
> Well, if you find out what the problems are, I'll see about fixing them.

I am sorry, but I can't help with that at the moment.

>> But no one has said that cl.el should not be used. I have not seen any
>> problems when the instructions for its use are followed.
>
> Since those instructions amount to 'don't use it except at compile-
> time', this is not incredibly surprising.

I think you are misundering.. ;-) -- or am I misunderstanding you...

The code that uses cl.el must be compiled. Is that a very sever limitation?

Re: CL package serious deficiencies

[Nix]

On 7 Feb 2012, Lennart Borgman spake thusly:

> On Tue, Feb 7, 2012 at 22:10, Nix <nix@esperi.org.uk> wrote:
>>> But no one has said that cl.el should not be used. I have not seen any
>>> problems when the instructions for its use are followed.
>>
>> Since those instructions amount to 'don't use it except at compile-
>> time', this is not incredibly surprising.
>
> I think you are misundering.. ;-) -- or am I misunderstanding you...
>
> The code that uses cl.el must be compiled. Is that a very sever limitation?

Er, yeah, it's that you can't do

(require 'cl)

without getting moaned at by the byte-compiler and rejected from
inclusion into Emacs, a limitation true of no other elisp package in
existence (not even apel, which really *did* break things back in the
day, perhaps because it was last maintained in 1996).

Yeah, you can do

(eval-when-compile
   (require 'cl))

or

(require 'cl-macs)

but those do not do the same thing.

So as a result, if one of us wants to use `cadr' in ordinary Lisp
functions that are part of Emacs, we can, but if we want to use `caddr',
we can't. Now does that restriction make any sense at all? I'd say it
doesn't: if there are bugs preventing the whole byte-compile-cl-warn
mess being dropped from bytecomp.el, then those bugs should be fixed.

(Obviously, `caddr' is hardly essential -- I used it only for rhetorical
purposes, becuase it is quite clear that it is no more reasonable to ban
its use at runtime than it is to ban use of 'car' -- but some things in
cl and especially cl-extra really are useful and pointless to
reimplement: random* for example, or the extra mapping functions.)

(FWIW, the *last* time I asked about this, many years ago, I was told
that runtime use of cl was out of the question because it used too much
memory. I presume that this argument is obsolete :) )

-- 
NULL && (void)

Re: CL package serious deficiencies

[Lennart Borgman]

On Tue, Feb 7, 2012 at 22:23, Nix <nix@esperi.org.uk> wrote:
> On 7 Feb 2012, Lennart Borgman spake thusly:
>
>
> So as a result, if one of us wants to use `cadr' in ordinary Lisp
> functions that are part of Emacs, we can, but if we want to use `caddr',
> we can't. Now does that restriction make any sense at all? I'd say it

I am using caddr quite a lot. Without any whining byte-compiler or
other problems... ;-)

Re: CL package serious deficiencies

[Nix]

On 7 Feb 2012, Lennart Borgman stated:

> On Tue, Feb 7, 2012 at 22:23, Nix <nix@esperi.org.uk> wrote:
>> On 7 Feb 2012, Lennart Borgman spake thusly:
>>
>>
>> So as a result, if one of us wants to use `cadr' in ordinary Lisp
>> functions that are part of Emacs, we can, but if we want to use `caddr',
>> we can't. Now does that restriction make any sense at all? I'd say it
>
> I am using caddr quite a lot. Without any whining byte-compiler or
> other problems... ;-)

Ah, there's a macro implementation I overlooked 'cos I have cl loaded at
runtime :)

-- 
NULL && (void)

Re: CL package serious deficiencies

[Glenn Morris]

Nix wrote:

> functions that are part of Emacs, we can, but if we want to use `caddr',
> we can't.

Oh yes you can, since it has a compiler macro. There's not even a
byte-compiler warning about it.

Re: CL package serious deficiencies

[Daniel Colascione]

[-- Attachment #1: Type: text/plain, Size: 717 bytes --]

On 2/7/12 1:23 PM, Nix wrote:
> (FWIW, the *last* time I asked about this, many years ago, I was told
> that runtime use of cl was out of the question because it used too much
> memory. I presume that this argument is obsolete :) )

I've long been an advocate of dumping cl with Emacs; XEmacs does so
without problems. CLisms simply result in cleaner, smaller code than
one can write using nothing but elisp primitives.

The latest objection is that dumping CL would Emacs would increase the
size of the printed and bound Emacs Lisp Reference Manual from the
FSF. I find this objection unconvincing because it applies to _any_
new feature, and I don't think we're against adding new features to Emacs.


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 235 bytes --]

Re: CL package serious deficiencies

[Tom Tromey]

>>>>> "Daniel" == Daniel Colascione <dancol@dancol.org> writes:

Daniel> The latest objection is that dumping CL would Emacs would
Daniel> increase the size of the printed and bound Emacs Lisp Reference
Daniel> Manual from the FSF. I find this objection unconvincing because
Daniel> it applies to _any_ new feature, and I don't think we're against
Daniel> adding new features to Emacs.

I think the existence of EIEIO makes the documentation objection
especially moot.  EIEIO is basically CLOS and has a manual of its own,
but AFAICT doesn't have the same use restrictions as cl.el.  I have
never understood this discrepancy.

Tom

Re: CL package serious deficiencies

[Leo]

On 2012-02-08 05:42 +0800, Tom Tromey wrote:
> I think the existence of EIEIO makes the documentation objection
> especially moot.  EIEIO is basically CLOS and has a manual of its own,
> but AFAICT doesn't have the same use restrictions as cl.el.  I have
> never understood this discrepancy.
>
> Tom

I agree.

I think it is a mistake to object cl.el and let every package author
re-invent their own half-baked solution.

If cl.el gets more use, I am sure its quality will improve. The status
quo is partly due to the objection.

BTW, I have (require 'cl) as the first thing in .emacs since a few years
ago and never had a problem.

Leo

Re: CL package serious deficiencies

[Alan Mackenzie]

Hi, Daniel.

On Tue, Feb 07, 2012 at 01:34:06PM -0800, Daniel Colascione wrote:
> On 2/7/12 1:23 PM, Nix wrote:
> > (FWIW, the *last* time I asked about this, many years ago, I was told
> > that runtime use of cl was out of the question because it used too much
> > memory. I presume that this argument is obsolete :) )

> I've long been an advocate of dumping cl with Emacs; XEmacs does so
> without problems.

The CC Mode test suite (in particular, 000tests.el, available from the CC
Mode site at SourceForge), uses cl at run time.  It's been getting wierd
"can't happen" errors for years:  some error starts appearing,
repeatedly, then after some unrelated change in CC Mode, stops appearing.
Then another wierd error starts happening, then stops.  This has been
going on for years.  The current error, which has been happening for ~6
months, is:

    Testing awk-face-1.awk (fonts)  Buffer is read-only: #<buffer *cc-test*>

.  This doesn't happen with XEmacs (though other errors do).

I believe that the cl files are somehow responsible; Barry Warsaw (who
originally wrote 000tests.el) was and is a competent hacker.

> CLisms simply result in cleaner, smaller code than one can write using
> nothing but elisp primitives.

> The latest objection is that dumping CL would Emacs would increase the
> size of the printed and bound Emacs Lisp Reference Manual from the
> FSF. I find this objection unconvincing because it applies to _any_ new
> feature, and I don't think we're against adding new features to Emacs.

I'd recommend not changing the policy on CL until it acquires a rigorous
test suite.  I don't really trust it beyond use at byte-compile time.

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: CL package serious deficiencies

[Nix]

On 7 Feb 2012, Alan Mackenzie said:
>     Testing awk-face-1.awk (fonts)  Buffer is read-only: #<buffer *cc-test*>
>
> .  This doesn't happen with XEmacs (though other errors do).

Thank you! Something for me to look at this weekend :) I may get
nowhere, we'll see...

>> The latest objection is that dumping CL would Emacs would increase the
>> size of the printed and bound Emacs Lisp Reference Manual from the
>> FSF. I find this objection unconvincing because it applies to _any_ new
>> feature, and I don't think we're against adding new features to Emacs.
>
> I'd recommend not changing the policy on CL until it acquires a rigorous
> test suite.  I don't really trust it beyond use at byte-compile time.

To be honest, if the errors are that weird they may not be in cl at all
but in the Lisp evaluator (it is unarguable that cl makes use of some
elisp features to a degree that few other packages do). If the bug is in
there, would it still suggest that a cl testsuite was necessary?

-- 
NULL && (void)

Re: CL package serious deficiencies

[Richard Stallman]

    The latest objection is that dumping CL would Emacs would increase the
    size of the printed and bound Emacs Lisp Reference Manual from the
    FSF. I find this objection unconvincing because it applies to _any_
    new feature, and I don't think we're against adding new features to Emacs.

The point is that it would make the CL functions part of the standard
Emacs namespace.  That has two problems:

* CL is ugly.  It is not well integrated with the rest of Emacs Lisp.

* We would have to document the CL functions in the manual, which is a
big increase in the size.  It is not that that is totally intolerable,
it's that the benefit is not worth the burden.


-- 
Dr Richard Stallman
President, Free Software Foundation
51 Franklin St
Boston MA 02110
USA
www.fsf.org  www.gnu.org
Skype: No way! That's nonfree (freedom-denying) software.
  Use free telephony http://directory.fsf.org/category/tel/

Re: CL package serious deficiencies

[Tom Tromey]

>>>>> "RMS" == Richard Stallman <rms@gnu.org> writes:

RMS> * CL is ugly.  It is not well integrated with the rest of Emacs Lisp.

The second part seems to me to be a circular argument.  It could be
better integrated; but it will not be as long as it is not acceptable.

RMS> * We would have to document the CL functions in the manual, which is a
RMS> big increase in the size.  It is not that that is totally intolerable,
RMS> it's that the benefit is not worth the burden.

Why does this consideration not apply to EIEIO?

Tom

Re: CL package serious deficiencies

[Richard Stallman]

    RMS> * CL is ugly.  It is not well integrated with the rest of Emacs Lisp.

    The second part seems to me to be a circular argument.  It could be
    better integrated; but it will not be as long as it is not acceptable.

Maybe we are miscommunicating.  The ugliness I am talking about is in
the specs of these functions, not the implementation.  The specs of
the CL functions don't fit in well with Emacs Lisp.  This isn't
something that could be fixed with more work on the code.

-- 
Dr Richard Stallman
President, Free Software Foundation
51 Franklin St
Boston MA 02110
USA
www.fsf.org  www.gnu.org
Skype: No way! That's nonfree (freedom-denying) software.
  Use free telephony http://directory.fsf.org/category/tel/

Re: CL package serious deficiencies

[Eric Schulte]

Richard Stallman <rms@gnu.org> writes:

>     RMS> * CL is ugly.  It is not well integrated with the rest of Emacs Lisp.
>
>     The second part seems to me to be a circular argument.  It could be
>     better integrated; but it will not be as long as it is not acceptable.
>
> Maybe we are miscommunicating.  The ugliness I am talking about is in
> the specs of these functions, not the implementation.  The specs of
> the CL functions don't fit in well with Emacs Lisp.  This isn't
> something that could be fixed with more work on the code.

Ugliness is a more normative than descriptive word.

I assume you are referring the use of keyword arguments in the CL
functions, as (to my knowledge) keyword arguments were consciously
excluded from elisp and are not used elsewhere.

Assuming the above is correct and there are to be no keyword arguments
in core elisp functions, then Emacs is in an undesirable position in
which core functionality (e.g., `reduce' and `remove-if') is not
provided by default, and the common names for such functionality is
reserved for functions which will never be included in the core, so it
doesn't seem likely that acceptable versions of such functions could
ever be included into elisp.

-- 
Eric Schulte
http://cs.unm.edu/~eschulte/

Re: CL package serious deficiencies

[Richard Stallman]

    I assume you are referring the use of keyword arguments in the CL
    functions, as (to my knowledge) keyword arguments were consciously
    excluded from elisp and are not used elsewhere.

Yes.  I excluded them from list-manipulation functions because they
are clunky.

There are some Emacs Lisp functions that have keyword arguments.
These functions do specialized things which are complex; if they are
clunky, that is acceptable.

-- 
Dr Richard Stallman
President, Free Software Foundation
51 Franklin St
Boston MA 02110
USA
www.fsf.org  www.gnu.org
Skype: No way! That's nonfree (freedom-denying) software.
  Use free telephony http://directory.fsf.org/category/tel/

Re: CL package serious deficiencies

[Tom Tromey]

>>>>> "RMS" == Richard Stallman <rms@gnu.org> writes:

RMS> There are some Emacs Lisp functions that have keyword arguments.
RMS> These functions do specialized things which are complex; if they are
RMS> clunky, that is acceptable.

Thanks; this distinction makes sense to me.

Tom

Re: CL package serious deficiencies

[Tom Tromey]

>>>>> "Eric" == Eric Schulte <eric.schulte@gmx.com> writes:

Eric> I assume you are referring the use of keyword arguments in the CL
Eric> functions, as (to my knowledge) keyword arguments were consciously
Eric> excluded from elisp and are not used elsewhere.

Elisp sort of has keyword arguments.  They just aren't part of the
language, so they are reimplemented by hand in various functions.
You can find many examples in Emacs, including even functions in C,
e.g., dbus-call-method or make-network-process.

Tom

RE: CL package serious deficiencies

[Drew Adams]

> Elisp sort of has keyword arguments.  They just aren't part of the
> language, so they are reimplemented by hand in various functions.
> You can find many examples in Emacs, including even functions in C,
> e.g., dbus-call-method or make-network-process.

Right.

It might also be worth distinguishing (a) the relative lack/dearth of
function-valued arguments that have the effect of certain important keyword
args, such as :test and :key from (b) the lack of support for keyword arguments
per se.

I like CL keyword arguments personally, but I think that (a) is more important
than (b), if there is pushback for such stuff.

Example: It is far more important to have a sequence function that accepts a
function-valued TEST argument than it is to be able to pass it such an argument
using the keyword :test.  Being able to pass args in any order by specifying
them using keywords (e.g. :test) is, in my book, less important.

Yes, keywords can be convenient and sometimes less verbose (yes), but what's
really missing IMO are the Common-Lisp sequence etc. functions that accept such
function-valued args, and not the fact that you can pass those args using
keywords.

I could be wrong, but my guess is that Richard's resistance to keyword args is
mainly about (b), not (a), that is, the use of keywords per se, and not general
sequence etc. functions that accept function-valued args.

Emacs Lisp does have some functions that accept functions as args, of course,
but the language has generally not been optimized to support passing functional
args, so the use of such higher-order functions has been somewhat limited in
practice.

Support for lexical binding changes that game, opening more possibilities.  But
additional work would be needed to really make such Common-Lisp thingies
performant in Emacs Lisp.

Re: CL package serious deficiencies

[Ted Zlatanov]

On Fri, 10 Feb 2012 08:52:31 -0700 Tom Tromey <tromey@redhat.com> wrote: 

Tom> Elisp sort of has keyword arguments.  They just aren't part of the
Tom> language, so they are reimplemented by hand in various functions.
Tom> You can find many examples in Emacs, including even functions in C,
Tom> e.g., dbus-call-method or make-network-process.

Don't forget Customize, which would be unmanageable without keyword
arguments.  That's probably the one keyword-based API every ELisp writer
has experienced.

Ted

Re: CL package serious deficiencies

[Richard Stallman]

    RMS> * We would have to document the CL functions in the manual, which is a
    RMS> big increase in the size.  It is not that that is totally intolerable,
    RMS> it's that the benefit is not worth the burden.

    Why does this consideration not apply to EIEIO?

Why would it apply to EIEIO?
I don't think I ever saw a program that used EIEIO.
If it is not likely to be used by most programs
then it would not be crucial to describe it in the Lisp manual..

-- 
Dr Richard Stallman
President, Free Software Foundation
51 Franklin St
Boston MA 02110
USA
www.fsf.org  www.gnu.org
Skype: No way! That's nonfree (freedom-denying) software.
  Use free telephony http://directory.fsf.org/category/tel/

Re: CL package serious deficiencies

[Ted Zlatanov]

On Thu, 09 Feb 2012 01:30:00 -0500 Richard Stallman <rms@gnu.org> wrote: 

RS> I don't think I ever saw a program that used EIEIO.
RS> If it is not likely to be used by most programs
RS> then it would not be crucial to describe it in the Lisp manual..

I use it in the Gnus registry.el package.  I like it and think it's a
good base for building data containers, more so than CL's `defstruct'.
It would be more widely used if it were in the manual, IMO.

Ted

Re: CL package serious deficiencies

[Richard Stallman]

Is eieio one self-contained feature?  If so, that is another
difference between eieio and CL.

-- 
Dr Richard Stallman
President, Free Software Foundation
51 Franklin St
Boston MA 02110
USA
www.fsf.org  www.gnu.org
Skype: No way! That's nonfree (freedom-denying) software.
  Use free telephony http://directory.fsf.org/category/tel/

Re: CL package serious deficiencies

[Tom Tromey]

>>>>> "RMS" == Richard Stallman <rms@gnu.org> writes:

RMS> Is eieio one self-contained feature?  If so, that is another
RMS> difference between eieio and CL.

I think it depends on how you look at it.

EIEIO is a subset of CLOS.  So in one sense it is a single feature.

When I read (info "(eieio) Introduction"), though, I see that it
provides many things.

Tom

Re: CL package serious deficiencies

[Tom Tromey]

>>>>> "RMS" == Richard Stallman <rms@gnu.org> writes:

RMS> * We would have to document the CL functions in the manual, which is a
RMS> big increase in the size.  It is not that that is totally intolerable,
RMS> it's that the benefit is not worth the burden.

>>     Why does this consideration not apply to EIEIO?

RMS> Why would it apply to EIEIO?

Because to me EIEIO seems to fit the same criteria as cl.el: it is
pretty big, it is derived from Common Lisp, and it uses Common Lisp
names (so intrudes on the "user namespace").

RMS> I don't think I ever saw a program that used EIEIO.

CEDET uses it.  This was a major feature addition to Emacs.  Presumably
many people use it.

But this discussion is also about what elisp writers may do, not what
they do already.

Tom

Re: CL package serious deficiencies

[Stefan Monnier]

RMS> Why would it apply to EIEIO?
> Because to me EIEIO seems to fit the same criteria as cl.el: it is
> pretty big, it is derived from Common Lisp, and it uses Common Lisp
> names (so intrudes on the "user namespace").

The namespace use is indeed problematic.  It needs a serious cleanup.
The (defvar this nil) is *really* nasty, since the unsuspecting user may
want to do

  (let ((this ...) (that ...))
    ...
    (lambda (x) (...this...that...)))

and be surprised to discover that his closure did not close over `this'
because it's a special (i.e. dynamically bound) variable.


        Stefan

Re: CL package serious deficiencies

[Stephen J. Turnbull]

Nix writes:

 > (FWIW, the *last* time I asked about this, many years ago, I was told
 > that runtime use of cl was out of the question because it used too much
 > memory. I presume that this argument is obsolete :) )

The feeling I get is that some senior Emacs developers simply rather
dislike Common Lisp, think that the programming style it encourages
sucks, and don't want to see it in code they work on frequently.

Ah, the sweet smell of "technical differences" in the design fog....

Re: CL package serious deficiencies

[Glenn Morris]

Nix wrote:

> Well, if you find out what the problems are, I'll see about fixing them.

You could look at

http://debbugs.gnu.org/cgi/bugreport.cgi?bug=5644
http://debbugs.gnu.org/cgi/bugreport.cgi?bug=6583
http://debbugs.gnu.org/cgi/bugreport.cgi?bug=7070
http://debbugs.gnu.org/cgi/bugreport.cgi?bug=7604
http://debbugs.gnu.org/cgi/bugreport.cgi?bug=8841
http://debbugs.gnu.org/cgi/bugreport.cgi?bug=9103

if you want something to do (send any follow-ups to the relevant bug
address, not here). Though it won't hasten Emacs 24.1 at all, so is not
the most useful thing to be doing right now. Also none of these have any
relevance to whether cl is acceptable to use at run-time.

Re: CL package serious deficiencies

[Nix]

On 7 Feb 2012, Glenn Morris uttered the following:

> Nix wrote:
>
>> Well, if you find out what the problems are, I'll see about fixing them.
>
> You could look at
[...]
> if you want something to do (send any follow-ups to the relevant bug
> address, not here).

Thanks! on the list...

>                                              Also none of these have any
> relevance to whether cl is acceptable to use at run-time.

So... unreliability is *not* the reason, after all? And documentation-
weightiness, as Tom points out, cannot be the reason. So... what *is*
the reason? I (and others, it seems) are distinctly curious.

-- 
NULL && (void)

Re: CL package serious deficiencies

[Eric Schulte]

Nix <nix@esperi.org.uk> writes:

> On 7 Feb 2012, Glenn Morris uttered the following:
>
>> Nix wrote:
>>
>>> Well, if you find out what the problems are, I'll see about fixing them.
>>
>> You could look at
> [...]
>> if you want something to do (send any follow-ups to the relevant bug
>> address, not here).
>
> Thanks! on the list...
>
>>                                              Also none of these have any
>> relevance to whether cl is acceptable to use at run-time.
n>
> So... unreliability is *not* the reason, after all? And documentation-
> weightiness, as Tom points out, cannot be the reason. So... what *is*
> the reason? I (and others, it seems) are distinctly curious.

I was under the impression that the use of keyword arguments in cl.el
was the reason that it was unpalatable to core Emacs devs.  That and a
general desire for elisp to be a simpler language than cl.

Given that cl inclusion suddenly seems possible, I would like to say
that from the perspective of working on Org-mode the availability of cl
functions would be a great help.  We already have at least the following
Org-specific re-writes of existing cl-functions, and we frequently have
to cull other cl functions from the code base.

org-count
org-find-if
org-reduce
org-remove-if
org-remove-if-not

M-x apropos "remove-if" yields the following 9 functions [1] all of
which are currently distributed as part of Emacs.  At this point I think
there is an argument for cl inclusion simplifying Emacs, or at least
making life easier for developers of Emacs packages.

Cheers,

Footnotes: 
[1]  
ert--remove-if-not
  Function: A reimplementation of `remove-if-not'.
gnus-remove-if
  Function: Return a copy of SEQUENCE with all items satisfying
            PREDICATE removed.
gnus-remove-if-not
  Function: Return a copy of SEQUENCE with all items not satisfying
            PREDICATE removed.
org-remove-if
  Function: Remove everything from SEQ that fulfills PREDICATE.
org-remove-if-not
  Function: Remove everything from SEQ that does not fulfill
            PREDICATE.
recentf-remove-if-non-kept
  Function: Remove FILENAME from the recent list, if file is not kept.
  Properties: byte-optimizer
remove-if
  Function: Remove all items satisfying PREDICATE in SEQ.
  Properties: autoload
remove-if-not
  Function: Remove all items not satisfying PREDICATE in SEQ.
  Properties: autoload
widget-remove-if
  Function: (not documented)


-- 
Eric Schulte
http://cs.unm.edu/~eschulte/

Re: CL package serious deficiencies

[Lennart Borgman]

On Wed, Feb 8, 2012 at 14:42, Eric Schulte <eric.schulte@gmx.com> wrote:
>
> Given that cl inclusion suddenly seems possible, I would like to say
> that from the perspective of working on Org-mode the availability of cl
> functions would be a great help.  We already have at least the following
> Org-specific re-writes of existing cl-functions, and we frequently have
> to cull other cl functions from the code base.
>
> org-count
> org-find-if
> org-reduce
> org-remove-if
> org-remove-if-not

I must be misremembering. Isn't it ok to use the cl functions here?
(You would be using it in compiled code.)

Re: CL package serious deficiencies

[Eric Schulte]

Lennart Borgman <lennart.borgman@gmail.com> writes:

> On Wed, Feb 8, 2012 at 14:42, Eric Schulte <eric.schulte@gmx.com> wrote:
>>
>> Given that cl inclusion suddenly seems possible, I would like to say
>> that from the perspective of working on Org-mode the availability of cl
>> functions would be a great help.  We already have at least the following
>> Org-specific re-writes of existing cl-functions, and we frequently have
>> to cull other cl functions from the code base.
>>
>> org-count
>> org-find-if
>> org-reduce
>> org-remove-if
>> org-remove-if-not
>
> I must be misremembering. Isn't it ok to use the cl functions here?
> (You would be using it in compiled code.)
>

You are misremembering.  If it were ok org, gnus, ert and widget would
not all have their own remove-if variants?

-- 
Eric Schulte
http://cs.unm.edu/~eschulte/

Re: CL package serious deficiencies

[Lennart Borgman]

On Wed, Feb 8, 2012 at 14:51, Eric Schulte <eric.schulte@gmx.com> wrote:
>>>
>>> org-count
>>> org-find-if
>>> org-reduce
>>> org-remove-if
>>> org-remove-if-not
>>
>> I must be misremembering. Isn't it ok to use the cl functions here?
>> (You would be using it in compiled code.)
>>
>
> You are misremembering.  If it were ok org, gnus, ert and widget would
> not all have their own remove-if variants?

Thanks. Or, I might be misunderstanding ;-)
The manual says in D.1 Emacs Lisp Coding Conventions that

    However, there is no problem with using the `cl' package at
     compile time, with `(eval-when-compile (require 'cl))'.  That's
     sufficient for using the macros in the `cl' package, because the
     compiler expands them before generating the byte-code.

Is that not enough, or is it not applicable to code included in Emacs?

Re: CL package serious deficiencies

[Eric Schulte]

Lennart Borgman <lennart.borgman@gmail.com> writes:

> On Wed, Feb 8, 2012 at 14:51, Eric Schulte <eric.schulte@gmx.com> wrote:
>>>>
>>>> org-count
>>>> org-find-if
>>>> org-reduce
>>>> org-remove-if
>>>> org-remove-if-not
>>>
>>> I must be misremembering. Isn't it ok to use the cl functions here?
>>> (You would be using it in compiled code.)
>>>
>>
>> You are misremembering.  If it were ok org, gnus, ert and widget would
>> not all have their own remove-if variants?
>
> Thanks. Or, I might be misunderstanding ;-)
> The manual says in D.1 Emacs Lisp Coding Conventions that
>
>     However, there is no problem with using the `cl' package at
>      compile time, with `(eval-when-compile (require 'cl))'.  That's
>      sufficient for using the macros in the `cl' package, because the
>      compiler expands them before generating the byte-code.
>
> Is that not enough, or is it not applicable to code included in Emacs?
>

The above only applies to *macros* which are expanded at compile time.
The expanded macros leave no reference to cl in the resulting compiled
code.  On the contrary cl functions (even in compiled code) when called
at runtime still require cl to be loaded and available at runtime.

Hope this makes it clear,

-- 
Eric Schulte
http://cs.unm.edu/~eschulte/

Re: CL package serious deficiencies

[Lennart Borgman]

On Wed, Feb 8, 2012 at 16:39, Eric Schulte <eric.schulte@gmx.com> wrote:
> Lennart Borgman <lennart.borgman@gmail.com> writes:
>
>> On Wed, Feb 8, 2012 at 14:51, Eric Schulte <eric.schulte@gmx.com> wrote:
>>>>>
>>>>> org-count
>>>>> org-find-if
>>>>> org-reduce
>>>>> org-remove-if
>>>>> org-remove-if-not
>>>>
>>>> I must be misremembering. Isn't it ok to use the cl functions here?
>>>> (You would be using it in compiled code.)
>>>>
>>>
>>> You are misremembering.  If it were ok org, gnus, ert and widget would
>>> not all have their own remove-if variants?
>>
>> Thanks. Or, I might be misunderstanding ;-)
>> The manual says in D.1 Emacs Lisp Coding Conventions that
>>
>>     However, there is no problem with using the `cl' package at
>>      compile time, with `(eval-when-compile (require 'cl))'.  That's
>>      sufficient for using the macros in the `cl' package, because the
>>      compiler expands them before generating the byte-code.
>>
>> Is that not enough, or is it not applicable to code included in Emacs?
>>
>
> The above only applies to *macros* which are expanded at compile time.
> The expanded macros leave no reference to cl in the resulting compiled
> code.  On the contrary cl functions (even in compiled code) when called
> at runtime still require cl to be loaded and available at runtime.
>
> Hope this makes it clear,

Yes, I have not looked much at cl.el and my impression was that only
the macros were important. So I was mistaken there.

RE: CL package serious deficiencies

[Drew Adams]

> We already have at least the following Org-specific re-writes
> of existing cl-functions, and we frequently have
> to cull other cl functions from the code base.
> 
> org-count
> org-find-if
> org-reduce
> org-remove-if
> org-remove-if-not

Likewise, for my code (e.g. Icicles, Bookmark+) - same usual suspects: simple
sequence functions.  Not a biggee, but a clear hint, no?

> M-x apropos "remove-if" yields the following 9 functions all of
> which are currently distributed as part of Emacs. 
>
> ert--remove-if-not
> gnus-remove-if
> gnus-remove-if-not
> org-remove-if
> org-remove-if-not
> recentf-remove-if-non-kept
> remove-if
> remove-if-not
> widget-remove-if

This is precisely what I meant by:

>> 5. IMO it makes sense to proceed gradually, and to start by
>>    including more of the simpler things that do work well,
>>    even if in a limited way, into regular Emacs Lisp.

Such sequence functions (or similar) are one obvious place to start, IMHO.  And
preferably _with_ support for at least some keyword arguments (or else
equivalent, non-keyword args) - in particular, :test and :key.

But hey, we still do not have a fast version of `remove-duplicates' -
http://debbugs.gnu.org/cgi/bugreport.cgi?bug=10386#11,
http://article.gmane.org/gmane.emacs.devel/139546/match=remove+dups.

And that's perhaps an example of one of the problems with cl.el: the cl.el
version of `remove-duplicates' is actually much _slower_ than the typical naive
Lisp definition.

Unfortunately:

>> 6. We've been through this (#5) before.  Candidate functions
>>    have been nominated and discussed.  Some people have longer
>>    wishlists of functions than others.  But little to nothing
>>    comes of it.  Dommage, IMO.

Re: CL package serious deficiencies

[Teemu Likonen]

* Lennart Borgman [2012-02-07 20:11:11 +0100] wrote:

> There are things that do not work if you include cl.el at run time.
> You may not notice the problems and tracking it down has been quite
> difficult some times.

About two weeks ago I noticed the opposite. I switched from Emacs 23.2
to 24.0 and experienced that some spam- or registry-related things in
Gnus didn't work _without_ run-time cl package. I support the idea of
making Emacs Lisp closer to CL so I don't care to complain about it.

spam- or registry-related things in Gnus need cl at run-time? (was: CL package serious deficiencies)

[Reiner Steib]

On Wed, Feb 08 2012, Teemu Likonen wrote:

> * Lennart Borgman [2012-02-07 20:11:11 +0100] wrote:
>
>> There are things that do not work if you include cl.el at run time.
>> You may not notice the problems and tracking it down has been quite
>> difficult some times.
>
> About two weeks ago I noticed the opposite. I switched from Emacs 23.2
> to 24.0 and experienced that some spam- or registry-related things in
> Gnus didn't work _without_ run-time cl package.

Please file a bug report (for Gnus) about this.

Bye, Reiner.
-- 
       ,,,
      (o o)
---ooO-(_)-Ooo---  |  PGP key available  |  http://rsteib.home.pages.de/

Re: spam- or registry-related things in Gnus need cl at run-time?

[Teemu Likonen]

* Reiner Steib [2012-02-09 08:33:19 +0100] wrote:

> On Wed, Feb 08 2012, Teemu Likonen wrote:
>> About two weeks ago I noticed the opposite. I switched from Emacs
>> 23.2 to 24.0 and experienced that some spam- or registry-related
>> things in Gnus didn't work _without_ run-time cl package.
>
> Please file a bug report (for Gnus) about this.

For me the "bug" is already fixed: (require 'cl). I'm sorry but I'm not
motivated to find out how to reproduce it from "emacs -Q" situation and
write a detailed report. I'll give this general info from my memory,
though:

  - I have Gnus spam package and registry in use.

  - Some nntp groups are configured to be automatically checked for spam
    (bogofilter) when I enter to the group.

  - I enter to such group and then exit it. All seems fine.

  - In the *Group* buffer trying to save with "s" key
    (gnus-group-save-newsrc) causes some error message that definition
    of something is void.

  - (require 'cl) makes if work.

  - This thing happened the first time after I had switched from Emacs
    23.2 (Debian package) to 24.0 from <git://repo.or.cz/emacs.git>
    (branch "master").

Re: CL package serious deficiencies

[Stefan Monnier]

> I've never understood what's wrong with including cl.el, nor why the

The main issue is namespace.  If someone goes through the code to rename
it all to "cl-*", then we won't need to avoid using it.


        Stefan

Re: CL package serious deficiencies

[Nix]

On 7 Feb 2012, Stefan Monnier said:

>> I've never understood what's wrong with including cl.el, nor why the
>
> The main issue is namespace.  If someone goes through the code to rename
> it all to "cl-*", then we won't need to avoid using it.

Aha. I'd agree with *that*: it's always been hellishly unclear which
things are in cl or not. Its intention to sort of add to the Emacs Lisp
core with things with very similar names was perhaps praiseworthy, but
once those things started to move *into* the non-cl.el core, often with
subtly different semantics, it just got fearfully confusing.

For compatibility reasons I suspect we need to put the renamed versions
in something new ('cl-dumped', say, or 'cl-clean'), and have the old
cl.el versions become defaliases to them. (The Emacs core's uses of cl
can be fixed up, but cl is widely used in third-party modules, so we
can't just drop those names.)

I can easily do that this weekend.

-- 
NULL && (void)

Re: CL package serious deficiencies

[Lennart Borgman]

On Wed, Feb 8, 2012 at 00:52, Nix <nix@esperi.org.uk> wrote:
> On 7 Feb 2012, Stefan Monnier said:
>
>>> I've never understood what's wrong with including cl.el, nor why the
>>
>> The main issue is namespace.  If someone goes through the code to rename
>> it all to "cl-*", then we won't need to avoid using it.
>
> Aha. I'd agree with *that*: it's always been hellishly unclear which
> things are in cl or not. Its intention to sort of add to the Emacs Lisp
> core with things with very similar names was perhaps praiseworthy, but
> once those things started to move *into* the non-cl.el core, often with
> subtly different semantics, it just got fearfully confusing.
>
> For compatibility reasons I suspect we need to put the renamed versions
> in something new ('cl-dumped', say, or 'cl-clean'), and have the old
> cl.el versions become defaliases to them. (The Emacs core's uses of cl
> can be fixed up, but cl is widely used in third-party modules, so we
> can't just drop those names.)

But we can have the same policy as now regarding the use of the old
names for things in cl.el.

Re: CL package serious deficiencies

[Nix]

On 8 Feb 2012, Lennart Borgman said:

> On Wed, Feb 8, 2012 at 00:52, Nix <nix@esperi.org.uk> wrote:
>> For compatibility reasons I suspect we need to put the renamed versions
>> in something new ('cl-dumped', say, or 'cl-clean'), and have the old
>> cl.el versions become defaliases to them. (The Emacs core's uses of cl
>> can be fixed up, but cl is widely used in third-party modules, so we
>> can't just drop those names.)
>
> But we can have the same policy as now regarding the use of the old
> names for things in cl.el.

Sure. One downside is that this means that the intricate code in
bytecomp.el can't be removed (surely not until use of cl-clean, or
whatever we call it, becomes second nature to elisp writers). I'm not
sure what to do about that: it feels inelegant to have a bunch of code
in the byte-compiler whose only purpose is to discriminate against one
particular package, be it ever so ugly. But that's a minor thing.

-- 
NULL && (void)

Re: CL package serious deficiencies

[Stefan Monnier]

> Sure.  One downside is that this means that the intricate code in
> bytecomp.el can't be removed (surely not until use of cl-clean, or
> whatever we call it, becomes second nature to elisp writers).

It could be simplified, tho, to just deprecate the use of (require 'cl)
altogether in favor of the new names.  Of course, this wouldn't be
convenient for packages that want to work on older Emacs and XEmacs
as well.


        Stefan

Re: CL package serious deficiencies

[Nix]

On 8 Feb 2012, Stefan Monnier stated:

>> Sure.  One downside is that this means that the intricate code in
>> bytecomp.el can't be removed (surely not until use of cl-clean, or
>> whatever we call it, becomes second nature to elisp writers).
>
> It could be simplified, tho, to just deprecate the use of (require 'cl)
> altogether in favor of the new names.  Of course, this wouldn't be
> convenient for packages that want to work on older Emacs and XEmacs
> as well.

XEmacs could easily hack up a package that defaliases the other way if
it wanted to (or defaliases the same way). Not a problem.

-- 
NULL && (void)

Re: CL package serious deficiencies

[Stefan Monnier]

>>> Sure.  One downside is that this means that the intricate code in
>>> bytecomp.el can't be removed (surely not until use of cl-clean, or
>>> whatever we call it, becomes second nature to elisp writers).
>> It could be simplified, tho, to just deprecate the use of (require 'cl)
>> altogether in favor of the new names.  Of course, this wouldn't be
>> convenient for packages that want to work on older Emacs and XEmacs
>> as well.
> XEmacs could easily hack up a package that defaliases the other way if
> it wanted to (or defaliases the same way).  Not a problem.

Yes, it's an inconvenience, but it's not a big problem.


        Stefan

Re: CL package serious deficiencies

[Stephen J. Turnbull]

Nix writes:
 > On 7 Feb 2012, Stefan Monnier said:
 > 
 > >> I've never understood what's wrong with including cl.el, nor why the
 > >
 > > The main issue is namespace.  If someone goes through the code to rename
 > > it all to "cl-*", then we won't need to avoid using it.
 > 
 > Aha. I'd agree with *that*: it's always been hellishly unclear which
 > things are in cl or not.

Oh, s**t.  *We* do *not* agree, since we're moving in the direction of
exact Common Lisp conformance for the subset of features we provide (I
think Aidan has the intention of reducing cl.el to

    ;;; cl --- Common Lisp emulation for Emacsen
    (provide 'cl)
    ;;; end of cl.el

in the near future).

Not that anybody on this list *should* care about that, but just in
case.... ;-)

Re: CL package serious deficiencies

[Lennart Borgman]

On Wed, Feb 8, 2012 at 03:13, Stephen J. Turnbull <stephen@xemacs.org> wrote:
> Nix writes:
>  > On 7 Feb 2012, Stefan Monnier said:
>  >
>  > >> I've never understood what's wrong with including cl.el, nor why the
>  > >
>  > > The main issue is namespace.  If someone goes through the code to rename
>  > > it all to "cl-*", then we won't need to avoid using it.
>  >
>  > Aha. I'd agree with *that*: it's always been hellishly unclear which
>  > things are in cl or not.
>
> Oh, s**t.  *We* do *not* agree, since we're moving in the direction of
> exact Common Lisp conformance for the subset of features we provide (I
> think Aidan has the intention of reducing cl.el to
>
>    ;;; cl --- Common Lisp emulation for Emacsen
>    (provide 'cl)
>    ;;; end of cl.el
>
> in the near future).
>
> Not that anybody on this list *should* care about that, but just in
> case.... ;-)

This reminds me that the help functions does not work for cl things. A
serious attempt to make those functions useful should probably fix
that too.

Re: CL package serious deficiencies

[Stephen J. Turnbull]

Lennart Borgman writes:

 > This reminds me that the help functions does not work for cl things. A
 > serious attempt to make those functions useful should probably fix
 > that too.

Which help functions?  What do you mean "doesn't work?"

I really wish you would stop throwing FUD around, Lennart.  You keep
saying you've seen problems but you can never tell us what they were.
That's no help at all.

Re: CL package serious deficiencies

[Lennart Borgman]

On Wed, Feb 8, 2012 at 05:23, Stephen J. Turnbull <stephen@xemacs.org> wrote:
> Lennart Borgman writes:
>
>  > This reminds me that the help functions does not work for cl things. A
>  > serious attempt to make those functions useful should probably fix
>  > that too.
>
> Which help functions?  What do you mean "doesn't work?"

Sorry, I thought that was clear.

I never use CL myself, because I have not had time to learn it. (The
only lisp dialect I have been using - except for some small course, 5
weeks, in logic programming - is elisp.)

When trying to tweak ert.el so I could use it for tests in nXhtml I
ran across cl structures and did not understand at all how to read the
values in them. I therefore wrote some small functions to help myself
and others in this dilemma:

Those functions are all in ourcomments-util.el in nXhtml. The main
function is describe-defstruct. If I remember correctly I had to make
a minor change in cl.el too to make it all work.

Re: CL package serious deficiencies

[Richard Stallman]

There could be two packages:

* cl-real.el in which every CL function had a name starting with `cl-'.

* cl.el, which defines aliases for those functions.

The issues about cl.el would be the same as now,
but cl-real.el would not raise all these issues.
It might be ok for cl-real.el to be preloaded.

However, the more common the use of these functions becomes, the more
we would need to include them in the Emacs Lisp manual anyway.

-- 
Dr Richard Stallman
President, Free Software Foundation
51 Franklin St
Boston MA 02110
USA
www.fsf.org  www.gnu.org
Skype: No way! That's nonfree (freedom-denying) software.
  Use free telephony http://directory.fsf.org/category/tel/

Re: CL package serious deficiencies

[Stefan Monnier]

> It might be ok for cl-real.el to be preloaded.

More importantly, it can be required at run-time.


        Stefan

Re: CL package serious deficiencies

[Daniel Colascione]

[-- Attachment #1: Type: text/plain, Size: 559 bytes --]

On 2/7/12 3:48 PM, Stefan Monnier wrote:
>> I've never understood what's wrong with including cl.el, nor why the
> 
> The main issue is namespace.  If someone goes through the code to rename
> it all to "cl-*", then we won't need to avoid using it.

I don't think the namespace is an issue. A _lot_ of people do, in
fact, (require 'cl), and packages continue to work for them. If there
were any serious conflicts, they'd already be fixed.

In other words, I don't think you can find an elisp package that only
works properly if cl isn't loaded.


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 235 bytes --]

Re: CL package serious deficiencies

[Lennart Borgman]

On Wed, Feb 8, 2012 at 01:38, Daniel Colascione <dancol@dancol.org> wrote:
> On 2/7/12 3:48 PM, Stefan Monnier wrote:
>>> I've never understood what's wrong with including cl.el, nor why the
>>
>> The main issue is namespace.  If someone goes through the code to rename
>> it all to "cl-*", then we won't need to avoid using it.
>
> I don't think the namespace is an issue. A _lot_ of people do, in
> fact, (require 'cl), and packages continue to work for them. If there
> were any serious conflicts, they'd already be fixed.
>
> In other words, I don't think you can find an elisp package that only
> works properly if cl isn't loaded.

I have seen the problem. My conclusion is rather that most users do
not understand complex problems very well.

Re: CL package serious deficiencies

[Stefan Monnier]

>>> I've never understood what's wrong with including cl.el, nor why the
>> The main issue is namespace.  If someone goes through the code to rename
>> it all to "cl-*", then we won't need to avoid using it.

> I don't think the namespace is an issue. A _lot_ of people do, in
> fact, (require 'cl), and packages continue to work for them. If there
> were any serious conflicts, they'd already be fixed.
> In other words, I don't think you can find an elisp package that only
> works properly if cl isn't loaded.

That's not the issue.  The issue is that CL's namespace is really
unclean and that's bad, regardless of whether other packages conflict.


        Stefan

Re: CL package serious deficiencies

[Daniel Colascione]

[-- Attachment #1: Type: text/plain, Size: 1144 bytes --]

On 2/7/12 5:53 PM, Stefan Monnier wrote:
>>>> I've never understood what's wrong with including cl.el, nor why the
>>> The main issue is namespace.  If someone goes through the code to rename
>>> it all to "cl-*", then we won't need to avoid using it.
> 
>> I don't think the namespace is an issue. A _lot_ of people do, in
>> fact, (require 'cl), and packages continue to work for them. If there
>> were any serious conflicts, they'd already be fixed.
>> In other words, I don't think you can find an elisp package that only
>> works properly if cl isn't loaded.
> 
> That's not the issue.  The issue is that CL's namespace is really
> unclean and that's bad, regardless of whether other packages conflict.

While this allegation may be true, cl is so old and so generally
useful that it should be grandfathered in.

Also, cl.el _does_ take pains to use the "cl-" prefix for internal
symbols. It's the public interface that's unprefixed, and for the sake
of both convenience and backward compatibility, we shouldn't move the
entire CL library into its own namespace until we have some kind of
symbol namespace support.


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 235 bytes --]

Re: CL package serious deficiencies

[Lars Ingebrigtsen]

Nix <nix@esperi.org.uk> writes:

> I've never understood what's wrong with including cl.el, nor why the
> byte-compiler should warn specially about it, any more than it warns
> about any other package.

Since the beginning of time, the Emacs maintainer (whoever they were at
the time) just hasn't liked Common Lisp.  The stated rationale for not
"allowing" cl.el usage has shifted around a lot over the years, though.
("It's too big run-time-wise", "we're going to reimplement Emacs in
Scheme", and now "the manual will be too big" and "it uses the wrong
prefix".)

Meanwhile, most of the people who program Emacs Lisp daily (i.e., people
like me) have always been in favour of including it.  Who doesn't want
`incf'?  `plusp'?  `delete-if-not'?  `position'?  So you get all these
hundreds of reimplementations of all these necessary functions, only
spread over all the different packages.

-- 
(domestic pets only, the antidote for overdose, milk.)
  http://lars.ingebrigtsen.no  *  Sent from my Rome

Re: CL package serious deficiencies

[Lennart Borgman]

On Wed, Feb 8, 2012 at 23:28, Lars Ingebrigtsen <larsi@gnus.org> wrote:
> Nix <nix@esperi.org.uk> writes:
>
>> I've never understood what's wrong with including cl.el, nor why the
>> byte-compiler should warn specially about it, any more than it warns
>> about any other package.
>
> Since the beginning of time, the Emacs maintainer (whoever they were at
> the time) just hasn't liked Common Lisp.  The stated rationale for not
> "allowing" cl.el usage has shifted around a lot over the years, though.
> ("It's too big run-time-wise", "we're going to reimplement Emacs in
> Scheme", and now "the manual will be too big" and "it uses the wrong
> prefix".)
>
> Meanwhile, most of the people who program Emacs Lisp daily (i.e., people
> like me) have always been in favour of including it.  Who doesn't want
> `incf'?  `plusp'?  `delete-if-not'?  `position'?  So you get all these
> hundreds of reimplementations of all these necessary functions, only
> spread over all the different packages.

From someone who do not know at all: Should perhaps some of the
functions in cl.el etc be reimplemented now that elisp knows lexical
binding?

Re: CL package serious deficiencies

[Nix]

On 8 Feb 2012, Lars Ingebrigtsen verbalised:

> Meanwhile, most of the people who program Emacs Lisp daily (i.e., people
> like me) have always been in favour of including it.  Who doesn't want
> `incf'?  `plusp'?  `delete-if-not'?  `position'?  So you get all these
> hundreds of reimplementations of all these necessary functions, only
> spread over all the different packages.

Exactly. It's the latter ugliness that I think is really, well, ugly.
Duplicated code: duplicated overheads: duplicated bugs. Bleah.

-- 
NULL && (void)

Re: CL package serious deficiencies

[Richard Stallman]

Duplication of code is not necessarily a bad thing.
I learned in writing GCC that it's often cleaner for each
program to have a function that does what it wants
than to push to make them shared.

However, when it is clear a standard function would be useful, we can
add one.  We have added several functions that were previously in CL.
Maybe some others are worth adding now.  But that does not mean we
have to choose all or nothing.

-- 
Dr Richard Stallman
President, Free Software Foundation
51 Franklin St
Boston MA 02110
USA
www.fsf.org  www.gnu.org
Skype: No way! That's nonfree (freedom-denying) software.
  Use free telephony http://directory.fsf.org/category/tel/

Re: CL package serious deficiencies

[Nix]

On 9 Feb 2012, Richard Stallman outgrape:

> Duplication of code is not necessarily a bad thing.
> I learned in writing GCC that it's often cleaner for each
> program to have a function that does what it wants
> than to push to make them shared.

This is often true, particularly when the functions differ in small but
annoying-to-characterize ways.

I really, really doubt that this true of e.g. 'plusp' though. :)

> However, when it is clear a standard function would be useful, we can
> add one.  We have added several functions that were previously in CL.
> Maybe some others are worth adding now.  But that does not mean we
> have to choose all or nothing.

OK. It's probably best to at least generate a starting list by looking
at the heavily-duplicated stuff. Things in cl that nobody is
reimplementing are clearly not worth migrating out... and if we can make
the things we migrate more elispy at the same time, so much the better!

-- 
NULL && (void)

Re: CL package serious deficiencies

[John Wiegley]

>>>>> Lars Ingebrigtsen <larsi@gnus.org> writes:

> Meanwhile, most of the people who program Emacs Lisp daily (i.e., people
> like me) have always been in favour of including it.  Who doesn't want
> `incf'?  `plusp'?  `delete-if-not'?  `position'?  So you get all these
> hundreds of reimplementations of all these necessary functions, only spread
> over all the different packages.

I completely agree, Lars.  Eshell had to reimplement several functions that I
knew were in cl.el, simply because of this restriction.

Some of the things in cl.el -- such as loop, defun*, and a few others -- I
could care less about.  These are more about coding in a CL style as opposed
to the equivalent Elisp style.  But other functions that concern basic
algorithms, such as `reduce', I really would like to see dumped into Emacs
core.  *With* keywords (they make some usages very clear and easy to read).

I'm also one of those who has had cl.el required into my Emacs for over ten
years now, and I've never once encountered a hiccup because of it.  It's time
has come!

John

Re: CL package serious deficiencies

[Stefan Monnier]

>> I've never understood what's wrong with including cl.el, nor why the
>> byte-compiler should warn specially about it, any more than it warns
>> about any other package.
> Since the beginning of time, the Emacs maintainer (whoever they were at
> the time) just hasn't liked Common Lisp.

Actually, I like many parts of Common Lisp.  E.g. I intend to move the
compiler macros to the Elisp core (they're pretty much there already since
they require support in the bytecompiler).  I also like the `setf' macro
and related machinery.  Of course defstruct is also good (because "more
types" is good in my world).

> Meanwhile, most of the people who program Emacs Lisp daily (i.e., people
> like me) have always been in favour of including it.  Who doesn't want
> `incf'?

I've used incf occasionally, but don't find it terribly important.

> `plusp'?

Never used it.  (> n 0) is no shorter than (plusp n) and is just as
clear, so I really don't see the benefit.

> `delete-if-not'?

That's the big one: some kind of "filter elements based on a predicate"
is really handy and we definitely need to have this in core Elisp.
OTOH I don't like the -if-not/-if duplication nor all the keyword
arguments it takes, which just complicate significantly the
implementation with very little benefit to the user/reader.

> `position'?

I think having cl-position (after (require 'cl-lib)) is good enough for
this one (it's handy and more readable than the Elisp replacement, but
it's not used often).


        Stefan

Re: CL package serious deficiencies

[Juanma Barranquero]

On Fri, Feb 10, 2012 at 15:38, Stefan Monnier <monnier@iro.umontreal.ca> wrote:

> I've used incf occasionally, but don't find it terribly important.

Is not terribly important, but it is more expresive than (setq x (1+
x)) when x is this-very-longish-variable-whose-name-will-never-really-end,
and you have too look closely to be sure that the second instance
isn't really this-other-very-longish-variable-whose-name-will-not-end-either.

Another one I use a lot is pushnew.

> That's the big one: some kind of "filter elements based on a predicate"
> is really handy and we definitely need to have this in core Elisp.

Yes, definitely.

> OTOH I don't like the -if-not/-if duplication nor all the keyword
> arguments it takes

The -if/-if-not duplication is not needed, but as for the keywords, I
find most of them useful and handy, except perhaps :test-not.

> I think having cl-position (after (require 'cl-lib)) is good enough for
> this one (it's handy and more readable than the Elisp replacement, but
> it's not used often).

Agreed that position is less versatile than remove* or
delete-duplicates, but that's a bit self-defeating. Many of these
functions, position included, would get more use if they were better
integrated.

    Juanma

RE: CL package serious deficiencies

[Drew Adams]

I agree with what Juanma and others have said about some of the more commonly
used Common-Lisp functions, particularly the sequence functions.

I would add that being able to use generalized variables (`setf' generally, in
addition to `incf' etc.) would also be on my short list.

http://www.gnu.org/software/emacs/manual/html_node/cl/Generalized-Variables.html
#Generalized-Variables

http://www.cs.cmu.edu/Groups/AI/html/cltl/clm/node80.html

Note that this would also mean generalizing some existing Emacs things like
`push' in Emacs, letting you use generalized variables, not just variables
(symbols), as their args.  And it [wc]ould obsolete some existing things, like
`add-to-list' - see `pushnew'.

More generally, if generalized variable support were added, there would be other
existing Emacs functions/macros that currently require a variable as arg but
which could be generalized to accept a generalized variable.

IOW, adding a given CL feature doesn't necessarily mean just adding that feature
per se - it could also mean taking advantage of that feature to improve Emacs
Lisp more generally.

(But we've been through all of this more than once before, discussing various
lists of candidate CL features to add.  And little to nothing has come of it.
That does not mean that this time won't be the charm, but I'm skeptical...)

Re: CL package serious deficiencies

[Stefan Monnier]

> I would add that being able to use generalized variables (`setf'
> generally, in addition to `incf' etc.) would also be on my short list.

I like generalized variables as well.

> (But we've been through all of this more than once before, discussing various
> lists of candidate CL features to add.  And little to nothing has come of it.
> That does not mean that this time won't be the charm, but I'm skeptical...)

I think a first step is to provide a new package `cl-lib' which would
provide the CL functions (i.e. all those functions people seem to like
but can't use because we disallow CL at runtime) under a clean
"cl-" prefix.  This would allow all of CL to be used (either via
(require 'cl-lib) or (eval-when-compile (require 'cl))) and would hence
address the most pressing issues.


        Stefan

RE: CL package serious deficiencies

[Drew Adams]

> I think a first step is to provide a new package `cl-lib' which would
> provide the CL functions (i.e. all those functions people seem to like
> but can't use because we disallow CL at runtime) under a clean
> "cl-" prefix.  This would allow all of CL to be used (either via
> (require 'cl-lib) or (eval-when-compile (require 'cl))) and 
> would hence address the most pressing issues.

1+

(But I don't see how (eval-when-compile (require 'cl))) would allow all of CL to
be used, unless you meant only at compile time.)

That also has the advantage of skirting, for now, (a) the problem of conflicts
with Emacs stuff with the same name and (b) the work needed to replace the more
limited Emacs stuff that has the same name.

Re: CL package serious deficiencies

[Lars Ingebrigtsen]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> I've used incf occasionally, but don't find it terribly important.

I find it adds readability, since you don't have to see that
"(setq foo (1+ foo))" has the same `foo' both places.

>> `plusp'?
>
> Never used it.  (> n 0) is no shorter than (plusp n) and is just as
> clear, so I really don't see the benefit.

Again, I think it's usually more readable, because (as with `zerop')
it's immediately obvious what the condition is.  Especially with longer
parameters:

(plusp (1+ (foo-thing-that-computes-something)))

vs

(> (1+ (foo-thing-that-computes-something)) 0)

>> `delete-if-not'?
>
> That's the big one: some kind of "filter elements based on a predicate"
> is really handy and we definitely need to have this in core Elisp.
> OTOH I don't like the -if-not/-if duplication nor all the keyword
> arguments it takes, which just complicate significantly the
> implementation with very little benefit to the user/reader.

I think the `-if' and `-if-not' help with reading comprehension, because
you're signalling in the function name what you mean to do, and you
don't have to read the predicate as thoroughly.  And the predicate can
be written clearer, too.  If you don't have both forms, the predicate
often tends to look like `(lambda (foo) (not ...))', which makes my
brain hurt.

>> `position'?
>
> I think having cl-position (after (require 'cl-lib)) is good enough for
> this one (it's handy and more readable than the Elisp replacement, but
> it's not used often).

Hey, are we listing all the CL functions that we find most useful?

`mapcan', `some', `every', `notany', `reduce', `remove-duplicates'...
Uhm...  I think that's it for me...  (In addition to the ones
mentioned.)

-- 
(domestic pets only, the antidote for overdose, milk.)
  http://lars.ingebrigtsen.no  *  Sent from my Rome

Re: CL package serious deficiencies

[Stefan Monnier]

>> I've used incf occasionally, but don't find it terribly important.
> I find it adds readability, since you don't have to see that
> "(setq foo (1+ foo))" has the same `foo' both places.

No disagreement (I also dislike such duplication, which is why I insist
on using (set (make-local-variable 'var) val)), I just don't find it's
often enough important (in my experience it's not used that often, and
most times it's used, it's used on locally-bound variables with short
names).
This said, if/when we have `setf', then there's no reason not to have
`incf'.

>>> `plusp'?
>> Never used it.  (> n 0) is no shorter than (plusp n) and is just as
>> clear, so I really don't see the benefit.
> Again, I think it's usually more readable, because (as with `zerop')
> it's immediately obvious what the condition is.  Especially with longer
> parameters:

> (plusp (1+ (foo-thing-that-computes-something)))
> vs
> (> (1+ (foo-thing-that-computes-something)) 0)

that's because you write it wrong:

  (< 0 (1+ (foo-thing-that-computes-something)))
aka
  (< -1 (foo-thing-that-computes-something))

aka (assuming it's integer)

  (<= 0 (foo-thing-that-computes-something))

> I think the `-if' and `-if-not' help with reading comprehension, because
> you're signalling in the function name what you mean to do, and you
> don't have to read the predicate as thoroughly.  And the predicate can
> be written clearer, too.  If you don't have both forms, the predicate
> often tends to look like `(lambda (foo) (not ...))', which makes my
> brain hurt.

That's just a question of habit, really.

[ You might prefer "delete-if (¬ ∘ pred) list", of course.  ]


        Stefan

Re: CL package serious deficiencies

[Lars Ingebrigtsen]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> that's because you write it wrong:
>
>   (< 0 (1+ (foo-thing-that-computes-something)))
> aka
>   (< -1 (foo-thing-that-computes-something))

By all that's holy!  (I.e. Emacs!)  That's just wrong!

I mean, really.

-- 
(domestic pets only, the antidote for overdose, milk.)
  http://lars.ingebrigtsen.no  *  Sent from my Rome

Re: CL package serious deficiencies

[Johan Bockgård]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> I think the `-if' and `-if-not' help with reading comprehension, because
>> you're signalling in the function name what you mean to do, and you
>> don't have to read the predicate as thoroughly.  And the predicate can
>> be written clearer, too.  If you don't have both forms, the predicate
>> often tends to look like `(lambda (foo) (not ...))', which makes my
>> brain hurt.
>
> That's just a question of habit, really.
>
> [ You might prefer "delete-if (¬ ∘ pred) list", of course.  ]

AKA `complement' in Common Lisp:

--8<---------------cut here---------------start------------->8---

      (complement x) == #'(lambda (&rest arguments) (not (apply x arguments)))

In Common Lisp, functions with names like "xxx-if-not" are related to
functions with names like "xxx-if" in that

     (xxx-if-not f . arguments) == (xxx-if (complement f) . arguments)

For example,

      (find-if-not #'zerop '(0 0 3)) ==
      (find-if (complement #'zerop) '(0 0 3)) =>  3

--8<---------------cut here---------------end--------------->8---

Re: CL package serious deficiencies

[Helmut Eller]

* Stefan Monnier [2012-02-10 14:38] writes:

>>> I've never understood what's wrong with including cl.el, nor why the
>>> byte-compiler should warn specially about it, any more than it warns
>>> about any other package.
>> Since the beginning of time, the Emacs maintainer (whoever they were at
>> the time) just hasn't liked Common Lisp.
>
> Actually, I like many parts of Common Lisp.  E.g. I intend to move the
> compiler macros to the Elisp core (they're pretty much there already since
> they require support in the bytecompiler).  I also like the `setf' macro
> and related machinery.  Of course defstruct is also good (because "more
> types" is good in my world).

At the risk of stating the obvious: we could define compiler-macros for
keyword-using functions like position and automatically rewrite all uses
to a keyword-less implementation like cl-position.  Then everybody could
use the prefix-less version without the need to (require 'cl) at
runtime.

Helmut

Re: CL package serious deficiencies

[Stefan Monnier]

> At the risk of stating the obvious: we could define compiler-macros for
> keyword-using functions like position and automatically rewrite all uses
> to a keyword-less implementation like cl-position.  Then everybody could
> use the prefix-less version without the need to (require 'cl) at
> runtime.

The problem with this approach is that you end up having to first
implement the function (complex because of all the args you have to
handle), then re-implement it all as a compiler-macro (which happens to
be more complex and bug-prone and more difficult to debug, usually).
In the end, a lot of complexity, lots of bugs, difficult to maintain,
and the function itself is dog-slow because it has to handle all the
weird combinations.
Even a pure macro makes more sense at that point.

I much prefer a leaner approach where instead of (delete-if #'foo :key #'bar)
you have to write (delete-if (lambda (x) (foo (bar x)))).
As for :start :end :from-end and :count, I've never even seen them used
with delete-if.

The :key in `sort*' makes sense.  Not in `delete-if'.


        Stefan

Re: CL package serious deficiencies

[Lars Ingebrigtsen]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> I much prefer a leaner approach where instead of (delete-if #'foo :key #'bar)
> you have to write (delete-if (lambda (x) (foo (bar x)))).
> As for :start :end :from-end and :count, I've never even seen them used
> with delete-if.
>
> The :key in `sort*' makes sense.  Not in `delete-if'.

I agree.  I suspect the CL standard adds all these keywords to all the
sequence functions mostly for consistency.

:key is quite nice for `assoc' and `member', though.

-- 
(domestic pets only, the antidote for overdose, milk.)
  http://lars.ingebrigtsen.no  *  Sent from my Rome

Re: CL package serious deficiencies

[Teemu Likonen]

* Stefan Monnier [2012-02-10 13:51:14 -0500] wrote:

> As for :start :end :from-end and :count, I've never even seen them
> used with delete-if.

Here's one example:

    (defun delete-nth (n sequence)
      (delete-if (constantly t) sequence :start n :count 1))

Re: CL package serious deficiencies

[Juanma Barranquero]

On Fri, Feb 10, 2012 at 20:08, Teemu Likonen <tlikonen@iki.fi> wrote:

> Here's one example:
>
>    (defun delete-nth (n sequence)
>      (delete-if (constantly t) sequence :start n :count 1))

BTW,

(defun delete-last-nth (n sequence)
    (delete-if (lambda (_) (minusp (decf n))) sequence :from-end t :count 1))

should work, and it does in SBCL

  * (defun delete-last-nth (n sequence) (delete-if (lambda (_) (minusp
(decf n))) sequence :from-end t :count 1))
  ; in: DEFUN DELETE-LAST-NTH
  ;     (LAMBDA (_) (MINUSP (DECF N)))
  ; ==>
  ;   #'(LAMBDA (_) (MINUSP (DECF N)))
  ;
  ; caught STYLE-WARNING:
  ;   The variable _ is defined but never used.
  ;
  ; compilation unit finished
  ;   caught 1 STYLE-WARNING condition

  DELETE-LAST-NTH
  * (delete-last-nth 0 '(a b c))

  (A B)
  * (delete-last-nth 1 '(a b c))

  (A C)
  * (delete-last-nth 2 '(a b c))

  (B C)


but not in elisp:

  ELISP> (delete-nth-from-end 0 '(a b c))
  (a b)

  ELISP> (delete-nth-from-end 1 '(a b c))
  (a b)

  ELISP> (delete-nth-from-end 2 '(a b c))
  (a b)

That is a bug, I gather.

    Juanma

Re: CL package serious deficiencies

[Johan Bockgård]

Juanma Barranquero <lekktu@gmail.com> writes:

>   ELISP> (delete-nth-from-end 0 '(a b c))
>   (a b)
>
>   ELISP> (delete-nth-from-end 1 '(a b c))
>   (a b)
>
>   ELISP> (delete-nth-from-end 2 '(a b c))
>   (a b)
>
> That is a bug, I gather.

There is no promise about in which order the sequence will be traversed
(only from which end elements are :count'ed).

Re: CL package serious deficiencies

[Juanma Barranquero]

On Sat, Feb 11, 2012 at 17:56, Johan Bockgård <bojohan@gnu.org> wrote:

> There is no promise about in which order the sequence will be traversed
> (only from which end elements are :count'ed).

You're right. At least, I see no such guarantee in the appropriate 17.3 section.

It is surprising, though, that the promise does exist for COUNT,
COUNT-IF, COUNT-IF-NOT:

"The from-end has no direct effect on the result. However, if from-end
is true, the elements of sequence will be supplied as arguments to the
test, test-not, and key in reverse order, which may change the
side-effects, if any, of those functions."

I mean, :from-end would not be useful for COUNT otherwise, so the
language designers gave it something to do. But the result is that
COUNT and friends are the only sequence functions which guarantee the
order (and so, side effects) of the test and key.

Thanks,

    Juanma

Re: CL package serious deficiencies

[Stefan Monnier]

>> As for :start :end :from-end and :count, I've never even seen them
>> used with delete-if.
> Here's one example:
>     (defun delete-nth (n sequence)
>       (delete-if (constantly t) sequence :start n :count 1))

Two problems:
1- that can be implemented just as well with pop+nthcdr.
2- I can't remember seeing any code that needs such a functionality either.


        Stefan

Re: CL package serious deficiencies

[Teemu Likonen]

* Stefan Monnier [2012-02-11 23:41:07 -0500] wrote:

>>> As for :start :end :from-end and :count, I've never even seen them
>>> used with delete-if.
>> Here's one example:
>>     (defun delete-nth (n sequence)
>>       (delete-if (constantly t) sequence :start n :count 1))
>
> Two problems:
> 1- that can be implemented just as well with pop+nthcdr.

NTHCDR is not an accessor so (pop (nthcdr n list)) doesn't work. Both
POP and NTHCDR are for lists. DELETE-IF is for all sequences.

Those keyword arguments are not useless. Some of them are rarely needed
and that's why it's good that they are &key and not &optional.

I think

    (replace-match "foo" :subexp 2)

is much more readable than

    (replace-match "foo" nil nil nil 2)

Re: CL package serious deficiencies

[Stefan Monnier]

> NTHCDR is not an accessor

Shouldn't be hard to fix.


        Stefan

Re: CL package serious deficiencies

[Stefan Monnier]

> Those keyword arguments are not useless. Some of them are rarely needed
> and that's why it's good that they are &key and not &optional.

> I think
>     (replace-match "foo" :subexp 2)
> is much more readable than
>     (replace-match "foo" nil nil nil 2)

I'm not opposed to keyword args at all (I introduced them in several
macros, as a matter of fact).  I'm opposed to args that are almost
never used and where the same result can be obtained fairly easily some
other way.


        Stefan

Re: CL package serious deficiencies

[Helmut Eller]

* Stefan Monnier [2012-02-10 18:51] writes:

>> At the risk of stating the obvious: we could define compiler-macros for
>> keyword-using functions like position and automatically rewrite all uses
>> to a keyword-less implementation like cl-position.  Then everybody could
>> use the prefix-less version without the need to (require 'cl) at
>> runtime.
>
> The problem with this approach is that you end up having to first
> implement the function (complex because of all the args you have to
> handle), then re-implement it all as a compiler-macro (which happens to
> be more complex and bug-prone and more difficult to debug, usually).
> In the end, a lot of complexity, lots of bugs, difficult to maintain,
> and the function itself is dog-slow because it has to handle all the
> weird combinations.
> Even a pure macro makes more sense at that point.
>
> I much prefer a leaner approach where instead of (delete-if #'foo :key #'bar)
> you have to write (delete-if (lambda (x) (foo (bar x)))).

This transformation can be done automatically by a compiler-macro.

> As for :start :end :from-end and :count, I've never even seen them used
> with delete-if.

Well, dropping the keyword args of delete-if would not be backward
compatible and quite a useless move.  And the :count argument is pretty
cool; surely beats stuff like byte-compile-delete-first.

> The :key in `sort*' makes sense.  Not in `delete-if'.

Sure it does.  Here's an example for remove-if with :key arg:
https://raw.github.com/nablaone/slime/7dfadd8716d2542d9290231b79467df803e8803f/slime.el

Helmut

Re: CL package serious deficiencies

[Stefan Monnier]

>> I much prefer a leaner approach where instead of (delete-if
>> #'foo :key #'bar) you have to write (delete-if (lambda (x) (foo (bar
>> x)))).
> This transformation can be done automatically by a compiler-macro.

Not sure what you mean by "automatically", here.  Do you mean "without
prompting the user to do the transformation by hand"?


        Stefan

Re: CL package serious deficiencies

[Helmut Eller]

* Stefan Monnier [2012-02-12 04:47] writes:

>>> I much prefer a leaner approach where instead of (delete-if
>>> #'foo :key #'bar) you have to write (delete-if (lambda (x) (foo (bar
>>> x)))).
>> This transformation can be done automatically by a compiler-macro.
>
> Not sure what you mean by "automatically", here.  Do you mean "without
> prompting the user to do the transformation by hand"?

Yes.

Helmut

Re: CL package serious deficiencies

[Stefan Monnier]

>>>> I much prefer a leaner approach where instead of (delete-if
>>>> #'foo :key #'bar) you have to write (delete-if (lambda (x) (foo (bar
>>>> x)))).
>>> This transformation can be done automatically by a compiler-macro.
>> Not sure what you mean by "automatically", here.  Do you mean "without
>> prompting the user to do the transformation by hand"?
> Yes.

That's the expected behavior of a compiler-macro, so really what you're
saying is that "This transformation can be done by a compiler-macro",
but the problem remains: you have to write the compiler-macro by hand,
and the function still has to handle the :key argument anyway, in case
the compiler-macro can't do its job.


        Stefan

Re: CL package serious deficiencies

[Helmut Eller]

* Stefan Monnier [2012-02-12 18:15] writes:

>>>>> I much prefer a leaner approach where instead of (delete-if
>>>>> #'foo :key #'bar) you have to write (delete-if (lambda (x) (foo (bar
>>>>> x)))).
>>>> This transformation can be done automatically by a compiler-macro.
>>> Not sure what you mean by "automatically", here.  Do you mean "without
>>> prompting the user to do the transformation by hand"?
>> Yes.
>
> That's the expected behavior of a compiler-macro, so really what you're
> saying is that "This transformation can be done by a compiler-macro",
> but the problem remains: you have to write the compiler-macro by hand,
> and the function still has to handle the :key argument anyway, in case
> the compiler-macro can't do its job.

Yes, compiler-macros have to written and the cases that aren't handled
by compiler-macros could then only be used when 'cl is loaded (that code
is already there).  Many uses of :key and :test can be handled by such
compiler-macros.  If things need to be fast, some parts must be written
in C.

Using compiler-macros would be a relatively easy way to provide
efficient implementations for the most important uses of cl-sequence
functions.  That's all.  If you like to spend your time designing
similar but incompatible functions just go ahead.

Helmut

Re: CL package serious deficiencies

[Dimitri Fontaine]

Lars Ingebrigtsen <larsi@gnus.org> writes:
> Meanwhile, most of the people who program Emacs Lisp daily (i.e., people
> like me) have always been in favour of including it.  Who doesn't want
> `incf'?  `plusp'?  `delete-if-not'?  `position'?  So you get all these
> hundreds of reimplementations of all these necessary functions, only
> spread over all the different packages.

Totally agreed.

Also allow me to add that I (require 'cl) in el-get, where 536 different
elisp packages are included (and used, as they were submitted all by
el-get users), plus all of emacswiki and ELPA, featuring more than 2100
packages supported. We also have easy support for all emacsmirror
packages, that's 3197 packages (though only 1000 more from the 2100
already counted I guess, considering duplicates.).

After closing something like 500 bugs in 2 years, I'm yet to see a
report from a user having a problem related to (require 'cl).

So yes please include that library in elisp itself, and in passing,
please do tell me again why elisp is not having some namespace support
already?

Regards,
-- 
dim