Why aren't there functions such as filter, take-while, etc. "by default"?

Why aren't there functions such as filter, take-while, etc. "by default"?

[Deniz Dogan]

Why does it have to be so hard to write Emacs Lisp without requiring cl?

Why aren't there functions such as filter (the equivalent of
remove-if-not in cl-seq.el), take-while, reduce etc. "native" in
Emacs?

-- 
Deniz Dogan

Re: Why aren't there functions such as filter, take-while, etc. "by default"?

[David Kastrup]

Deniz Dogan <deniz.a.m.dogan@gmail.com> writes:

> Why does it have to be so hard to write Emacs Lisp without requiring cl?
>
> Why aren't there functions such as filter (the equivalent of
> remove-if-not in cl-seq.el), take-while, reduce etc. "native" in
> Emacs?

They make for no-surprise efficient programs primarily when the language
has lexical closures.

-- 
David Kastrup

Re: Why aren't there functions such as filter, take-while, etc. "by default"?

[Štěpán Němec]

David Kastrup <dak@gnu.org> writes:

> Deniz Dogan <deniz.a.m.dogan@gmail.com> writes:
>
>> Why does it have to be so hard to write Emacs Lisp without requiring cl?
>>
>> Why aren't there functions such as filter (the equivalent of
>> remove-if-not in cl-seq.el), take-while, reduce etc. "native" in
>> Emacs?
>
> They make for no-surprise efficient programs primarily when the language
> has lexical closures.

What about `case', though?

Or `position'?

It's ridiculous -- you have to either define helpers again and again all
the time, or write unnecessarily verbose code, or require the whole of
cl (and I don't think `eval-when-compile' makes so much of a difference
outside Emacs core; it's not like people compile all third-party
packages they use).

The Emacs cl.el "policy" seems rather schizophrenic to me -- if there's
something unsatisfactory about it, then it should be fixed instead of
being banished to compile-time-only. Or failing that, some generally
useful forms like the above should be provided in Emacs core.


  Štěpán

Re: Why aren't there functions such as filter, take-while, etc. "by default"?

[Deniz Dogan]

2010/4/25 David Kastrup <dak@gnu.org>:
> Deniz Dogan <deniz.a.m.dogan@gmail.com> writes:
>
>> Why does it have to be so hard to write Emacs Lisp without requiring cl?
>>
>> Why aren't there functions such as filter (the equivalent of
>> remove-if-not in cl-seq.el), take-while, reduce etc. "native" in
>> Emacs?
>
> They make for no-surprise efficient programs primarily when the language
> has lexical closures.
>

I see, I had never thought of that.

What about including these functions "by default" (or other
implementations of them) but with warnings in the docstrings about the
possibly unexpected behavior due to the dynamic scoping?

There are already functions that don't always act like one could
expect, e.g. `sort' which given a list and a predicate sorts the list
destructively (rendering the original list useless) and then returns a
sorted copy of the original list. These two behaviors are usually
mutually exclusive in other languages.

(let* ((my-list (list 3 2 1))
       (sorted (sort my-list '<)))
  my-list) ;; returns (3)

-- 
Deniz Dogan

RE: Why aren't there functions such as filter, take-while, etc. "by default"?

[Drew Adams]

> `sort' which given a list and a predicate sorts the list
> destructively (rendering the original list useless) and then returns a
> sorted copy of the original list.

`sort' does not return a copy. It returns the sorted list, that is, the modified
list.

,----
| sort is a built-in function in `C source code'.
| 
| (sort LIST PREDICATE)
| 
| Sort LIST, stably, comparing elements using PREDICATE.
| Returns the sorted list.  LIST is modified by side effects.
          ^^^^^^^^^^^^^^^           ^^^^^^^^
| PREDICATE is called with two elements of LIST, and should return non-nil
| if the first element should sort before the second.
| 
| [back]
`----

> (let* ((my-list (list 3 2 1))
>        (sorted (sort my-list '<)))
>   my-list) ;; returns (3)

The sorted list is (1 2 3). If your sexp returned `sorted' instead of `my-list'
then that is what you would get.

Your sexp returns (3) because `my-list' points to the cons cell whose car is 3,
and after sorting that same cons cell is the last one in the sorted list. If
`sort' returned a complete copy, then its result (`sorted') would not share any
list structure with the original list. Try this, and you will see that `sorted'
is (1 2 5). The last cons cell in `sorted' is the cons cell pointed to by
`my-list'.

(let* ((my-list (list 3 2 1))
       (sorted (sort my-list '<)))
  (setcar my-list 5)
  (message "sorted: %S" sorted) (sit-for 3) ; (1 2 5)
  my-list) ; returns (5)

Re: Why aren't there functions such as filter, take-while, etc. "by default"?

[Deniz Dogan]

2010/4/26 Drew Adams <drew.adams@oracle.com>:
>> `sort' which given a list and a predicate sorts the list
>> destructively (rendering the original list useless) and then returns a
>> sorted copy of the original list.
>
> `sort' does not return a copy. It returns the sorted list, that is, the modified
> list.
>
> ,----
> | sort is a built-in function in `C source code'.
> |
> | (sort LIST PREDICATE)
> |
> | Sort LIST, stably, comparing elements using PREDICATE.
> | Returns the sorted list.  LIST is modified by side effects.
>          ^^^^^^^^^^^^^^^           ^^^^^^^^
> | PREDICATE is called with two elements of LIST, and should return non-nil
> | if the first element should sort before the second.
> |
> | [back]
> `----
>
>> (let* ((my-list (list 3 2 1))
>>        (sorted (sort my-list '<)))
>>   my-list) ;; returns (3)
>
> The sorted list is (1 2 3). If your sexp returned `sorted' instead of `my-list'
> then that is what you would get.
>
> Your sexp returns (3) because `my-list' points to the cons cell whose car is 3,
> and after sorting that same cons cell is the last one in the sorted list. If
> `sort' returned a complete copy, then its result (`sorted') would not share any
> list structure with the original list. Try this, and you will see that `sorted'
> is (1 2 5). The last cons cell in `sorted' is the cons cell pointed to by
> `my-list'.
>
> (let* ((my-list (list 3 2 1))
>       (sorted (sort my-list '<)))
>  (setcar my-list 5)
>  (message "sorted: %S" sorted) (sit-for 3) ; (1 2 5)
>  my-list) ; returns (5)
>
>

Thank you for the explanation. However, I can't help but feel that
this is just proving my point... :-)

-- 
Deniz Dogan

Re: Why aren't there functions such as filter, take-while, etc. "by default"?

[Stefan Monnier]

>> Why does it have to be so hard to write Emacs Lisp without requiring cl?
>> Why aren't there functions such as filter (the equivalent of
>> remove-if-not in cl-seq.el), take-while, reduce etc. "native" in
>> Emacs?
> They make for no-surprise efficient programs primarily when the language
> has lexical closures.

I do not understand this.  What do you mean to say, and why is dynamic
scoping relevant (i.e. why wouldn't the same argument apply to mapcar,
mapconcat, etc...).


        Stefan

Re: Why aren't there functions such as filter, take-while, etc. "by default"?

[David Kastrup]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>>> Why does it have to be so hard to write Emacs Lisp without requiring cl?
>>> Why aren't there functions such as filter (the equivalent of
>>> remove-if-not in cl-seq.el), take-while, reduce etc. "native" in
>>> Emacs?
>> They make for no-surprise efficient programs primarily when the language
>> has lexical closures.
>
> I do not understand this.  What do you mean to say, and why is dynamic
> scoping relevant (i.e. why wouldn't the same argument apply to mapcar,
> mapconcat, etc...).

It does apply to the latter functions equally well.  However, those tend
to be usable with primitive functions without dynamic consequences
somewhat more often.

The more complex filtering functions of Common Lisp are very much
employed as an alternate programming style for iteration.  If there is
no conceivable way to transform them into byte code (or whatever) with a
similar efficiency and predictability as the equivalent iteration code,
they seem like a bad idea.

When learning Elisp, I have been negatively surprised by the bad
performance of using map* functions as opposed to straight iteration.

I should really like to see a version of Elisp where one-shot lambda
functions with outer references
a) need not be written
  (let ((outer ...)) ... `(lambda (x) (... ',outer ...
b) byte compile into the same or equally efficient byte code as their
   open-coded iteration equivalents

It is ok if the debugger can use debugging info/signatures for
approximating dynamical access for the user to the anonymous stack slots
of lexical variables, but normal operation should not be affected.

Since Lisp, after all, is supposed to be a functional language, I'd like
to see Emacs Lisp not penalize functional programming styles.  Currently
it does, and so I don't think general-purpose iteration constructs
encouraging lambda predicates are a good idea for now.

It would appear that both the lexbind branch as well as an all-out guile
strategy could provide paths forward.  The latter would seem to have the
advantage that we can get rid of the whole byte compilation issue in the
Emacs codebase (which would likely get more rather than less complex
with lexbind).  _And_ give guile a large kick forward in improvement
incentives.  Likely more than the boost that the Emacs project gives
Bazaar, and with more immediate benefits.

-- 
David Kastrup

Re: Why aren't there functions such as filter, take-while, etc. "by default"?

[David Kastrup]

David Kastrup <dak@gnu.org> writes:

> Since Lisp, after all, is supposed to be a functional language, I'd like
> to see Emacs Lisp not penalize functional programming styles.  Currently
> it does,

Not just With regard to performance, but also input readability (neither
`(lambda ... ,outer ...) nor the crutches involving make-symbol are
really straightforward), and predictability (dynamic rebinding!).

> and so I don't think general-purpose iteration constructs encouraging
> lambda predicates are a good idea for now.

-- 
David Kastrup