How to create a higher order function?

How to create a higher order function?

[Marcin Borkowski]

Hi all,

assume that I want a function which "inverts" the meaning of a given
predicate.  Something that would turn `<' into `>=' etc.

Here's my first attempt (A):

--8<---------------cut here---------------start------------->8---
;;; -*- lexical-binding: nil; -*-
(defun negate (fun)
  "Try to return a function returning the logical opposite of FUN."
  (lambda (&rest args)
    (not (apply fun args))))
--8<---------------cut here---------------end--------------->8---

Of course, it doesn't work, since (under dynamic scope) it just creates
a function which uses a dynamic binding for `fun' (which is probably
nil).

So, here's a better (at least: working) version (B):

--8<---------------cut here---------------start------------->8---
;;; -*- lexical-binding: nil; -*-
(defun negate (fun)
  "Return a function returning the logical opposite of FUN."
  `(lambda (&rest args)
    (not (apply ,(symbol-function fun) args))))
--8<---------------cut here---------------end--------------->8---

It seems to work fine, even if it's a bit complicated.

Of course, the more obvious way to do it is to use the first definition
under lexical scope (C):

--8<---------------cut here---------------start------------->8---
;;; -*- lexical-binding: t; -*-
(defun negate (fun)
  "Try to return a function returning the logical opposite of FUN."
  (lambda (&rest args)
    (not (apply fun args))))
--8<---------------cut here---------------end--------------->8---

My question is: which of (B) and (C) is better?  (C) is definitely
simpler, and with lexical scope becoming the default (perhaps at some
point in time in the future) or the recommended (even now, I guess)
setting, is probably the way to go.  But maybe (B) is better under some
circumstances?  Is it faster?  (I don't think so, and my simple
benchmarks were inconclusive.)  Is it more memory-efficient?  (Could
be.)  Does it have any other advantage?

Disclaimer: I'd like to study the replies and put the answer on my blog
and in my book.  I'll give the credit where it's due, of course.

TIA,

--
Marcin Borkowski
http://mbork.pl

Re: How to create a higher order function?

[Emanuel Berg via Users list for the GNU Emacs text editor]

Marcin Borkowski wrote:

> ;;; -*- lexical-binding: nil; -*-
> (defun negate (fun)
>   "Return a function returning the logical opposite of FUN."
>   `(lambda (&rest args)
>     (not (apply ,(symbol-function fun) args))))
>
> It seems to work fine, even if it's a bit complicated.

symbol -> list

If you want a symbol to symbol mapping you can use
`make-symbol' and then `setf' on the `symbol-function' of the
argument symbol - maybe ...

-- 
underground experts united
https://dataswamp.org/~incal

Re: How to create a higher order function?

[Emanuel Berg via Users list for the GNU Emacs text editor]

Or `cl-gensym' and `funcall' as in ...

(require 'cl-lib)

(defun add-one-shot-hook (hook fun)
  (let ((name (cl-gensym)))
    (setf (symbol-function name)
          (lambda (&rest _)
            (remove-hook hook name)
            (funcall fun) ))
    (add-hook hook name) ))

Looks like something from Pascal J. B. -
maybe he wrote it ...

-- 
underground experts united
https://dataswamp.org/~incal

Re: How to create a higher order function?

[Leo Butler]

Marcin Borkowski <mbork@mbork.pl> writes:

> Hi all,
>
> assume that I want a function which "inverts" the meaning of a given
> predicate.  Something that would turn `<' into `>=' etc.
>
> So, here's a better (at least: working) version (B):
>
> ;;; -*- lexical-binding: nil; -*-
> (defun negate (fun)
>   "Return a function returning the logical opposite of FUN."
>   `(lambda (&rest args)
>     (not (apply ,(symbol-function fun) args))))

I think negate would need to be a defmacro in this case, not a defun.
This construction, as it is, would return a list, not a lambda function.

 `(foo ,X bar) is equivalent to (list 'foo X 'bar).

>
>
> It seems to work fine, even if it's a bit complicated.
>
> Of course, the more obvious way to do it is to use the first definition
> under lexical scope (C):
>
> ;;; -*- lexical-binding: t; -*-
> (defun negate (fun)
>   "Try to return a function returning the logical opposite of FUN."
>   (lambda (&rest args)
>     (not (apply fun args))))

Obviously C is better.

Q: I see you use mu4e. How do you get the code highlighted? (in my Gnus,
the background is distinctive blue).

Leo

Re: How to create a higher order function?

[Emanuel Berg via Users list for the GNU Emacs text editor]

Leo Butler wrote:

>> It seems to work fine, even if it's a bit complicated.
>>
>> Of course, the more obvious way to do it is to use the
>> first definition under lexical scope (C):
>>
>> ;;; -*- lexical-binding: t; -*-
>> (defun negate (fun)
>>   "Try to return a function returning the logical opposite of FUN."
>>   (lambda (&rest args)
>>     (not (apply fun args))))
>
> Obviously C is better.

Let's RETURN to the POINT(ER): you can do it with C ...

> Q: I see you use mu4e. How do you get the code highlighted?
> (in my Gnus, the background is distinctive blue).

Yep, looks baaad!

The face is `mm-uu-extract', one can disable it that way.

Do `gnus-summary-show-article' with C-u to see that it looks
like this:

--8<---------------cut here---------------start------------->8---
;;; -*- lexical-binding: nil; -*-
(defun negate (fun)
  "Try to return a function returning the logical opposite of FUN."
  (lambda (&rest args)
    (not (apply fun args))))
--8<---------------cut here---------------end--------------->8---

-- 
underground experts united
https://dataswamp.org/~incal

Re: mm-uu-extract was: How to create a higher order function?

[Leo Butler]

Emanuel Berg via Users list for the GNU Emacs text editor
<help-gnu-emacs@gnu.org> writes:

> Leo Butler wrote:
>
>> Q: I see you use mu4e. How do you get the code highlighted?
>> (in my Gnus, the background is distinctive blue).
>
> Yep, looks baaad!
>
> The face is `mm-uu-extract', one can disable it that way.
>
> Do `gnus-summary-show-article' with C-u to see that it looks
> like this:
>
> --8<---------------cut here---------------start------------->8---
> ;;; -*- lexical-binding: nil; -*-
> (defun negate (fun)
>   "Try to return a function returning the logical opposite of FUN."
>   (lambda (&rest args)
>     (not (apply fun args))))
> --8<---------------cut here---------------end--------------->8---

Thanks. Funny, Gnus used to show those markers, but it no longer
does...which seems like a regression to me.

Anyhow, what is the function/key sequence that inserts those markers
into a Message buffer?

TIA,
Leo

Re: mm-uu-extract was: How to create a higher order function?

[Emanuel Berg via Users list for the GNU Emacs text editor]

Leo Butler wrote:

>> ;;; -*- lexical-binding: nil; -*-
>> (defun negate (fun)
>>   "Try to return a function returning the logical opposite of FUN."
>>   (lambda (&rest args)
>>     (not (apply fun args))))
>
> Thanks. Funny, Gnus used to show those markers, but it no
> longer does...which seems like a regression to me.

Looks immature (leet backfire) and don't serve any real or
clear purpose to me.

> Anyhow, what is the function/key sequence that inserts those
> markers into a Message buffer?

Try this:

(defun gnus-article-wash-more ()
  (interactive)
   (article-translate-strings
    '(("Skickades från E-post för Windows 10"       "")
      ("Sendt fra min iPad"                         "")
      ("Skickat från Yahoo Mail för iPhone"         "")
      ("--8<---------------cut"                     "")
      ("here---------------start------------->8---" "")
      ("here---------------end--------------->8---" "") ))
   (gnus-article-strip-multiple-blank-lines) )

;; (setq gnus-article-prepare-hook nil)
(defun gnus-article-prepare-hook-f ()
  (gnus-with-article-buffer
    (gnus-article-wash-more) ))
(add-hook 'gnus-article-prepare-hook #'gnus-article-prepare-hook-f)

-- 
underground experts united
https://dataswamp.org/~incal

Re: mm-uu-extract was: How to create a higher order function?

[Marcin Borkowski]

On 2021-09-21, at 22:31, Leo Butler <leo.butler@umanitoba.ca> wrote:

> Emanuel Berg via Users list for the GNU Emacs text editor
> <help-gnu-emacs@gnu.org> writes:
>
>> Leo Butler wrote:
>>
>>> Q: I see you use mu4e. How do you get the code highlighted?
>>> (in my Gnus, the background is distinctive blue).
>>
>> Yep, looks baaad!
>>
>> The face is `mm-uu-extract', one can disable it that way.
>>
>> Do `gnus-summary-show-article' with C-u to see that it looks
>> like this:
>>
>> --8<---------------cut here---------------start------------->8---
>> ;;; -*- lexical-binding: nil; -*-
>> (defun negate (fun)
>>   "Try to return a function returning the logical opposite of FUN."
>>   (lambda (&rest args)
>>     (not (apply fun args))))
>> --8<---------------cut here---------------end--------------->8---
>
> Thanks. Funny, Gnus used to show those markers, but it no longer
> does...which seems like a regression to me.
>
> Anyhow, what is the function/key sequence that inserts those markers
> into a Message buffer?

Ah, so that's what you meant.

C-c M-m runs the command message-mark-inserted-region

Best,

-- 
Marcin Borkowski
http://mbork.pl

Re: How to create a higher order function?

[Marcin Borkowski]

On 2021-09-21, at 13:27, Leo Butler <leo.butler@umanitoba.ca> wrote:

> Marcin Borkowski <mbork@mbork.pl> writes:
>
>> Hi all,
>>
>> assume that I want a function which "inverts" the meaning of a given
>> predicate.  Something that would turn `<' into `>=' etc.
>>
>> So, here's a better (at least: working) version (B):
>>
>> ;;; -*- lexical-binding: nil; -*-
>> (defun negate (fun)
>>   "Return a function returning the logical opposite of FUN."
>>   `(lambda (&rest args)
>>     (not (apply ,(symbol-function fun) args))))
>
> I think negate would need to be a defmacro in this case, not a defun.

Maybe.  I'm not really convinced.

> This construction, as it is, would return a list, not a lambda function.
>
>  `(foo ,X bar) is equivalent to (list 'foo X 'bar).
>
>>
>>
>> It seems to work fine, even if it's a bit complicated.
>>
>> Of course, the more obvious way to do it is to use the first definition
>> under lexical scope (C):
>>
>> ;;; -*- lexical-binding: t; -*-
>> (defun negate (fun)
>>   "Try to return a function returning the logical opposite of FUN."
>>   (lambda (&rest args)
>>     (not (apply fun args))))
>
> Obviously C is better.
>
> Q: I see you use mu4e. How do you get the code highlighted? (in my Gnus,
> the background is distinctive blue).

??? I don't do anything in particular apart from using a pretty old
version of mu/mu4e.  And code is not highlighted in my emails.

Best,

-- 
Marcin Borkowski
http://mbork.pl

RE: [External] : How to create a higher order function?

[Drew Adams]

> My question is: which of (B) and (C) is better?  (C) is definitely
> simpler, and with lexical scope becoming the default (perhaps at some
> point in time in the future) or the recommended (even now, I guess)
> setting, is probably the way to go.  But maybe (B) is better under some
> circumstances?  Is it faster?  (I don't think so, and my simple
> benchmarks were inconclusive.)  Is it more memory-efficient?  (Could
> be.)  Does it have any other advantage?

(C) is definitely better, in general.

(That's really the takeaway answer.)

The lexical environment, i.e., the one where
the function is defined, is the environment you
generally want for the function - it's, well,
the environment that _defines_ the function.

Generally speaking that's the only proper
environment for defining the function.  Of
course, this is Lisp, and nothing says you're
obliged to use the "proper", static, definition
of anything.
___

Two counter cases to (C), of sorts:

1. If you specifically want to allow other code
to be able to _change the behavior_ of the
function by simply providing an arbitrary
runtime value for some variable, then you might
want to dynamically bind that variable.

2. If a free variable in the function body is
never used _as a variable_, so that only its
_value_ at the point of the function definition
is needed, then you can do what you did, which
is to just substitute the value for the var.

This means there's no carrying around that var
binding in the closure, no need to look up its
value, etc.  But see next: that generally
doesn't mean a benefit in performance - the
contrary.

Wrt #2: Be aware that, at least with current
Elisp and its byte-compiler, #2 essentially
_loses_ the function definition at the point
where it's defined.  Instead, it substitutes
a _list_, a constructed lambda form, for the
function.  It's only later, in some other
context/environment, that that list can get
interpreted as a function.

Among other things, this means that that
representation of a function can't be compiled
as a function.  It can only be interpreted
(any number of times, in any contexts).

It can't in any way be treated or understood
by Emacs as a _function_ - it's just a list
with a lambda form, until it's eval'd.  Put
differently, it's just a _mention_ till it's
explicitly _used_.  It's a template function
definition, to be filled in and interpreted
as a function wherever and whenever.

On the other hand, for better and worse, it
can be treated as a list (it is one).  IOW,
instead of a function, you have essentially
the _name_, a representation, of a function.
If you want to do things to or with such a
representation, you can.  But this is rare.
___

Wrt the advantages and uses of dynamic binding
in the context of _Emacs_ - i.e., the reason
why Emacs has, and should continue to have,
dynamic binding (along with lexical binding),
see RMS's arguments here:

https://www.gnu.org/software/emacs/emacs-paper.html#SEC17

https://www.gnu.org/software/emacs/emacs-paper.html#SEC18

Those arguments are as valid today (& tomorrow)
as they were when written in 1981.

Wrt the uses and behavior of dynamic & lexical
binding in _Lisp_ (not particular to Emacs), I
recommend reading the relevant parts of "Common
Lisp The Language".

https://www.cs.cmu.edu/Groups/AI/html/cltl/clm/node43.html

Here's my SO post on reading that section in
relation to Emacs:

https://stackoverflow.com/a/7135315/729907

Re: [External] : How to create a higher order function?

[Marcin Borkowski]

On 2021-09-21, at 17:49, Drew Adams <drew.adams@oracle.com> wrote:

>> My question is: which of (B) and (C) is better?  (C) is definitely
>> simpler, and with lexical scope becoming the default (perhaps at some
>> point in time in the future) or the recommended (even now, I guess)
>> setting, is probably the way to go.  But maybe (B) is better under some
>> circumstances?  Is it faster?  (I don't think so, and my simple
>> benchmarks were inconclusive.)  Is it more memory-efficient?  (Could
>> be.)  Does it have any other advantage?
>
> (C) is definitely better, in general.
>
> (That's really the takeaway answer.)
>
> The lexical environment, i.e., the one where
> the function is defined, is the environment you
> generally want for the function - it's, well,
> the environment that _defines_ the function.
>
> Generally speaking that's the only proper
> environment for defining the function.  Of
> course, this is Lisp, and nothing says you're
> obliged to use the "proper", static, definition
> of anything.
> ___
>
> Two counter cases to (C), of sorts:
>
> 1. If you specifically want to allow other code
> to be able to _change the behavior_ of the
> function by simply providing an arbitrary
> runtime value for some variable, then you might
> want to dynamically bind that variable.
>
> 2. If a free variable in the function body is
> never used _as a variable_, so that only its
> _value_ at the point of the function definition
> is needed, then you can do what you did, which
> is to just substitute the value for the var.
>
> This means there's no carrying around that var
> binding in the closure, no need to look up its
> value, etc.  But see next: that generally
> doesn't mean a benefit in performance - the
> contrary.
>
> Wrt #2: Be aware that, at least with current
> Elisp and its byte-compiler, #2 essentially
> _loses_ the function definition at the point
> where it's defined.  Instead, it substitutes
> a _list_, a constructed lambda form, for the
> function.  It's only later, in some other
> context/environment, that that list can get
> interpreted as a function.
>
> Among other things, this means that that
> representation of a function can't be compiled
> as a function.  It can only be interpreted
> (any number of times, in any contexts).
>
> It can't in any way be treated or understood
> by Emacs as a _function_ - it's just a list
> with a lambda form, until it's eval'd.  Put
> differently, it's just a _mention_ till it's
> explicitly _used_.  It's a template function
> definition, to be filled in and interpreted
> as a function wherever and whenever.
>
> On the other hand, for better and worse, it
> can be treated as a list (it is one).  IOW,
> instead of a function, you have essentially
> the _name_, a representation, of a function.
> If you want to do things to or with such a
> representation, you can.  But this is rare.
> ___
>
> Wrt the advantages and uses of dynamic binding
> in the context of _Emacs_ - i.e., the reason
> why Emacs has, and should continue to have,
> dynamic binding (along with lexical binding),
> see RMS's arguments here:
>
> https://www.gnu.org/software/emacs/emacs-paper.html#SEC17
>
> https://www.gnu.org/software/emacs/emacs-paper.html#SEC18

Yes, I am aware of that, thanks!

> Those arguments are as valid today (& tomorrow)
> as they were when written in 1981.
>
> Wrt the uses and behavior of dynamic & lexical
> binding in _Lisp_ (not particular to Emacs), I
> recommend reading the relevant parts of "Common
> Lisp The Language".
>
> https://www.cs.cmu.edu/Groups/AI/html/cltl/clm/node43.html
>
> Here's my SO post on reading that section in
> relation to Emacs:
>
> https://stackoverflow.com/a/7135315/729907

Thanks, I'll look into those, too!

Best,

-- 
Marcin Borkowski
http://mbork.pl

Re: [External] : How to create a higher order function?

[Emanuel Berg via Users list for the GNU Emacs text editor]

Drew Adams wrote:

> Wrt the advantages and uses of dynamic binding in the
> context of _Emacs_ - i.e., the reason why Emacs has, and
> should continue to have, dynamic binding (along with
> lexical binding), see RMS's arguments here:
>
> https://www.gnu.org/software/emacs/emacs-paper.html#SEC17
> https://www.gnu.org/software/emacs/emacs-paper.html#SEC18

If that's the "options" argument I don't see what's so
dynamic/special about that idea, to me they are global
variables rather like any global variable found in any
programming language ...

And BTW that idea is very basic so perhaps nothing to go
around hitting the big drum about :)

The only use-case I can see for dynamic/special variables is
to set value(s) to something different in a limited scope when
this can't practically be done because these values are not
settable thru function parameters. E.g. what `let' does under
dynamic binding (i.e., `lexical-binding' is nil). Maybe for
experimental/testing purposes ...

However as this is the rare case (if it happens often, the
functions should be changed to include such parameters, rather
- perhaps optional parameters and/or with keys) which is why
think static/lexical scope should be default, `let' should
_always_ be static/lexical and a/the dedicated let should
always be used if and when dynamic/special scope is
desired ...

-- 
underground experts united
https://dataswamp.org/~incal

RE: [External] : How to create a higher order function?

[Drew Adams]

[-- Attachment #1: Type: text/plain, Size: 1006 bytes --]

> > Wrt the advantages and uses of dynamic binding in the
> > context of _Emacs_ - i.e., the reason why Emacs has, and
> > should continue to have, dynamic binding (along with
> > lexical binding), see RMS's arguments here:
> >
> > https://www.gnu.org/software/emacs/emacs-paper.html#SEC17
> > https://www.gnu.org/software/emacs/emacs-paper.html#SEC18
> 
> If that's the "options" argument I don't see what's so
> dynamic/special about that idea, to me they are global
> variables rather like any global variable found in any
> programming language ...

You might want to read it again.  It's not
complicated.  And it's not trivial.

You might also want to check out "Common Lisp
The Language".  Unlike the CL "hyperspec", it
presents (also) the design of the language
and the logic behind it.

RMS's arguments are specifically for users
of an editing environment such as Emacs.
They go beyond the general arguments for
providing dynamic binding (in addition to
lexical) in Lisp.

[-- Attachment #2: winmail.dat --]
[-- Type: application/ms-tnef, Size: 13848 bytes --]

Re: [External] : How to create a higher order function?

[Emanuel Berg via Users list for the GNU Emacs text editor]

Drew Adams wrote:

> RMS's arguments are specifically for users of an editing
> environment such as Emacs. They go beyond the general
> arguments for providing dynamic binding (in addition to
> lexical) in Lisp.

Again, if it is the options argument I agree that's a good
method to do it however I would formulate that as "we
implement options by way of global variables", I don't see how
dynamic/special vs static/lexical really applies?

I also would not hit the big drum over that since it is such
a basic idea which everyone understands. It is like putting
a nail under a spotlight on a huge pedestal in a museum on the
industrial revolution. Because everyone already understands
the nail - at least conceptually, so there's no reason to
do it.

And, while I don't know who the "RSM" of the industrial
revolution was, I'm still 100% confident he was very familiar
with the nail.

-- 
underground experts united
https://dataswamp.org/~incal

Re: [External] : How to create a higher order function?

[Marcin Borkowski]

On 2021-09-24, at 19:11, Emanuel Berg via Users list for the GNU Emacs text editor <help-gnu-emacs@gnu.org> wrote:

> Drew Adams wrote:
>
>> RMS's arguments are specifically for users of an editing
>> environment such as Emacs. They go beyond the general
>> arguments for providing dynamic binding (in addition to
>> lexical) in Lisp.
>
> Again, if it is the options argument I agree that's a good
> method to do it however I would formulate that as "we
> implement options by way of global variables", I don't see how
> dynamic/special vs static/lexical really applies?

IIUC, if these (global) variables were lexically bound, the whole
idea wouldn't work.  Global/local has nothing to do with it.

Best,

-- 
Marcin Borkowski
http://mbork.pl

Re: [External] : How to create a higher order function?

[Emanuel Berg via Users list for the GNU Emacs text editor]

Marcin Borkowski wrote:

>>> RMS's arguments are specifically for users of an editing
>>> environment such as Emacs. They go beyond the general
>>> arguments for providing dynamic binding (in addition to
>>> lexical) in Lisp.
>>
>> Again, if it is the options argument I agree that's a good
>> method to do it however I would formulate that as "we
>> implement options by way of global variables", I don't see
>> how dynamic/special vs static/lexical really applies?
>
> IIUC, if these (global) variables were lexically bound, the
> whole idea wouldn't work.

Global variables are present in many languages that do not
mention dynamic/special vs static/lexical.

> Global/local has nothing to do with it.

If they were local, how would people set them? Also if they
were local, how would developers add e.g. another function
that used them?

-- 
underground experts united
https://dataswamp.org/~incal

Re: [External] : How to create a higher order function?

[Marcin Borkowski]

On 2021-09-26, at 01:48, Emanuel Berg via Users list for the GNU Emacs text editor <help-gnu-emacs@gnu.org> wrote:

> Marcin Borkowski wrote:
>
>>>> RMS's arguments are specifically for users of an editing
>>>> environment such as Emacs. They go beyond the general
>>>> arguments for providing dynamic binding (in addition to
>>>> lexical) in Lisp.
>>>
>>> Again, if it is the options argument I agree that's a good
>>> method to do it however I would formulate that as "we
>>> implement options by way of global variables", I don't see
>>> how dynamic/special vs static/lexical really applies?
>>
>> IIUC, if these (global) variables were lexically bound, the
>> whole idea wouldn't work.
>
> Global variables are present in many languages that do not
> mention dynamic/special vs static/lexical.
>
>> Global/local has nothing to do with it.
>
> If they were local, how would people set them? Also if they
> were local, how would developers add e.g. another function
> that used them?

Again: of course they couldn't be local.  But if they were global
static, the whole "option setting" wouldn't work.  Simple as that.  If
you want to see for yourself, create a lexically bound global "option"
and try to make it work.

Best,

-- 
Marcin Borkowski
http://mbork.pl

Re: [External] : How to create a higher order function?

[Emanuel Berg via Users list for the GNU Emacs text editor]

Marcin Borkowski wrote:

>> If they were local, how would people set them? Also if they
>> were local, how would developers add e.g. another function
>> that used them?
>
> Again: of course they couldn't be local. But if they were
> global static, the whole "option setting" wouldn't work.

Why not? Many programming languages with no notion of
dynamic/special vs static/lexical scope has global variables
with no problem using them to implement user options ...

-- 
underground experts united
https://dataswamp.org/~incal

Re: [External] : How to create a higher order function?

[Marcin Borkowski]

On 2021-09-28, at 03:41, Emanuel Berg via Users list for the GNU Emacs text editor <help-gnu-emacs@gnu.org> wrote:

> Marcin Borkowski wrote:
>
>>> If they were local, how would people set them? Also if they
>>> were local, how would developers add e.g. another function
>>> that used them?
>>
>> Again: of course they couldn't be local. But if they were
>> global static, the whole "option setting" wouldn't work.
>
> Why not? Many programming languages with no notion of
> dynamic/special vs static/lexical scope has global variables
> with no problem using them to implement user options ...

Try this code.  Make sure you set `lexical-binding' to t, e.g., by
saving it to a file and visiting it again or something.

--8<---------------cut here---------------start------------->8---
;;; -*- lexical-binding: t; -*-

(setq global-lexical 24)

(defun lexical-test ()
  (message "%s" global-lexical))

(setq global-lexical 17)

(lexical-test)

(let ((global-lexical 12))
  (lexical-test))


(defvar global-dynamic 24)

(defun dynamic-test ()
  (message "%s" global-dynamic))

(dynamic-test)

(setq global-dynamic 17)

(dynamic-test)

(let ((global-dynamic 12))
  (dynamic-test))
--8<---------------cut here---------------end--------------->8---

Can you spot the difference?  Can you see why the dynamic one is better
suited for options, i.e., you can temporarily set it with `let' and it
Just Works™?

Best,

-- 
Marcin Borkowski
http://mbork.pl

Re: [External] : How to create a higher order function?

[Emanuel Berg via Users list for the GNU Emacs text editor]

Marcin Borkowski wrote:

> Try this code. Make sure you set `lexical-binding' to t,
> e.g., by saving it to a file and visiting it again
> or something.
>
> ;;; -*- lexical-binding: t; -*-
>
> (setq global-lexical 24)
>
> (defun lexical-test ()
>   (message "%s" global-lexical))
>
> (setq global-lexical 17)
>
> (lexical-test)
>
> (let ((global-lexical 12))
>   (lexical-test))
>
> (defvar global-dynamic 24)
>
> (defun dynamic-test ()
>   (message "%s" global-dynamic))
>
> (dynamic-test)
>
> (setq global-dynamic 17)
>
> (dynamic-test)
>
> (let ((global-dynamic 12))
>   (dynamic-test))
>
> Can you spot the difference? Can you see why the dynamic one
> is better suited for options, i.e., you can temporarily set
> it with `let' and it Just Works?

You mean options not from a human user, but an Elisp program
perspective? I.e., different subsystems use other subsystems
with their own set of options?

I guess that is/would be cool, in particular if it would work
in parallel ...

Other than that, what I can see, you have provided the same
old example we have seen many times in slightly different
versions. It shows that lexical binding is less confusing and
less error-prone. It also shows the confusion that arises from
`let' because it not only behaves differently under lexical
binding and dynamic binding, it also behaves differently with
lexically and dynamically bound individual variables ...

Split up the `let' is what I think, into slet, `dlet' (which
already exists), and llet, an alias to `let' (or the other way
around, doesn't matter), which would be the same as our `let'
under lexical binding. Then remove all ugly preprocessor-style
;;; -*- lexical-binding: t; -*- and everyone can just use
whatever they like with the different lets having their (and
only their) behavior in the docstrings, and that behavior
wouldn't change no matter any outside fiddling around.

-- 
underground experts united
https://dataswamp.org/~incal

Re: [External] : How to create a higher order function?

[Marcin Borkowski]

On 2021-09-29, at 06:28, Emanuel Berg via Users list for the GNU Emacs text editor <help-gnu-emacs@gnu.org> wrote:

> Marcin Borkowski wrote:
>
>> Try this code. Make sure you set `lexical-binding' to t,
>> e.g., by saving it to a file and visiting it again
>> or something.
>>
>> ;;; -*- lexical-binding: t; -*-
>>
>> (setq global-lexical 24)
>>
>> (defun lexical-test ()
>>   (message "%s" global-lexical))
>>
>> (setq global-lexical 17)
>>
>> (lexical-test)
>>
>> (let ((global-lexical 12))
>>   (lexical-test))
>>
>> (defvar global-dynamic 24)
>>
>> (defun dynamic-test ()
>>   (message "%s" global-dynamic))
>>
>> (dynamic-test)
>>
>> (setq global-dynamic 17)
>>
>> (dynamic-test)
>>
>> (let ((global-dynamic 12))
>>   (dynamic-test))
>>
>> Can you spot the difference? Can you see why the dynamic one
>> is better suited for options, i.e., you can temporarily set
>> it with `let' and it Just Works?
>
> You mean options not from a human user, but an Elisp program
> perspective? I.e., different subsystems use other subsystems
> with their own set of options?

Not sure if I understand.

By an "option" I mean a variable I (the user) can set, either via `setq'
(or customize) or via `let' (in my code) so that Emacs behavior is
changed (temporarily in the case of `let').

> I guess that is/would be cool, in particular if it would work
> in parallel ...
>
> Other than that, what I can see, you have provided the same
> old example we have seen many times in slightly different
> versions. It shows that lexical binding is less confusing and
> less error-prone. It also shows the confusion that arises from
> `let' because it not only behaves differently under lexical
> binding and dynamic binding, it also behaves differently with
> lexically and dynamically bound individual variables ...

I wouldn't say "less confusing" - both lexical and dynamical binding
serve their purposes, which are different.

> Split up the `let' is what I think, into slet, `dlet' (which
> already exists), and llet, an alias to `let' (or the other way
> around, doesn't matter), which would be the same as our `let'
> under lexical binding. Then remove all ugly preprocessor-style
> ;;; -*- lexical-binding: t; -*- and everyone can just use
> whatever they like with the different lets having their (and
> only their) behavior in the docstrings, and that behavior
> wouldn't change no matter any outside fiddling around.

You mean you could use the same variable sometimes with `slet',
sometimes with `dlet'?  I'm not sure if that is easy or difficult to
implement.

Best,

-- 
Marcin Borkowski
http://mbork.pl

Re: [External] : How to create a higher order function?

[Emanuel Berg via Users list for the GNU Emacs text editor]

Marcin Borkowski wrote:

> By an "option" I mean a variable I (the user) can set,
> either via `setq' (or customize) or via `let' (in my code)
> so that Emacs behavior is changed (temporarily in the case
> of `let').

Yes, but are people really using `let' to set options? Well,
okay, maybe they do ...

>> Split up the `let' is what I think, into slet, `dlet'
>> (which already exists), and llet, an alias to `let' (or the
>> other way around, doesn't matter), which would be the same
>> as our `let' under lexical binding. Then remove all ugly
>> preprocessor-style ;;; -*- lexical-binding: t; -*- and
>> everyone can just use whatever they like with the different
>> lets having their (and only their) behavior in the
>> docstrings, and that behavior wouldn't change no matter any
>> outside fiddling around.
>
> You mean you could use the same variable sometimes with
> `slet', sometimes with `dlet'?

Yes, with `dlet' and `let' it would/could be that way - with
"slet" it would _always_ be static/lexical so then it would be
"same variable name" rather ...

> I'm not sure if that is easy or difficult to implement.

I'm sure it wasn't easy but Mr. Monnier already have all three
in his vault :)

-- 
underground experts united
https://dataswamp.org/~incal

dlet/let/slet (was: Re: [External] : How to create a higher order function?)

[Emanuel Berg via Users list for the GNU Emacs text editor]

>> You mean you could use the same variable sometimes with
>> `slet', sometimes with `dlet'?
>
> Yes, with `dlet' and `let' it would/could be that way - with
> "slet" it would _always_ be static/lexical so then it would
> be "same variable name" rather ...

That's it, that's what we should have!

dlet - special/dynamic, always

let  - special/dynamic on existing such variables, otherwise
       static/lexical

slet - static/lexical, always

So dlet would be the same as `let' today if `lexical-binding'
is nil, which is the default.

And let would be the same as `let' today if `lexical-binding'
is t, which is recommended for everyday Elisp use.

Then there would be no need for ;;; -*- lexical-binding: t -*-
and no confusing alternating behavior for `let' and if you
wanted to explain/learn about dlet, let, and slet one would
just go to their individual docstrings, i.e. no need for CS
theory with respect to scoping or anything, they would just be
three Lisp functions (or special forms) among all other ...

-- 
underground experts united
https://dataswamp.org/~incal

Re: How to create a higher order function?

[Stefan Monnier via Users list for the GNU Emacs text editor]

> (defun negate (fun)
>   "Return a function returning the logical opposite of FUN."
>   `(lambda (&rest args)
>     (not (apply ,(symbol-function fun) args))))

This will fail when `fun` is not a function name but a (lambda ...).
It will also fail when `fun` is a function alias (hence (symbol-function
fun) returns the name of another function).
It will also fail when `fun` is a symbol whose content is an interpreted
lexically scoped function (hence (symbol-function fun) returns
something of the form (closure ...)).
It will also fail when `fun` is an autoloaded function that's not yet
loaded (hence (symbol-function fun) returns something of the form
(autoload ...)).

I think what you meant was

    (defun negate (fun)
      "Return a function returning the logical opposite of FUN."
      `(lambda (&rest args)
        (not (apply ',fun args))))

> But maybe (B) is better under some circumstances?

I'm hoping B can be banned at some point in the future ;-)

> Is it faster?

I don't think so.  Currently closures constructed in lexical-binding
mode are not super efficient (back in Emacs-24, the focus was on
providing lexical-scoping with clean semantics and without breaking or
slowing down dynamic scoping.  Efficiency of lexical-binding was
secondary, and we haven't really attacked that yet).

More specifically, constructing a closure will allocate two new vectors
(whereas the ideal would be 1 vector), and one of them holds a lot more
than the set of free variables (it also holds all the constants used in
the code, such as the names of all the functions it calls).

But the above construction of a (lambda ...) list to mimic a true
closure isn't very efficient either: if I count correctly it will need
to allocate 9 new cons cells.

Which is currently faster, I don't know.  But if the construction of the
true closure is slower, at least there's the theoretical possibility to
invest implementation efforts into making it more efficient by using
a more conventional closure representation.

As for efficiency when calling the constructed function, I'm pretty sure
that C will be significantly faster (tho you'll have to call them many
times to see the difference ;-), but I suspect that in practice for this
specific example the difference may not be very large because most of
the time will be spent processing the `&rest` (i.e. converting the args
(stored in a vector) into a list) and the `apply` (i.e. converting the
list back to a vector), and the benchmark will likely show that you're
spending a lot of time in the GC (because of the need to recover all
those temporary lists generated by the `&rest`).

So to better see the effect you might want to use something like:

    (defun negate (fun)
      "Return a function returning the logical opposite of FUN."
      (lambda (arg1 arg2)
        (not (funcall fun arg1 arg2))))

where I'd expect the (C) version to be faster by a quite
significant margin (like more than a factor 2).


        Stefan

Re: How to create a higher order function?

[Emanuel Berg via Users list for the GNU Emacs text editor]

Stefan Monnier via Users list for the GNU Emacs text editor wrote:

> I think what you meant was
>
>     (defun negate (fun)
>       "Return a function returning the logical opposite of FUN."
>       `(lambda (&rest args)
>         (not (apply ',fun args))))

Awesome docstring for `backquote' BTW (the complete set of
examples), make it even better by removing some whitespace
prior to the equals sign chars so the semicolon chars (the
Elisp comment initiators) can be aligned as well.

But didn't you always say one shouldn't quote lambdas?

Is this the exception that confirms the rule?

> I don't think so. Currently closures constructed in
> lexical-binding mode are not super efficient (back in
> Emacs-24, the focus was on providing lexical-scoping with
> clean semantics and without breaking or slowing down dynamic
> scoping. Efficiency of lexical-binding was secondary, and we
> haven't really attacked that yet).

Hm ... maybe this can be visualized somehow or
expressed diagramically.

What about the supposedly and maybe rightly so "industrial
strength" Lisps, how did they do it? (Yuk. That term isn't
appealing/useful when it comes to programming IMO.)

>     (defun negate (fun)
>       "Return a function returning the logical opposite of FUN."
>       (lambda (arg1 arg2)
>         (not (funcall fun arg1 arg2))))
>
> where I'd expect the (C) version to be faster by a quite
> significant margin (like more than a factor 2).

See? That's what I meant when I said `funcall' should be used.

Keep it uptime ...

-- 
underground experts united
https://dataswamp.org/~incal

Re: How to create a higher order function?

[Emanuel Berg via Users list for the GNU Emacs text editor]

> (defun negate (fun)
>   "Return a function returning the logical opposite of FUN."
>   `(lambda (&rest args)
>     (not (apply ,(symbol-function fun) args))))

What does it really mean to be human? I mean a function?
What makes one list a function and the other list just
a list?

That the list begins with certain words like `defun' or
`lambda'. But then quoting that list - what does that mean -
that makes the function-defining functions just data instead?

Actually both defun and lambdas are Lisp macros ...

-- 
underground experts united
https://dataswamp.org/~incal

Re: How to create a higher order function?

[Marcin Borkowski]

On 2021-09-22, at 21:03, Stefan Monnier via Users list for the GNU Emacs text editor <help-gnu-emacs@gnu.org> wrote:

> I think what you meant was
>
>     (defun negate (fun)
>       "Return a function returning the logical opposite of FUN."
>       `(lambda (&rest args)
>         (not (apply ',fun args))))

Interesting, I didn't think of this.

>> But maybe (B) is better under some circumstances?
>
> I'm hoping B can be banned at some point in the future ;-)

This is great news, since I read it "don't bother with it". :-)

>> Is it faster?
>
> I don't think so.  Currently closures constructed in lexical-binding
> mode are not super efficient (back in Emacs-24, the focus was on
> providing lexical-scoping with clean semantics and without breaking or
> slowing down dynamic scoping.  Efficiency of lexical-binding was
> secondary, and we haven't really attacked that yet).
>
> More specifically, constructing a closure will allocate two new vectors
> (whereas the ideal would be 1 vector), and one of them holds a lot more
> than the set of free variables (it also holds all the constants used in
> the code, such as the names of all the functions it calls).
>
> But the above construction of a (lambda ...) list to mimic a true
> closure isn't very efficient either: if I count correctly it will need
> to allocate 9 new cons cells.
>
> Which is currently faster, I don't know.  But if the construction of the
> true closure is slower, at least there's the theoretical possibility to
> invest implementation efforts into making it more efficient by using
> a more conventional closure representation.
>
> As for efficiency when calling the constructed function, I'm pretty sure
> that C will be significantly faster (tho you'll have to call them many
> times to see the difference ;-), but I suspect that in practice for this
> specific example the difference may not be very large because most of
> the time will be spent processing the `&rest` (i.e. converting the args
> (stored in a vector) into a list) and the `apply` (i.e. converting the
> list back to a vector), and the benchmark will likely show that you're
> spending a lot of time in the GC (because of the need to recover all
> those temporary lists generated by the `&rest`).
>
> So to better see the effect you might want to use something like:
>
>     (defun negate (fun)
>       "Return a function returning the logical opposite of FUN."
>       (lambda (arg1 arg2)
>         (not (funcall fun arg1 arg2))))
>
> where I'd expect the (C) version to be faster by a quite
> significant margin (like more than a factor 2).

Thanks for your detailed answer!  As usual, it turns out that there's
always so much to learn!

Best,

-- 
Marcin Borkowski
http://mbork.pl