* thunk.el: Document that thunk-force == funcall?
@ 2020-11-17 15:17 Michael Heerdegen
2020-11-17 17:08 ` Tomas Hlavaty
` (2 more replies)
0 siblings, 3 replies; 27+ messages in thread
From: Michael Heerdegen @ 2020-11-17 15:17 UTC (permalink / raw)
To: Emacs Development; +Cc: Nicolas Petton, Stefan Monnier
Hello,
`thunk-force' is equivalent to `funcall' - thunks are functions. I
wonder if we could/should officially document that fact?
The background: because thunks are functions, one is allowed (and it is
useful) to directly pass them to higher order functions e.g. as a test
predicate, or bind them using `cl-flet', or `cl-letf' to a
`symbol-function' place. Currently, the official solution would require
to use a lambda wrapper.
I have encountered this requirement several times (and using `thunk-let'
was not always the solution) - thus my question.
TIA,
Michael.
^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: thunk.el: Document that thunk-force == funcall?
2020-11-17 15:17 thunk.el: Document that thunk-force == funcall? Michael Heerdegen
@ 2020-11-17 17:08 ` Tomas Hlavaty
2020-11-17 17:38 ` Michael Heerdegen
2020-11-18 9:04 ` Alfred M. Szmidt
2020-11-17 17:32 ` Drew Adams
2020-11-17 21:51 ` Stefan Monnier
2 siblings, 2 replies; 27+ messages in thread
From: Tomas Hlavaty @ 2020-11-17 17:08 UTC (permalink / raw)
To: Emacs Development
On Tue 17 Nov 2020 at 16:17, Michael Heerdegen <michael_heerdegen@web.de> wrote:
> thunks are functions.
thunks take no arguments (at least the definition i know)
> The background: because thunks are functions, one is allowed (and it is
> useful) to directly pass them to higher order functions e.g. as a test
> predicate,
a test predicate usually takes at least one argument
thus a thunk cannot be used as a test predicate
^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: thunk.el: Document that thunk-force == funcall?
2020-11-17 17:08 ` Tomas Hlavaty
@ 2020-11-17 17:38 ` Michael Heerdegen
2020-11-17 18:09 ` Tomas Hlavaty
2020-11-18 9:04 ` Alfred M. Szmidt
1 sibling, 1 reply; 27+ messages in thread
From: Michael Heerdegen @ 2020-11-17 17:38 UTC (permalink / raw)
To: emacs-devel
Tomas Hlavaty <tom@logand.com> writes:
> a test predicate usually takes at least one argument
I do not speak about tests involving an argument but rather things like
`something-is-the-case-but-it-takes-long-to-find-out-p' that are often
comfortably defined as a thunk if the result may matter at several
places.
Michael.
^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: thunk.el: Document that thunk-force == funcall?
2020-11-17 17:38 ` Michael Heerdegen
@ 2020-11-17 18:09 ` Tomas Hlavaty
2020-11-17 21:07 ` Michael Heerdegen
0 siblings, 1 reply; 27+ messages in thread
From: Tomas Hlavaty @ 2020-11-17 18:09 UTC (permalink / raw)
To: emacs-devel
On Tue 17 Nov 2020 at 18:38, Michael Heerdegen <michael_heerdegen@web.de> wrote:
>> a test predicate usually takes at least one argument
>
> I do not speak about tests involving an argument but rather things like
> `something-is-the-case-but-it-takes-long-to-find-out-p' that are often
> comfortably defined as a thunk if the result may matter at several
> places.
interesting
i am struggling to imagine a use-case
why would you delay such boolean value in a thunk?
to avoid computing the value in case it is not used at all?
when something-is-the-case how can it take long to find out? why it
cannot be substituted with t at think-time?
^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: thunk.el: Document that thunk-force == funcall?
2020-11-17 18:09 ` Tomas Hlavaty
@ 2020-11-17 21:07 ` Michael Heerdegen
2020-11-17 22:42 ` Tomas Hlavaty
0 siblings, 1 reply; 27+ messages in thread
From: Michael Heerdegen @ 2020-11-17 21:07 UTC (permalink / raw)
To: emacs-devel
Tomas Hlavaty <tom@logand.com> writes:
> why would you delay such boolean value in a thunk?
>
> to avoid computing the value in case it is not used at all?
>
> when something-is-the-case how can it take long to find out? why it
> cannot be substituted with t at think-time?
The result might not be defined at think time. It might depend on "the
environment".
Say the code loops over a list of files or so (not known at think time).
The thunk could "contain" a test that might take long (e.g. something
that might need to look at the file's contents) but the result is
interesting only sometimes, depending on the result of other tests (also
not known at think time).
Michael.
^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: thunk.el: Document that thunk-force == funcall?
2020-11-17 21:07 ` Michael Heerdegen
@ 2020-11-17 22:42 ` Tomas Hlavaty
2020-11-17 23:52 ` Stefan Monnier
0 siblings, 1 reply; 27+ messages in thread
From: Tomas Hlavaty @ 2020-11-17 22:42 UTC (permalink / raw)
To: emacs-devel
On Tue 17 Nov 2020 at 22:07, Michael Heerdegen <michael_heerdegen@web.de> wrote:
> The result might not be defined at think time. It might depend on "the
> environment".
>
> Say the code loops over a list of files or so (not known at think time).
> The thunk could "contain" a test that might take long (e.g. something
> that might need to look at the file's contents) but the result is
> interesting only sometimes, depending on the result of other tests (also
> not known at think time).
do you have a specific example i could learn from?
i use thunks a lot for a kind of minimalist portable streams (sometimes
called generators in lisp), e.g. see
https://logand.com/sw/emacs-pdf/file/emacs-pdf.el.html#l198
i am hoping to learn new use-case or trick here
your use-case seems to be like this:
(dolist (f (directory-files "/tmp/" t))
(when (and (some-other-test-p)
(something-is-the-case-but-it-takes-long-to-find-out-p))
(do-something)))
why does something-is-the-case-but-it-takes-long-to-find-out-p need to
be a thunk? the special form `and' is already lazy and skips computing
something-is-the-case-but-it-takes-long-to-find-out-p when
some-other-test-p is false.
also it is usually better to not depend on implicit environment but to
use function arguments when possible. in this case
something-is-the-case-but-it-takes-long-to-find-out-p should take the f
as arg (it might need to look at the file's contents after all):
(dolist (f (directory-files "/tmp/" t))
(when (and (some-other-test-p f)
(something-is-the-case-but-it-takes-long-to-find-out-p f))
(do-something f)))
one use-case for thunks here would be to make dolist lazy:
(setq lexical-binding t)
(require 'cl)
(defun brook (x)
(etypecase x
(list
(lambda ()
(pop x)))))
(defun brook-count (brook)
(loop
with z = nil
while (setq z (funcall brook))
count z))
(defun filter-brook (fn brook)
(lambda ()
(block yield
(let (z)
(while (setq z (funcall brook))
(when (funcall fn z)
(return-from yield z)))))))
;; now we can for example count regular files as we pull them lazily
(let ((some-other-test-p 'file-regular-p))
(brook-count
(filter-brook some-other-test-p
(brook (directory-files "/tmp/" t)))))
(defun map-brook (fn brook)
(lambda ()
(let ((x (funcall brook)))
(when x
(funcall fn x)))))
(defun brook-collect (brook)
(loop
with z = nil
while (setq z (funcall brook))
collect z))
(defun sha256sum (file)
(with-temp-buffer
(call-process "sha256sum" file (current-buffer))
(buffer-substring-no-properties (point-min) (+ 64 (point-min)))))
;; now we can for example collect sha256sum of each regular file
(let ((some-other-test-p 'file-regular-p)
(something-is-the-case-but-it-takes-long-to-find-out-p 'sha256sum))
(brook-collect
(map-brook
something-is-the-case-but-it-takes-long-to-find-out-p
(filter-brook some-other-test-p
(brook (directory-files "/tmp/" t))))))
;; or display progress during slow computation
(defun progress-brook (brook)
(lambda ()
(let ((z (funcall brook)))
(when z
(message (format "progress-brook: %s" z))
z))))
(let ((some-other-test-p 'file-regular-p)
(something-is-the-case-but-it-takes-long-to-find-out-p 'sha256sum))
(brook-collect
(progress-brook
(map-brook
something-is-the-case-but-it-takes-long-to-find-out-p
(progress-brook
(filter-brook some-other-test-p
(brook (directory-files "/tmp/" t))))))))
i can imagine that we might for example want to show progress also
during computation of sha256sum.
something-is-the-case-but-it-takes-long-to-find-out-p would then return
thunk and be run "step by step" like this:
(defun flat-brook (&rest brooks)
(lambda ()
(block yield
(while brooks
(let ((z (funcall (car brooks))))
(cond
((functionp z) (push z brooks))
(z (return-from yield z))
(t (pop brooks))))))))
(let ((some-other-test-p 'file-regular-p)
(something-is-the-case-but-it-takes-long-to-find-out-p
(lambda (f)
;; i cannot easily show progress during sha256sum here so
;; lets immitate that by showing progress before and after
(let ((pending
(list
(lambda () (message "before sha256sum %s" f))
(lambda ()
(let ((z (sha256sum f)))
(message "sha256sum %s %s" f z)
z))
(lambda () (message "after sha256sum %s" f)))))
(lambda ()
(when pending
(funcall (pop pending))))))))
(brook-collect
(flat-brook
(map-brook
something-is-the-case-but-it-takes-long-to-find-out-p
(filter-brook some-other-test-p
(brook (directory-files "/tmp/" t)))))))
but in which use-case should
something-is-the-case-but-it-takes-long-to-find-out-p be a thunk?
^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: thunk.el: Document that thunk-force == funcall?
2020-11-17 22:42 ` Tomas Hlavaty
@ 2020-11-17 23:52 ` Stefan Monnier
2020-11-18 8:01 ` Tomas Hlavaty
0 siblings, 1 reply; 27+ messages in thread
From: Stefan Monnier @ 2020-11-17 23:52 UTC (permalink / raw)
To: Tomas Hlavaty; +Cc: emacs-devel
> (setq lexical-binding t)
Side node: this usually doesn't do what the author thinks it does.
Stefan
^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: thunk.el: Document that thunk-force == funcall?
2020-11-17 23:52 ` Stefan Monnier
@ 2020-11-18 8:01 ` Tomas Hlavaty
2020-11-18 14:04 ` Stefan Monnier
0 siblings, 1 reply; 27+ messages in thread
From: Tomas Hlavaty @ 2020-11-18 8:01 UTC (permalink / raw)
To: emacs-devel
On Tue 17 Nov 2020 at 18:52, Stefan Monnier <monnier@iro.umontreal.ca> wrote:
>> (setq lexical-binding t)
>
> Side node: this usually doesn't do what the author thinks it does.
what does it do?
how to turn the *scratch* buffer to lexical binding?
for the quick experiment, i evaluated this
(setq lexical-binding t)
in the *scratch* buffer and it did what i wanted, namely allowed me to
use lexical binding by default for the purpose of the quick experiment.
i do not know how to switch the *scratch* buffer to lexical binding
otherwise. without that, closures did not work as i needed.
normally, i write
;;; -*- lexical-binding: t -*-
at the beginning of a buffer.
does this do what i think it does, i.e. switch to lexical binding for
the entire buffer?
^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: thunk.el: Document that thunk-force == funcall?
2020-11-18 8:01 ` Tomas Hlavaty
@ 2020-11-18 14:04 ` Stefan Monnier
2020-11-18 22:19 ` Tomas Hlavaty
0 siblings, 1 reply; 27+ messages in thread
From: Stefan Monnier @ 2020-11-18 14:04 UTC (permalink / raw)
To: Tomas Hlavaty; +Cc: emacs-devel
>>> (setq lexical-binding t)
>> Side node: this usually doesn't do what the author thinks it does.
> what does it do?
It changes the lexical-binding mode of the buffer in which the code
is executed, which means it will affect the code subsequently
read¯oexpanded from that buffer.
If that's inside a file that you're `load`ing, it will change the
lexical-binding of the buffer which happens to be current when the file
is loaded but won't affect the interpretation of the subsequent code in
that file.
> how to turn the *scratch* buffer to lexical binding?
By hitting C-j after the above line, for example ;-)
> ;;; -*- lexical-binding: t -*-
> at the beginning of a buffer.
> does this do what i think it does, i.e. switch to lexical binding for
> the entire buffer?
Yes, that's the recommended way.
Stefan
^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: thunk.el: Document that thunk-force == funcall?
2020-11-18 14:04 ` Stefan Monnier
@ 2020-11-18 22:19 ` Tomas Hlavaty
2020-11-18 22:49 ` Stefan Monnier
0 siblings, 1 reply; 27+ messages in thread
From: Tomas Hlavaty @ 2020-11-18 22:19 UTC (permalink / raw)
To: emacs-devel
On Wed 18 Nov 2020 at 09:04, Stefan Monnier <monnier@iro.umontreal.ca> wrote:
>>>> (setq lexical-binding t)
>>> Side node: this usually doesn't do what the author thinks it does.
>> what does it do?
>
> It changes the lexical-binding mode of the buffer in which the code
> is executed, which means it will affect the code subsequently
> read¯oexpanded from that buffer.
so it did what the author thought it did:-)
> If that's inside a file that you're `load`ing, it will change the
> lexical-binding of the buffer which happens to be current when the file
> is loaded but won't affect the interpretation of the subsequent code in
> that file.
ok, i see why you thought it did not do what the authour thought it did
>> how to turn the *scratch* buffer to lexical binding?
>
> By hitting C-j after the above line, for example ;-)
i just learned about C-j, thanks.
i used eval-defun which i have bound to C-M-x. (strange that it is
called eval-defun but it actually evaluates the top-level sexp, even if
it is not defun)
i don't know how else should have i indicated, that all the code bellow
needs lexical binding.
is there a way to do that without confusing anybody, so that it does not
look like it doesn't do what the author thinks it does?
^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: thunk.el: Document that thunk-force == funcall?
2020-11-18 22:19 ` Tomas Hlavaty
@ 2020-11-18 22:49 ` Stefan Monnier
2020-11-18 23:13 ` Tomas Hlavaty
0 siblings, 1 reply; 27+ messages in thread
From: Stefan Monnier @ 2020-11-18 22:49 UTC (permalink / raw)
To: Tomas Hlavaty; +Cc: emacs-devel
>>>>> (setq lexical-binding t)
>>>> Side node: this usually doesn't do what the author thinks it does.
>>> what does it do?
>> It changes the lexical-binding mode of the buffer in which the code
>> is executed, which means it will affect the code subsequently
>> read¯oexpanded from that buffer.
> so it did what the author thought it did:-)
Good!
>>> how to turn the *scratch* buffer to lexical binding?
>> By hitting C-j after the above line, for example ;-)
> i just learned about C-j, thanks.
*scratch* is actually similar to `ielm`, tho without the prompt and using
"newline" instead of "return" ;-)
> i used eval-defun which i have bound to C-M-x. (strange that it is
> called eval-defun but it actually evaluates the top-level sexp, even if
> it is not defun)
Ah, names!
> I don't know how else should have i indicated, that all the code
> bellow needs lexical binding. Is there a way to do that without
> confusing anybody, so that it does not look like it doesn't do what
> the author thinks it does?
There's no really satisfactory way to do it, but I use
;; -*- lexical-binding:t -*-
-- Stefan
^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: thunk.el: Document that thunk-force == funcall?
2020-11-18 22:49 ` Stefan Monnier
@ 2020-11-18 23:13 ` Tomas Hlavaty
2020-11-18 23:40 ` Stephen Leake
0 siblings, 1 reply; 27+ messages in thread
From: Tomas Hlavaty @ 2020-11-18 23:13 UTC (permalink / raw)
To: emacs-devel
On Wed 18 Nov 2020 at 17:49, Stefan Monnier <monnier@iro.umontreal.ca> wrote:
> *scratch* is actually similar to `ielm`, tho without the prompt and using
> "newline" instead of "return" ;-)
interesting
i haven't used ielm so far
>> I don't know how else should have i indicated, that all the code
>> bellow needs lexical binding. Is there a way to do that without
>> confusing anybody, so that it does not look like it doesn't do what
>> the author thinks it does?
>
> There's no really satisfactory way to do it, but I use
>
> ;; -*- lexical-binding:t -*-
hmm, my key bindings (like C-M-x eval-defun) don't work on that. i
would have to use files even for small examples
i will think about adopting that a bit more
^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: thunk.el: Document that thunk-force == funcall?
2020-11-17 17:08 ` Tomas Hlavaty
2020-11-17 17:38 ` Michael Heerdegen
@ 2020-11-18 9:04 ` Alfred M. Szmidt
2020-11-18 22:21 ` Tomas Hlavaty
1 sibling, 1 reply; 27+ messages in thread
From: Alfred M. Szmidt @ 2020-11-18 9:04 UTC (permalink / raw)
To: Tomas Hlavaty; +Cc: emacs-devel
> thunks are functions.
thunks take no arguments (at least the definition i know)
> The background: because thunks are functions, one is allowed (and it is
> useful) to directly pass them to higher order functions e.g. as a test
> predicate,
a test predicate usually takes at least one argument
And returns a value -- something which thunks also do not.
thus a thunk cannot be used as a test predicate
^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: thunk.el: Document that thunk-force == funcall?
2020-11-18 9:04 ` Alfred M. Szmidt
@ 2020-11-18 22:21 ` Tomas Hlavaty
0 siblings, 0 replies; 27+ messages in thread
From: Tomas Hlavaty @ 2020-11-18 22:21 UTC (permalink / raw)
To: emacs-devel
On Wed 18 Nov 2020 at 04:04, "Alfred M. Szmidt" <ams@gnu.org> wrote:
> a test predicate usually takes at least one argument
>
> And returns a value -- something which thunks also do not.
that doesn't seem right, thunks can and do return value:
https://en.wikipedia.org/wiki/Thunk
Thunks are primarily used to delay a calculation until its result is
needed, or to insert operations at the beginning or end of the other
subroutine. They have many other applications in compiler code
generation and modular programming.
The term originated as a humorous, incorrect, past participle of
"think". That is, a "thunk value" becomes available after its
calculation routine is thought through, or executed.[1]
^ permalink raw reply [flat|nested] 27+ messages in thread
* RE: thunk.el: Document that thunk-force == funcall?
2020-11-17 15:17 thunk.el: Document that thunk-force == funcall? Michael Heerdegen
2020-11-17 17:08 ` Tomas Hlavaty
@ 2020-11-17 17:32 ` Drew Adams
2020-11-18 23:05 ` Tomas Hlavaty
2020-11-19 9:49 ` Nicolas Petton
2020-11-17 21:51 ` Stefan Monnier
2 siblings, 2 replies; 27+ messages in thread
From: Drew Adams @ 2020-11-17 17:32 UTC (permalink / raw)
To: Michael Heerdegen, Emacs Development; +Cc: Nicolas Petton, Stefan Monnier
> `thunk-force' is equivalent to `funcall' - thunks are functions. I
> wonder if we could/should officially document that fact?
>
> The background: because thunks are functions, one is allowed (and it is
> useful) to directly pass them to higher order functions e.g. as a test
> predicate, or bind them using `cl-flet', or `cl-letf' to a
> `symbol-function' place. Currently, the official solution would
> require
> to use a lambda wrapper.
>
> I have encountered this requirement several times (and using `thunk-
> let' was not always the solution) - thus my question.
FWIW, that makes sense to me. Why not tell users this?
It's fine to have an abstraction, but in Lisp it can
help to know the implementation, especially in a
straightforward case like this.
^ permalink raw reply [flat|nested] 27+ messages in thread
* RE: thunk.el: Document that thunk-force == funcall?
2020-11-17 17:32 ` Drew Adams
@ 2020-11-18 23:05 ` Tomas Hlavaty
2020-11-18 23:25 ` Tomas Hlavaty
2020-11-19 17:18 ` Tomas Hlavaty
2020-11-19 9:49 ` Nicolas Petton
1 sibling, 2 replies; 27+ messages in thread
From: Tomas Hlavaty @ 2020-11-18 23:05 UTC (permalink / raw)
To: emacs-devel
On Tue 17 Nov 2020 at 09:32, Drew Adams <drew.adams@oracle.com> wrote:
>> `thunk-force' is equivalent to `funcall' - thunks are functions. I
>> wonder if we could/should officially document that fact?
> FWIW, that makes sense to me. Why not tell users this?
> It's fine to have an abstraction, but in Lisp it can
> help to know the implementation, especially in a
> straightforward case like this.
what is the reason for thunk.el in the first place?
it is not used in emacs at all.
it does not work in buffers without lexical-binding:
(setq delayed (thunk-delay (message "this message is delayed")))
(thunk-force delayed)
-> Debugger entered--Lisp error: (void-variable forced)
(setq lexical-binding t)
(setq delayed (thunk-delay (message "this message is delayed")))
(thunk-force delayed)
-> "this message is delayed"
(this is probably general problem with macros which assume
lexical-binding)
thunk is about delaying computation. thunk.el mixes in memoization.
it obscures the essence:
(setq delayed (lambda () (message "this message is delayed")))
(funcall delayed)
memoization would be better separate:
(let ((void (list nil)))
(defun memoize (thunk)
(let ((z void))
(lambda ()
(when (eq z void)
(setq z (funcall thunk)))
z))))
(let* ((i 0)
(f (lambda () (message "hi %d" (incf i)))))
(funcall f)
(funcall f)
(funcall f))
(let* ((i 0)
(f (memoize (lambda () (message "hi %d" (incf i))))))
(funcall f)
(funcall f)
(funcall f))
^ permalink raw reply [flat|nested] 27+ messages in thread
* RE: thunk.el: Document that thunk-force == funcall?
2020-11-18 23:05 ` Tomas Hlavaty
@ 2020-11-18 23:25 ` Tomas Hlavaty
2020-11-19 11:50 ` Mattias Engdegård
2020-11-19 17:18 ` Tomas Hlavaty
1 sibling, 1 reply; 27+ messages in thread
From: Tomas Hlavaty @ 2020-11-18 23:25 UTC (permalink / raw)
To: emacs-devel
On Thu 19 Nov 2020 at 00:05, Tomas Hlavaty <tom@logand.com> wrote:
> (let ((void (list nil)))
> (defun memoize (thunk)
> (let ((z void))
> (lambda ()
> (when (eq z void)
> (setq z (funcall thunk)))
> z))))
it would be better to "free" the thunk:
(let ((void (list nil)))
(defun memoize (thunk)
(let ((z void))
(lambda ()
(when (eq z void)
(setq z (funcall thunk)
thunk nil))
z))))
interesting drawback of thunk.el implementation is that thunk-delay does
not allow that and whatever is captured stays captured even after
thunk-force.
^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: thunk.el: Document that thunk-force == funcall?
2020-11-18 23:25 ` Tomas Hlavaty
@ 2020-11-19 11:50 ` Mattias Engdegård
2020-11-19 18:14 ` Tomas Hlavaty
0 siblings, 1 reply; 27+ messages in thread
From: Mattias Engdegård @ 2020-11-19 11:50 UTC (permalink / raw)
To: Tomas Hlavaty; +Cc: emacs-devel
19 nov. 2020 kl. 00.25 skrev Tomas Hlavaty <tom@logand.com>:
> interesting drawback of thunk.el implementation is that thunk-delay does
> not allow that and whatever is captured stays captured even after
> thunk-force.
Not after byte-compilation. The interpreter is very conservative when creating closures but the compiler makes more of an effort to capture only variables actually used.
^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: thunk.el: Document that thunk-force == funcall?
2020-11-19 11:50 ` Mattias Engdegård
@ 2020-11-19 18:14 ` Tomas Hlavaty
0 siblings, 0 replies; 27+ messages in thread
From: Tomas Hlavaty @ 2020-11-19 18:14 UTC (permalink / raw)
To: emacs-devel
On Thu 19 Nov 2020 at 12:50, Mattias Engdegård <mattiase@acm.org> wrote:
> 19 nov. 2020 kl. 00.25 skrev Tomas Hlavaty <tom@logand.com>:
>
>> interesting drawback of thunk.el implementation is that thunk-delay does
>> not allow that and whatever is captured stays captured even after
>> thunk-force.
>
> Not after byte-compilation. The interpreter is very conservative when
> creating closures but the compiler makes more of an effort to capture
> only variables actually used.
this has nothing to do with an effort to capture only variables actually
used
the problem is that current implementation of thunk-delay has memory
leak built-in by design and does not release captured values.
those are captured in the lambda due to ,@body and references to these
captured values are never released. it is not even possible to
explicitly release those values because it is not known what exactly is
captured.
(let ((huge-data (make-list (expt 2 60))))
(setq delayed (thunk-delay (count huge-data)))
;; huge-data captured by delayed, needed to compute the delayed value
(thunk-force delayed))
(garbage-collect)
;; huge-data still captured by delayed even though it is not needed anymore
this is bug in thunk-delay
having garbage collection does not eliminate memory leaks
fix would separate thunk and memoization and release the thunk when not
needed anymore:
(let ((void (list nil)))
(defun memoize (thunk)
(let ((z void))
(lambda ()
(when (eq z void)
(setq z (funcall thunk)
thunk nil))
z))))
(defmacro thunk-delay (&rest body)
`(memoize (lambda () ,@body)))
at that point thunk-delay is pretty much redundant and it is better not
to pretend that thunk is thunk+memoization and just teach people what
thunk really is (it is just a lambda with zero arguments)
minor note: the example in thunk.el would then also work in buffers
without lexical-binding enabled
^ permalink raw reply [flat|nested] 27+ messages in thread
* RE: thunk.el: Document that thunk-force == funcall?
2020-11-18 23:05 ` Tomas Hlavaty
2020-11-18 23:25 ` Tomas Hlavaty
@ 2020-11-19 17:18 ` Tomas Hlavaty
1 sibling, 0 replies; 27+ messages in thread
From: Tomas Hlavaty @ 2020-11-19 17:18 UTC (permalink / raw)
To: emacs-devel
On Thu 19 Nov 2020 at 00:05, Tomas Hlavaty <tom@logand.com> wrote:
> thunk is about delaying computation. thunk.el mixes in memoization.
even wikipedia mentions that memoization is a separat feature from thunk
https://en.wikipedia.org/wiki/Thunk
Compilers for these languages, such as the Glasgow Haskell Compiler,
have relied heavily on thunks, with the added feature that the thunks
save their initial result so that they can avoid recalculating it;[5]
this is known as memoization or call-by-need.
^ permalink raw reply [flat|nested] 27+ messages in thread
* RE: thunk.el: Document that thunk-force == funcall?
2020-11-17 17:32 ` Drew Adams
2020-11-18 23:05 ` Tomas Hlavaty
@ 2020-11-19 9:49 ` Nicolas Petton
1 sibling, 0 replies; 27+ messages in thread
From: Nicolas Petton @ 2020-11-19 9:49 UTC (permalink / raw)
To: Drew Adams, Michael Heerdegen, Emacs Development; +Cc: Stefan Monnier
[-- Attachment #1: Type: text/plain, Size: 404 bytes --]
Drew Adams <drew.adams@oracle.com> writes:
>> I have encountered this requirement several times (and using `thunk-
>> let' was not always the solution) - thus my question.
> FWIW, that makes sense to me. Why not tell users this?
> It's fine to have an abstraction, but in Lisp it can
> help to know the implementation, especially in a
> straightforward case like this.
Indeed, I agree.
Cheers,
Nico
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 507 bytes --]
^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: thunk.el: Document that thunk-force == funcall?
2020-11-17 15:17 thunk.el: Document that thunk-force == funcall? Michael Heerdegen
2020-11-17 17:08 ` Tomas Hlavaty
2020-11-17 17:32 ` Drew Adams
@ 2020-11-17 21:51 ` Stefan Monnier
2020-11-25 14:16 ` Michael Heerdegen
2 siblings, 1 reply; 27+ messages in thread
From: Stefan Monnier @ 2020-11-17 21:51 UTC (permalink / raw)
To: Michael Heerdegen; +Cc: Nicolas Petton, Emacs Development
Michael Heerdegen [2020-11-17 16:17:14] wrote:
> The background: because thunks are functions, one is allowed (and it is
> useful) to directly pass them to higher order functions e.g. as a test
> predicate, or bind them using `cl-flet', or `cl-letf' to a
> `symbol-function' place. Currently, the official solution would require
> to use a lambda wrapper.
I like the idea that we could replace the current representation with
something more like a `defstruct`, so I'm on the side of "don't document
that thunks *are* functions", but it's not a very strong opinion.
In any case even without documenting them to be functions, we could
provide
(defalias 'thunk-function #'identity)
to avoid the wasteful use of a lambda wrapper.
> I have encountered this requirement several times (and using `thunk-let'
> was not always the solution) - thus my question.
[...]
> Say the code loops over a list of files or so (not known at think time).
> The thunk could "contain" a test that might take long (e.g. something
> that might need to look at the file's contents) but the result is
> interesting only sometimes, depending on the result of other tests (also
> not known at think time).
[ That's still pretty hypothetical. ]
I'd be curious to see concrete examples, to get a better idea of
the tradeoffs.
Stefan
^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: thunk.el: Document that thunk-force == funcall?
2020-11-17 21:51 ` Stefan Monnier
@ 2020-11-25 14:16 ` Michael Heerdegen
2020-11-27 17:22 ` Michael Heerdegen
0 siblings, 1 reply; 27+ messages in thread
From: Michael Heerdegen @ 2020-11-25 14:16 UTC (permalink / raw)
To: Stefan Monnier; +Cc: Nicolas Petton, Emacs Development
Stefan Monnier <monnier@iro.umontreal.ca> writes:
> (defalias 'thunk-function #'identity)
I like that idea.
> [ That's still pretty hypothetical. ]
> I'd be curious to see concrete examples, to get a better idea of
> the tradeoffs.
I don't have any concrete real-life examples. Only the one somewhat
unconventional that led me to this question:
I had been implementing my own improved version of `dired-mark-sexp'.
It offers some more boolean valued tests, and some are a bit more
expensive and partly interdepend, so I made them thunks. When the input
expression is evaluated these are made accessible as functions.
I would rather do this via `thunk-let' and make these tests available as
contents of certain variables but then the repeated macroexpansion
slowed down the thing too much.
Regards,
Michael.
^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: thunk.el: Document that thunk-force == funcall?
2020-11-25 14:16 ` Michael Heerdegen
@ 2020-11-27 17:22 ` Michael Heerdegen
2020-12-17 4:37 ` Michael Heerdegen
0 siblings, 1 reply; 27+ messages in thread
From: Michael Heerdegen @ 2020-11-27 17:22 UTC (permalink / raw)
To: Stefan Monnier; +Cc: Nicolas Petton, Emacs Development
Michael Heerdegen <michael_heerdegen@web.de> writes:
> I don't have any concrete real-life examples. [...]
Actually I have one real-life example:
#+begin_src emacs-lisp
(defun el-search--change-p (posn revision)
;; Non-nil when sexp after POSN is part of a change
(if (buffer-modified-p)
(if (eq this-command 'el-search-pattern)
(user-error "Buffer is modified - please save")
nil)
(save-restriction
(widen)
(let ((changes (el-search--changes-from-diff-hl revision))
(sexp-end (el-search--end-of-sexp posn))
(atomic? (thunk-delay (el-search--atomic-p
(save-excursion (goto-char posn)
(el-search-read (current-buffer)))))))
(while (and changes (or (< (cdar changes) posn)
(and
;; a string spanning multiple lines is a change even when not all
;; lines are changed
(< (cdar changes) sexp-end)
(not (thunk-force atomic?)))))
(pop changes))
(and changes (or (<= (caar changes) posn)
(and (thunk-force atomic?)
(<= (caar changes) sexp-end))))))))
#+end_src
I can live with `thunk-force' here; being able to use it as a function
would make the code a bit more readable. Although, hiding the
function's nature as a thunk might not be good OTOH.
A thunk or fbound function have the advantage that they can be passed
without forcing. Contrary to symbols bound with `thunk-let'; a thunk
passed to a function this way is forced before that function is called,
even when the thunk's result is never used. That's why I'm wondering
whether `thunk-let' is as cool as I had thought.
Ok, anything else to say, apart from the empty string?
Regards,
Michael.
^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: thunk.el: Document that thunk-force == funcall?
2020-11-27 17:22 ` Michael Heerdegen
@ 2020-12-17 4:37 ` Michael Heerdegen
2020-12-18 2:58 ` Adam Porter
0 siblings, 1 reply; 27+ messages in thread
From: Michael Heerdegen @ 2020-12-17 4:37 UTC (permalink / raw)
To: Stefan Monnier; +Cc: Nicolas Petton, Emacs Development
Michael Heerdegen <michael_heerdegen@web.de> writes:
> Actually I have one real-life example:
An example involving side effects: being able to create a function
that one can add to a hook that performs some kind of (expensive) setup
but only when called for the first time.
Regards,
Michael.
^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: thunk.el: Document that thunk-force == funcall?
2020-12-17 4:37 ` Michael Heerdegen
@ 2020-12-18 2:58 ` Adam Porter
0 siblings, 0 replies; 27+ messages in thread
From: Adam Porter @ 2020-12-18 2:58 UTC (permalink / raw)
To: emacs-devel
Michael Heerdegen <michael_heerdegen@web.de> writes:
> Michael Heerdegen <michael_heerdegen@web.de> writes:
>
>> Actually I have one real-life example:
>
> An example involving side effects: being able to create a function
> that one can add to a hook that performs some kind of (expensive) setup
> but only when called for the first time.
FYI, someone recently showed me that the letrec macro can be used to do
this. There's an example of it in the Elisp manual, section 12.3 Local
Variables (added about a year ago, so I think most Emacs users don't
know of it yet).
^ permalink raw reply [flat|nested] 27+ messages in thread
end of thread, other threads:[~2020-12-18 2:58 UTC | newest]
Thread overview: 27+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2020-11-17 15:17 thunk.el: Document that thunk-force == funcall? Michael Heerdegen
2020-11-17 17:08 ` Tomas Hlavaty
2020-11-17 17:38 ` Michael Heerdegen
2020-11-17 18:09 ` Tomas Hlavaty
2020-11-17 21:07 ` Michael Heerdegen
2020-11-17 22:42 ` Tomas Hlavaty
2020-11-17 23:52 ` Stefan Monnier
2020-11-18 8:01 ` Tomas Hlavaty
2020-11-18 14:04 ` Stefan Monnier
2020-11-18 22:19 ` Tomas Hlavaty
2020-11-18 22:49 ` Stefan Monnier
2020-11-18 23:13 ` Tomas Hlavaty
2020-11-18 23:40 ` Stephen Leake
2020-11-18 9:04 ` Alfred M. Szmidt
2020-11-18 22:21 ` Tomas Hlavaty
2020-11-17 17:32 ` Drew Adams
2020-11-18 23:05 ` Tomas Hlavaty
2020-11-18 23:25 ` Tomas Hlavaty
2020-11-19 11:50 ` Mattias Engdegård
2020-11-19 18:14 ` Tomas Hlavaty
2020-11-19 17:18 ` Tomas Hlavaty
2020-11-19 9:49 ` Nicolas Petton
2020-11-17 21:51 ` Stefan Monnier
2020-11-25 14:16 ` Michael Heerdegen
2020-11-27 17:22 ` Michael Heerdegen
2020-12-17 4:37 ` Michael Heerdegen
2020-12-18 2:58 ` Adam Porter
Code repositories for project(s) associated with this public inbox
https://git.savannah.gnu.org/cgit/emacs.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).