From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Tomas Hlavaty Newsgroups: gmane.emacs.devel Subject: Re: using finalizers Date: Sun, 02 Jan 2022 08:53:31 +0100 Message-ID: <87r19qk1z8.fsf@logand.com> References: <878rw1pvcw.fsf@logand.com> <83r19tgqlq.fsf@gnu.org> <87v8z5nmsp.fsf@logand.com> <875yr3krs5.fsf@logand.com> <83tuemeif5.fsf@gnu.org> Mime-Version: 1.0 Content-Type: text/plain Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="37579"; mail-complaints-to="usenet@ciao.gmane.io" Cc: rudi@constantly.at, emacs-devel@gnu.org To: Eli Zaretskii Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Sun Jan 02 08:55:55 2022 Return-path: Envelope-to: ged-emacs-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1n3vid-0009VX-5d for ged-emacs-devel@m.gmane-mx.org; Sun, 02 Jan 2022 08:55:55 +0100 Original-Received: from localhost ([::1]:55848 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1n3vib-0007j2-Pe for ged-emacs-devel@m.gmane-mx.org; Sun, 02 Jan 2022 02:55:53 -0500 Original-Received: from eggs.gnu.org ([209.51.188.92]:39572) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1n3vgP-0006DN-Kq for emacs-devel@gnu.org; Sun, 02 Jan 2022 02:53:37 -0500 Original-Received: from logand.com ([37.48.87.44]:38120) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1n3vgN-0000lY-Db; Sun, 02 Jan 2022 02:53:37 -0500 Original-Received: by logand.com (Postfix, from userid 1001) id C876619EC87; Sun, 2 Jan 2022 08:53:32 +0100 (CET) X-Mailer: emacs 27.2 (via feedmail 11-beta-1 I) In-Reply-To: <83tuemeif5.fsf@gnu.org> Received-SPF: pass client-ip=37.48.87.44; envelope-from=tom@logand.com; helo=logand.com X-Spam_score_int: 0 X-Spam_score: -0.0 X-Spam_bar: / X-Spam_report: (-0.0 / 5.0 requ) SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Original-Sender: "Emacs-devel" Xref: news.gmane.io gmane.emacs.devel:283861 Archived-At: On Sun 02 Jan 2022 at 08:54, Eli Zaretskii wrote: > One common paradigm for such use cases is to use a callback: the > function that traverses some collection of objects calls that callback > for every object it finds. There is no such thing in directory-files. Also using callbacks in directory-files alone would not fix find-lisp-find-dired. I think an extra execution thread would be needed. Maybe using an extra execution thread could be seen as a work around for the negative consequences of stack discipline. If there was a callback in directory-files, it would allow for pushing directory items out of the execution thread when encountered, instead of waiting until the contents of the whole directory has been collected. > There are usually protocols for the callback to stop the process and > prevent the rest of objects from being examined/reported. There is no such thing in directory-files. I can't see such thing in any other mapping functions in Emacs Lisp. Could you point me to an example? Also, such protocols are non-trivial and usually complicate the code significantly. See for example how people deal with it JavaScript, where they have no other choice. >> But when should close be called, when such stream is lazy? I am >> sure a reasonable ad-hoc solution could be implemented. But what if >> the directory is traversed recursively? When should each close be >> called? A case for another ad-hoc solution? Looks like finalizers >> would be great here. > > We have the unwind-protect machinery in place for that. In fact, if > you look at the implementation of directory-files, you will see that > it already uses that machinery, and it most probably actually kicked > in when you typed C-g above. I see I did not manage to get the point across. directory-files is "eager" and follows stack discipline so unwind-protect there makes sense. But the idea of directory-files itself is broken as I demonstrated previously. Operating systems get it right and provide streaming API with open/read/close directory functions. Also the brokenness of directory-files propagates to many other points in Emacs that need to list directories, like find-lisp-find-dired and completion. But the point here was that in case of lazy sequences, the point in time where close should be called is not known upfront. One would have to split the code into two parts: eager one which would force stack discipline and lazy one which would do the lazy part. Maybe doable for trivial thing like lazy listing of a single directory, but it gets harder when doing something non-trivial like lazy listing of a directory recursively.