From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Tomas Hlavaty Newsgroups: gmane.emacs.devel Subject: Re: using finalizers Date: Sun, 02 Jan 2022 14:10:02 +0100 Message-ID: <87h7am5ln9.fsf@logand.com> References: <878rw1pvcw.fsf@logand.com> <83r19tgqlq.fsf@gnu.org> <87v8z5nmsp.fsf@logand.com> <875yr3krs5.fsf@logand.com> <83tuemeif5.fsf@gnu.org> <87r19qk1z8.fsf@logand.com> <83k0fiedu3.fsf@gnu.org> Mime-Version: 1.0 Content-Type: text/plain Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="11770"; mail-complaints-to="usenet@ciao.gmane.io" Cc: rudi@constantly.at, emacs-devel@gnu.org To: Eli Zaretskii Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Sun Jan 02 14:11:17 2022 Return-path: Envelope-to: ged-emacs-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1n40dp-0002w0-Fj for ged-emacs-devel@m.gmane-mx.org; Sun, 02 Jan 2022 14:11:17 +0100 Original-Received: from localhost ([::1]:49092 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1n40dn-0001y5-Fc for ged-emacs-devel@m.gmane-mx.org; Sun, 02 Jan 2022 08:11:15 -0500 Original-Received: from eggs.gnu.org ([209.51.188.92]:55676) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1n40cm-0001Ft-5w for emacs-devel@gnu.org; Sun, 02 Jan 2022 08:10:13 -0500 Original-Received: from logand.com ([37.48.87.44]:52728) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1n40ch-000565-JG; Sun, 02 Jan 2022 08:10:11 -0500 Original-Received: by logand.com (Postfix, from userid 1001) id 59E0319EC87; Sun, 2 Jan 2022 14:10:03 +0100 (CET) X-Mailer: emacs 27.2 (via feedmail 11-beta-1 I) In-Reply-To: <83k0fiedu3.fsf@gnu.org> Received-SPF: pass client-ip=37.48.87.44; envelope-from=tom@logand.com; helo=logand.com X-Spam_score_int: 0 X-Spam_score: -0.0 X-Spam_bar: / X-Spam_report: (-0.0 / 5.0 requ) SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Original-Sender: "Emacs-devel" Xref: news.gmane.io gmane.emacs.devel:283905 Archived-At: On Sun 02 Jan 2022 at 10:33, Eli Zaretskii wrote: >> From: Tomas Hlavaty >> Cc: rudi@constantly.at, emacs-devel@gnu.org >> Date: Sun, 02 Jan 2022 08:53:31 +0100 >> >> On Sun 02 Jan 2022 at 08:54, Eli Zaretskii wrote: >> > One common paradigm for such use cases is to use a callback: the >> > function that traverses some collection of objects calls that callback >> > for every object it finds. >> >> There is no such thing in directory-files. > > It should be easy to add, if we wanted to. Or we could implement a > separate set of primitives using that, if it were deemed useful. > > I fail to see how what's in Emacs now is of any essence for the > purposes of this discussion. What exactly are you getting at, and how > is what is in Emacs today relevant? Are you saying that Emacs > couldn't possibly provide such primitives due to some inherent > limitation? If so, please explain why you think so, because I don't > think I agree. I was replying to Rudolf about the use-cases where stack discipline can have visible negative consequences and demonstrating it with a few examples in Emacs. It is also related to the use of finalizers as an alternative mechanism to stack discipline. I am saying that there are things that could be improved: - Emacs Lisp should provide open-directory, read-directory and close-directory functions. I think that is the most important point. There are other related observations, like: - find-lisp-find-dired is fundamentally broken. - the brokenness is due to directory-files being the wrong API. It prevents people from writing better code. - the brokenness propagates to everything which uses directory-files, like find-lisp-find-dired, completion (e.g. completion on /nix/store blocks emacs completely for significant amount of time). Now I see that even lisp/url uses directory-files thus there are cases where it will be unuseable. - directory-files implies consing all the directory entries except a few pre-hardcoded filter use-cases. It does not allow anything else to be more efficient, where one would want to filter out unneeded directory entries at the moment they are first encountered. - return_count argument to directory_files_internal shows a desperate attempt to fix patological deficiency:-) return_count is missing from directory-files-recursively. return_count still does not provide sufficient control of previously uncontrollable behaviour. - directory-files-recursively brings yet another different interface for traversing file system - directory-files-recursively seems to be built on different primitive file-name-all-completions I do not understand the implementation of file-name-all-completions yet but from the observed behaviour, it seems to have the same flaws like directory-files. > I don't see why. A callback can easily insert files into the buffer, > and trigger redisplay either directly or via timers and such. But the control is in the hand of the mapping function. Triggering redisplay is not sufficient for it to behave like in find-dired, I think. It needs to handle input and everything as if it was running in a different execution thread, like find-dired does with the "mapping function" running in an external process (find program). > Keep in mind that when Emacs invokes an async subprocess, the display > of what that subprocess produces is still done in the same single > thread where the Lisp machine runs. So we already have the > capabilities of "pretending" we have multithreading, when these > aspects are involved. Yes, it works well when the mapping function (in the case of find-dired, the find process) runs in a different execution thread/process. It does not work well, when it runs in the same single execution thread. See find-lisp-find-dired. >> > There are usually protocols for the callback to stop the process and >> > prevent the rest of objects from being examined/reported. >> >> There is no such thing in directory-files. > > I think we have something like that in lisp/url/. Thanks for the hint. Unfortunatelly I can't find anything from after a quick search. I can see that it uses directory-files which means it also inherits its brokenness in those cases. >> Also, such protocols are non-trivial and usually complicate the code >> significantly. > > I don't see why this would be non-trivial. In the simplest case, the > callback returns a special value which tells to stop the iteration. The simplest case is also the least interesting. > Emacs is not an OS (the popular myth about that notwithstanding). It > provides primitives that are useful for writing editing and > text-processing applications. What you don't find in Emacs is > probably outside of its scope, or at least is rarely if ever needed. Directory traversal seems to be essential part of Emacs. >> But the point here was that in case of lazy sequences, the point in time >> where close should be called is not known upfront. One would have to >> split the code into two parts: eager one which would force stack >> discipline and lazy one which would do the lazy part. Maybe doable for >> trivial thing like lazy listing of a single directory, but it gets >> harder when doing something non-trivial like lazy listing of a directory >> recursively. > > Where you see something non-trivial, I see just simple use of our > unwinding infrastructure. There's no problem I see with using that > infrastructure in recursive processing of directories. If you see > such problems, please tell what they are, specifically, given the > unwind_protect functions we have and the mechanism that actually > implements the unwinding. In Emacs, many operations could signal an > error or throw, and the user could type C-g at any moment, so this > mechanism is in use all the time, and must handle correctly nested > unwinders. Problem is not with unwind-protect. Problem is with directory-files and all functions that use it. I think I pointed out the problems sufficiently. One fix could be implementing an alternative to directory-files, e.g. directory mapping function, plus running it in a extra thread. Another possibility would be to traverse the directory lazily, but that brings an issue of eagerly closing directories.