From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Tomas Hlavaty Newsgroups: gmane.emacs.devel Subject: Re: using finalizers Date: Sat, 01 Jan 2022 23:36:10 +0100 Message-ID: <875yr3krs5.fsf@logand.com> References: <878rw1pvcw.fsf@logand.com> <83r19tgqlq.fsf@gnu.org> <87v8z5nmsp.fsf@logand.com> Mime-Version: 1.0 Content-Type: text/plain Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="33762"; mail-complaints-to="usenet@ciao.gmane.io" To: Rudolf Schlatte , emacs-devel@gnu.org Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Sat Jan 01 23:37:22 2022 Return-path: Envelope-to: ged-emacs-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1n3n06-0008aN-Gf for ged-emacs-devel@m.gmane-mx.org; Sat, 01 Jan 2022 23:37:22 +0100 Original-Received: from localhost ([::1]:37788 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1n3n04-00050S-Nk for ged-emacs-devel@m.gmane-mx.org; Sat, 01 Jan 2022 17:37:20 -0500 Original-Received: from eggs.gnu.org ([209.51.188.92]:33556) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1n3mz3-0004J7-Cy for emacs-devel@gnu.org; Sat, 01 Jan 2022 17:36:17 -0500 Original-Received: from logand.com ([37.48.87.44]:38796) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1n3mz1-0000A7-Kf for emacs-devel@gnu.org; Sat, 01 Jan 2022 17:36:17 -0500 Original-Received: by logand.com (Postfix, from userid 1001) id 48AD019EC87; Sat, 1 Jan 2022 23:36:12 +0100 (CET) X-Mailer: emacs 27.2 (via feedmail 11-beta-1 I) In-Reply-To: Received-SPF: pass client-ip=37.48.87.44; envelope-from=tom@logand.com; helo=logand.com X-Spam_score_int: 0 X-Spam_score: -0.0 X-Spam_bar: / X-Spam_report: (-0.0 / 5.0 requ) SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Original-Sender: "Emacs-devel" Xref: news.gmane.io gmane.emacs.devel:283827 Archived-At: On Fri 31 Dec 2021 at 15:23, Rudolf Schlatte wrote: > I'm sure you knew this already, but in general, using gc for non-memory > resource management (e.g., "please close this file when this Lisp object > is GCed") is not a good idea--depending on the GC behavior, you'll run > out of file handles or whatnot. The RAII pattern in C++ > deterministically calls a destructor when a stack-allocated object goes > out of scope; in Lisp, the various `with-foo' macros serve the same > purpose. This is another topic. Somehow it seems very controversial. Some even violently oppose the idea of using gc for non-memory resources but I have not found anybody yet who would help me to understand real deep pros and cons and let me form my own opinion without forcing his own. Following stack discipline is great. Most of the time. But it comes with severe restrictions and consequences. There is a reason why gc was invented. For use-cases, which are not covered by with-foo stuff, one can implement own ad-hoc resource management, or reuse gc. Thus somehow there seems to be a need for finalizers. The negative consequences of following the stack discipline can be seen in many places. Lets have a look at the function directory-files. It is likely the worst possible API for accessing filesystem. (Common Lisp has the same flaw.) It opens a directory, does its work and closes the directory. Nice, it is done "properly": open, work, close. The problem is, that the amount of work done inside open/close is not under control of the programmer and is not bound in space and time. Additionally, the whole work needs to be done completely, or completely aborted. It means that unless the amount of work is trivial, the whole thing and all the things that are built on top of it are useless. That sounds a bit extreme claim. Lets test it with something real: M-x find-lisp-find-dired /nix/store [.]service$. While Emacs blocked, no results seen. C-g after several seconds. That was not very useful. Lets try M-x find-dired /nix/store -name '*.service'. That works nicely. Why? Because the directory traversal runs in a second process (the control loop pushes the results out of the process step by step) and the first emacs process displays the results step by step as they come (with directory entry granularity). How could find-lisp-find-dired be fixed? Could it be fixed while still using directory-files? I do not think so. I guess it can only be fixed by pushing directory entries out from another thread (like find-dired does) or using a "stream" of directory entries pulled on demand. But when should close be called, when such stream is lazy? I am sure a reasonable ad-hoc solution could be implemented. But what if the directory is traversed recursively? When should each close be called? A case for another ad-hoc solution? Looks like finalizers would be great here. Yes there would be issues, like hitting the open file limit. But are those issues show stoppers? Could there be a useful solution? Maybe run gc and try to open the file again? Or something more sophisticated? Have somebody explored this area?