From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: =?utf-8?Q?Ludovic_Court=C3=A8s?= Newsgroups: gmane.lisp.guile.user,gmane.lisp.guile.devel Subject: Re: mmap for guile Date: Mon, 04 Jul 2022 12:09:43 +0200 Message-ID: <87k08tfau0.fsf@gnu.org> References: <56ee7537-1666-3d04-7093-732a75624e9b@gmail.com> <0cf4e4ee80169487694b844996e04f3293eab92f.camel@telenet.be> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="1989"; mail-complaints-to="usenet@ciao.gmane.io" User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.1 (gnu/linux) Cc: guile-devel@gnu.org To: guile-user@gnu.org Cancel-Lock: sha1:r/BtfYTeBDwwgr13KcjooOkb6cE= Original-X-From: guile-user-bounces+guile-user=m.gmane-mx.org@gnu.org Mon Jul 04 12:10:36 2022 Return-path: Envelope-to: guile-user@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1o8J2J-0000N2-RS for guile-user@m.gmane-mx.org; Mon, 04 Jul 2022 12:10:35 +0200 Original-Received: from localhost ([::1]:45622 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1o8J2I-0001IG-HV for guile-user@m.gmane-mx.org; Mon, 04 Jul 2022 06:10:34 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]:50898) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1o8J1m-0001I2-Tz for guile-user@gnu.org; Mon, 04 Jul 2022 06:10:02 -0400 Original-Received: from ciao.gmane.io ([116.202.254.214]:35984) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1o8J1l-00022r-FC for guile-user@gnu.org; Mon, 04 Jul 2022 06:10:02 -0400 Original-Received: from list by ciao.gmane.io with local (Exim 4.92) (envelope-from ) id 1o8J1e-000AC9-5V for guile-user@gnu.org; Mon, 04 Jul 2022 12:09:54 +0200 X-Injected-Via-Gmane: http://gmane.org/ Original-Followup-To: gmane.lisp.guile.devel X-URL: http://www.fdn.fr/~lcourtes/ X-Revolutionary-Date: Sextidi 16 Messidor an 230 de la =?utf-8?Q?R=C3=A9vo?= =?utf-8?Q?lution=2C?= jour du Tabac X-PGP-Key-ID: 0x090B11993D9AEBB5 X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4 0CFB 090B 1199 3D9A EBB5 X-OS: x86_64-pc-linux-gnu Received-SPF: pass client-ip=116.202.254.214; envelope-from=guile-user@m.gmane-mx.org; helo=ciao.gmane.io X-Spam_score_int: -16 X-Spam_score: -1.7 X-Spam_bar: - X-Spam_report: (-1.7 / 5.0 requ) BAYES_00=-1.9, HEADER_FROM_DIFFERENT_DOMAINS=0.249, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: guile-user@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: General Guile related discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guile-user-bounces+guile-user=m.gmane-mx.org@gnu.org Original-Sender: "guile-user" Xref: news.gmane.io gmane.lisp.guile.user:18390 gmane.lisp.guile.devel:21250 Archived-At: Hello, Having ‘mmap’ bindings would be much welcome. Maxime Devos skribis: > +SCM_DEFINE (scm_mmap_search, "mmap/search", 2, 4, 0, > + (SCM addr, SCM len, SCM prot, SCM flags, SCM fd, SCM > offset), > + "See the unix man page for mmap. Returns a bytevector.\n" > + "Note that the region allocated will be searched by the > garbage\n" > + "collector for pointers. Defaults:\n" > > I think it would be a good idea to document it will be automatically > unmapped during GC, as this is a rather low-leel interface Agreed. I was wondering about offering an explicit ‘munmap’ interface: it would be useful for fine-grain OS resource management, just like ‘close-fdes’. Doing that naively would mean that one can trivially get a pure Scheme program to segfault, which is contrary to what we do. But we could provide special semantics: the bytevector would become zero-length (possible, but weird, as Maxime points out), or it would be turned into a /dev/zero mapping (weird as well). Thoughts? > Also, what if you mmap a region, use bytevector->pointer and pass it to > some C thing, which saves the pointer somewhere where boehm-gc can find > it and boehm-gc considers it to be live, is there something that > prevents boehm-gc from improperly calling the finalizer & unmapping the > region, causing a dangling pointer? There’s a risk, but I don’t think it’s specific to mmap. > Also, WDYT of using ports instead of raw fds in the API? That would > play nicer with move->fdes etc. Agreed. >>+ /* Invalidate further work on this bytevector. */ >>+ SCM_BYTEVECTOR_SET_LENGTH (bvec, 0); >>+ SCM_BYTEVECTOR_SET_CONTENTS (bvec, NULL); > > Possibly Guile's optimiser assumes that bytevectors never change in > length (needs to be checked). So unless the relevant optimiser code is > changed, and it is documented that bytevectors can change in length, I > think it would be safer to not have an unmapping procedure in Scheme > (though a procedure for remapping it as /dev/zero should be safe). I don’t think the optimizer makes any such assumption, except for literal bytevectors. Besides what Maxime points out, some more superficial issues: • In documentation, please refer to the relevant glibc section instead of “See man page” (info "(libc) Memory-mapped I/O"). • Please update doc/ref with a section on memory-mapped I/O. • Make sure to follow the GNU coding in C: space before opening paren, braces on a line of their own, etc. Since you already have a copyright assignment on file, there won’t be administrative delays, which is a good thing. :-) I hope we can have those ‘mmap’ bindings soonish! Thanks, Ludo’.