From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Maxime Devos Newsgroups: gmane.lisp.guile.user,gmane.lisp.guile.devel Subject: Re: mmap for guile Date: Sun, 26 Jun 2022 20:11:07 +0200 Message-ID: <0cf4e4ee80169487694b844996e04f3293eab92f.camel@telenet.be> References: <56ee7537-1666-3d04-7093-732a75624e9b@gmail.com> Mime-Version: 1.0 Content-Type: multipart/signed; micalg="pgp-sha512"; protocol="application/pgp-signature"; boundary="=-pwo4+W4VnqkAw3mXd0Bv" Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="27524"; mail-complaints-to="usenet@ciao.gmane.io" User-Agent: Evolution 3.38.3-1 To: Matt Wette , guile-devel@gnu.org, Guile User Original-X-From: guile-user-bounces+guile-user=m.gmane-mx.org@gnu.org Sun Jun 26 20:28:06 2022 Return-path: Envelope-to: guile-user@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1o5WzN-0006zG-1T for guile-user@m.gmane-mx.org; Sun, 26 Jun 2022 20:28:05 +0200 Original-Received: from localhost ([::1]:40986 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1o5WzL-0004Sa-So for guile-user@m.gmane-mx.org; Sun, 26 Jun 2022 14:28:03 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]:42978) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1o5WjF-0004OT-MC for guile-user@gnu.org; Sun, 26 Jun 2022 14:11:27 -0400 Original-Received: from andre.telenet-ops.be ([2a02:1800:120:4::f00:15]:43216) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1o5WjD-0004GD-9g for guile-user@gnu.org; Sun, 26 Jun 2022 14:11:25 -0400 Original-Received: from ptr-bvsjgyhxw7psv60dyze.18120a2.ip6.access.telenet.be ([IPv6:2a02:1811:8c09:9d00:3c5f:2eff:feb0:ba5a]) by andre.telenet-ops.be with bizsmtp id nuBJ2700D4UW6Th01uBJU0; Sun, 26 Jun 2022 20:11:18 +0200 In-Reply-To: <56ee7537-1666-3d04-7093-732a75624e9b@gmail.com> DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=telenet.be; s=r22; t=1656267079; bh=X+ziLPtS01pIowVcyHMBC6yhYe+aML/xdLJ+n6VIx8w=; h=Subject:From:To:Date:In-Reply-To:References; b=PdeDl+RV64w9TWK3+e2LYV+NLswrjUzOhwcX1Z487zN9L76jSAQm1Y9uSzGLeOSOj ifhbKmBJzMo5evjcK45F3ysSEjNL6FVyLZD/M3mJs5sHz6cbhaExzXj8oU/X+R8OmW BY3r2r2cvjDGrpk7/rhJgGfYPl4OcBim0dOShb4IPRkjqIApxZB7X+M+MmeF5kI9wx kr4MWyIE8u4EiDG3W3nmWAdeBF2OpD1mq3pvPq6wlz6WhtyfJSWLNIx6AiNe4a29/U m2Go74/wc5EaAg1bTl9/m9oNl0HP04Sxa9IOyzc7I7jrHrloCk1fMh55vOWmmu8jS/ M6M3FDnES1oDw== Received-SPF: pass client-ip=2a02:1800:120:4::f00:15; envelope-from=maximedevos@telenet.be; helo=andre.telenet-ops.be X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: guile-user@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: General Guile related discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guile-user-bounces+guile-user=m.gmane-mx.org@gnu.org Original-Sender: "guile-user" Xref: news.gmane.io gmane.lisp.guile.user:18348 gmane.lisp.guile.devel:21246 Archived-At: --=-pwo4+W4VnqkAw3mXd0Bv Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Some old mmap things that might be useful: * https://lists.nongnu.org/archive/html/guile-devel/2013-04/msg00235.html * https://lists.gnu.org/archive/html/bug-guile/2017-11/msg00033.html * https://lists.gnu.org/archive/html/guile-user/2006-11/msg00013.html +SCM_DEFINE (scm_mmap_search, "mmap/search", 2, 4, 0,=20 + (SCM addr, SCM len, SCM prot, SCM flags, SCM fd, SCM offset), + "See the unix man page for mmap. Returns a bytevector.\n" + "Note that the region allocated will be searched by the garbage\n" + "collector for pointers. Defaults:\n" I think it would be a good idea to document it will be automatically unmapped during GC, as this is a rather low-leel interface Also, what if you mmap a region, use bytevector->pointer and pass it to some C thing, which saves the pointer somewhere where boehm-gc can find it and boehm-gc considers it to be live, is there something that prevents boehm-gc from improperly calling the finalizer & unmapping the region, causing a dangling pointer? Also, WDYT of using ports instead of raw fds in the API? That would play nicer with move->fdes etc. > + GC_exclude_static_roots(ptr, (char*)ptr + len); After an unmap, will the GC=C2=A0properly forget the roots information? >+ /* Invalidate further work on this bytevector. */ >+ SCM_BYTEVECTOR_SET_LENGTH (bvec, 0); >+ SCM_BYTEVECTOR_SET_CONTENTS (bvec, NULL); Possibly Guile's optimiser assumes that bytevectors never change in length (needs to be checked). So unless the relevant optimiser code is changed, and it is documented that bytevectors can change in length, I think it would be safer to not have an unmapping procedure in Scheme (though a procedure for remapping it as /dev/zero should be safe). > + bvec =3D scm_c_take_typed_bytevector((signed char *) c_mem + c_offset,= c_len, > + SCM_ARRAY_ELEMENT_TYPE_VU8, pointer); Would scm_pointer_to_bytevector fit here? Also, scm_c_make_typed_bytevecto= r looks like something that can cause an out-of-memory-exception but the finaliser hasn't been set yet. >+// call fstat to get file size >+SCM_DEFINE (scm_mmap_file, "mmap-file", 1, 1, 0,=20 >+ (SCM file, SCM prot), >+ "This procedure accepts a file in the form of filename,\n" >+ " file-port or fd. It returns a bytevector. It must >not\n" >+ " contain scheme allocated objects as it will not be\n" >+ " searched for pointers. Default @var{prot} is=C2=A0@code{\"r= \"}.") I would restrict the C code to only ports and file descriptors, and leave f= ile names to a Scheme wrapper. That way, you automatically get appropriate E..= . errors in case open-file fails (and maybe &i/o-filename etc. if the core Gu= ile FS functions are later changed to R6RS), less chance on accidentally forget= ting to close a fd (*) ... (*) One possible problem: if the file is opened, and mmap fails, then you s= till need to close the file port (and rethrow), so some exception handling is st= ill required, though no C-style exception handling ... > +/* The following copied from bytevectors.c. Kludge? */ > +#define SCM_BYTEVECTOR_SET_LENGTH(_bv, _len) \ > + SCM_SET_CELL_WORD_1 ((_bv), (scm_t_bits) (_len)) > +#define SCM_BYTEVECTOR_SET_CONTENTS(_bv, _contents) \ > + SCM_SET_CELL_WORD_2 ((_bv), (scm_t_bits) (_contents)) To avoid accidental problems if bytevectors.c is modified later, I'd add ad= d a comment in bytevectors.c referring to this file, to add a reminder that mmunmap makes assumptions about the layout. > + scm_c_define ("PAGE_SIZE", scm_from_int (getpagesize())); =46rom local man page: SVr4, 4.4BSD, SUSv2. In SUSv2 the getpagesize() call is labeled = LEGACY, and in POSIX.1-2001 it has been dropped; HP-UX does not have this call. NOTES Portable applications should employ sysconf(_SC_PAGESIZE) instead= of getpage=E2=80=90 size(): Greetings, Maxime. --=-pwo4+W4VnqkAw3mXd0Bv Content-Type: application/pgp-signature; name="signature.asc" Content-Description: This is a digitally signed message part Content-Transfer-Encoding: 7bit -----BEGIN PGP SIGNATURE----- iI0EABYKADUWIQTB8z7iDFKP233XAR9J4+4iGRcl7gUCYrihOxccbWF4aW1lZGV2 b3NAdGVsZW5ldC5iZQAKCRBJ4+4iGRcl7itJAQDLHc8+1tDXptUnHnQViQTR6QK9 ENxt+TgcsOpHm2lkTQEA1Wwq5az6TJcJlL2VNcFubH6+gj/la3hPSYDkh2guowQ= =k30K -----END PGP SIGNATURE----- --=-pwo4+W4VnqkAw3mXd0Bv--