From: Matt Wette <matt.wette@gmail.com>
To: 27782@debbugs.gnu.org, guile-devel@gnu.org
Subject: bug#27782: patch for mmap and friends
Date: Fri, 13 Jan 2023 16:49:50 -0800 [thread overview]
Message-ID: <1ee846ab-e9ce-d616-94dd-0056e4b840f9__15725.49773134$1673657490$gmane$org@gmail.com> (raw)
In-Reply-To: <CD68D357-0F0F-471C-BEC1-EB601844A04E@gmail.com>
[-- Attachment #1: Type: text/plain, Size: 204 bytes --]
Please consider this patch for adding mmap(), munmap() and msync()
to libguile/filesys.c. Included is update for posix.texi and test
file mman.test.
Once included, feature 'mman should be #t.
Matt
[-- Attachment #2: 0001-Add-mmap-and-friends-munmap-msync.patch --]
[-- Type: text/x-patch, Size: 13066 bytes --]
From 6c944174d35d43f87340c8199d47f3f088fa6ca7 Mon Sep 17 00:00:00 2001
From: Matt Wette <mwette@alumni.caltech.edu>
Date: Fri, 13 Jan 2023 16:42:06 -0800
Subject: [PATCH] Add mmap and friends (munmap, msync).
* libguile/filesys.[ch]: added scm_mmap_search, scm_mmap, scm_msync, and
init_mman, built on availability of HAVE_MMAN_H; also provides feature
'mman
* doc/ref/posix.texi: added documentation for mmap and friends
* test-suite/Makefile.am: updated for mman.test
* test-suite/tests/mman.test: mmap tests
---
configure.ac | 2 +
doc/ref/posix.texi | 45 +++++++
libguile/filesys.c | 264 +++++++++++++++++++++++++++++++++++++++++
libguile/filesys.h | 4 +
test-suite/Makefile.am | 1 +
5 files changed, 316 insertions(+)
diff --git a/configure.ac b/configure.ac
index f8c12f0d7..c348d14a2 100644
--- a/configure.ac
+++ b/configure.ac
@@ -1018,6 +1018,8 @@ AC_CHECK_MEMBERS([struct tm.tm_gmtoff],,,
])
GUILE_STRUCT_UTIMBUF
+AC_CHECK_FUNCS([msync])
+
#--------------------------------------------------------------------
#
diff --git a/doc/ref/posix.texi b/doc/ref/posix.texi
index 5653d3758..16f3bbc49 100644
--- a/doc/ref/posix.texi
+++ b/doc/ref/posix.texi
@@ -1216,6 +1216,51 @@ valid separators. Thus, programs should not assume that
separator---e.g., when extracting the components of a file name.
@end defvr
+@deffn {Scheme Procedure} mmap addr len [prot [flags [fd [offset]]]]
+@deffnx {Scheme Procedure} mmap/search addr len [prot [flags [fd [offset]]]]
+Create a memory mapping, returning a bytevector. @var{addr}, if
+non-zero, is the staring address; or, if zero, is assigned by the
+system. @var{prot}, if provided, assigns protection. @var{fd},
+if provided associates the memory region with a file, starting
+at @var{offset}, if provided.
+The region returned by mmap will NOT be searched by the garbage
+ collector for pointers, while that returned by mmap/search will.
+Note that the finalizer for the returned bytevector will call munmap.
+Defaults for optional arguments are
+@table @asis
+@item prot
+(logior PROT_READ PROT_WRITE)
+@item flags
+(logior MAP_ANONYMOUS MAP_PRIVATE)
+@item fd
+-1
+@item offset
+0
+@end table
+@end deffn
+
+@deffn {Scheme Procedure} munmap bvec
+Given bytevector generated by mmap or mmap/search, unmap the
+the associated memory. The argument will be modified to
+reflect a zero length bv. The return value is unspecified.
+Note that munmap is called by finalizer associated with
+bytevectors returned from mmap and mmap/search.
+@end deffn
+
+@deffn {Scheme Procedure} msync addr length flag
+Flush changes made to the in-core copy of a file mapped using
+mmap or mmap/search. This should be executed on modified memory
+before calling munmap. The @var{flags} argument must be exactly one
+of the following:
+@table @code
+@item MS_ASYNC
+Initiate update, return immediately.
+@item MS_SYNC
+Initiate update, block until complete.
+@item MS_INVALIDATE
+Invalidate other mappings of the same file.
+@end table
+@end deffn
@node User Information
@subsection User Information
diff --git a/libguile/filesys.c b/libguile/filesys.c
index 1f0bba556..0ddb4cfee 100644
--- a/libguile/filesys.c
+++ b/libguile/filesys.c
@@ -67,11 +67,17 @@
# include <sys/sendfile.h>
#endif
+#ifdef HAVE_SYS_MMAN_H
+# include <sys/mman.h>
+#endif
+
#include "async.h"
#include "boolean.h"
#include "dynwind.h"
#include "fdes-finalizers.h"
#include "feature.h"
+#include "finalizers.h"
+#include "foreign.h"
#include "fports.h"
#include "gsubr.h"
#include "iselect.h"
@@ -2263,6 +2269,261 @@ scm_dir_free (SCM p)
\f
+#ifdef HAVE_SYS_MMAN_H
+/* see https://pubs.opengroup.org/onlinepubs/9699919799/functions/mmap.html */
+
+static void
+mmap_finalizer (void *ptr, void *data)
+{
+ SCM bvec;
+ void *c_addr;
+ size_t c_len;
+ int rv;
+
+ bvec = SCM_PACK_POINTER (ptr);
+ if (!SCM_BYTEVECTOR_P (bvec))
+ scm_misc_error ("mmap", "expecting bytevector", SCM_EOL);
+
+ c_addr = SCM_BYTEVECTOR_CONTENTS (bvec);
+ c_len = SCM_BYTEVECTOR_LENGTH (bvec);
+ SCM_SYSCALL (rv = munmap(c_addr, c_len));
+ if (rv != 0)
+ scm_misc_error ("mmap", "failed to munmap memory", SCM_EOL);
+}
+
+SCM_DEFINE (scm_mmap_search, "mmap/search", 2, 4, 0,
+ (SCM addr, SCM len, SCM prot, SCM flags, SCM fd, SCM offset),
+ "Create a memory mapping, returning a bytevector.. @var{addr},\n"
+ "if non-zero, is the staring address; or, if zero, is assigned by\n"
+ "the system. @var{prot}, if provided, assigns protection.\n"
+ "@var{fd}, if provided associates the memory region with a file\n"
+ "starting at @var{offset}, if provided.\n"
+ "The region returned by mmap WILL be searched by the garbage\n"
+ "collector for pointers. See also mmap. Note that the\n"
+ "finalizer for the returned bytevector will call munmap.\n"
+ "Defaults for optional arguments are\n"
+ "@table @asis\n"
+ "@item prot\n(logior PROT_READ PROT_WRITE)\n"
+ "@item flags\n(logior MAP_ANONYMOUS MAP_PRIVATE)\n"
+ "@item fd\n-1\n"
+ "@item offset\n0\n"
+ "@end table")
+#define FUNC_NAME s_scm_mmap_search
+{
+ void *c_mem, *c_addr;
+ size_t c_len;
+ int c_prot, c_flags, c_fd;
+ scm_t_off c_offset;
+ SCM pointer, bvec;
+
+ if (SCM_POINTER_P (addr))
+ c_addr = SCM_POINTER_VALUE (addr);
+ else if (scm_is_integer (addr))
+ c_addr = (void*) scm_to_uintptr_t (addr);
+ else
+ scm_misc_error ("mmap", "bad addr", addr);
+
+ c_len = scm_to_size_t (len);
+
+ if (SCM_UNBNDP (prot))
+ c_prot = PROT_READ | PROT_WRITE;
+ else
+ c_prot = scm_to_int (prot);
+
+ if (SCM_UNBNDP (flags))
+ c_flags = MAP_ANONYMOUS | MAP_PRIVATE;
+ else
+ c_flags = scm_to_int (flags);
+
+ if (SCM_UNBNDP (fd))
+ c_fd = -1;
+ else
+ c_fd = scm_to_int (fd);
+
+ if (SCM_UNBNDP (offset))
+ c_offset = 0;
+ else
+ c_offset = scm_to_off_t (offset);
+
+ if ((c_addr == NULL) && (c_flags & MAP_FIXED))
+ scm_misc_error ("mmap", "cannot have NULL addr w/ MAP_FIXED", SCM_EOL);
+
+ SCM_SYSCALL (c_mem = mmap(c_addr, c_len, c_prot, c_flags, c_fd, c_offset));
+ if (c_mem == MAP_FAILED)
+ scm_syserror ("mmap"); /* errno set */
+
+ pointer = scm_cell (scm_tc7_pointer, (scm_t_bits) c_mem);
+ bvec = scm_c_take_typed_bytevector((signed char *) c_mem + c_offset, c_len,
+ SCM_ARRAY_ELEMENT_TYPE_VU8, pointer);
+ assert(sizeof(void*) <= sizeof(size_t));
+ scm_i_set_finalizer (SCM2PTR (bvec), mmap_finalizer, (void*) c_len);
+ return bvec;
+}
+#undef FUNC_NAME
+
+SCM_DEFINE (scm_mmap, "mmap", 2, 4, 0,
+ (SCM addr, SCM len, SCM prot, SCM flags, SCM fd, SCM offset),
+ "Create a memory mapping, returning a bytevector. @var{addr}, if\n"
+ "non-zero, is the staring address; or, if zero, is assigned by the\n"
+ "system. @var{prot}, if provided, assigns protection. @var{fd},\n"
+ "if provided associates the memory region with a file, starting \n"
+ "at @var{offset}, if provided.\n"
+ "The region returned by mmap will NOT be searched by the garbage\n"
+ "collector for pointers. See also mmap/search. Note that the\n"
+ "finalizer for the returned bytevector will call munmap.\n"
+ "Defaults for arguments are:\n"
+ "@table @asis\n"
+ "@item prot\n(logior PROT_READ PROT_WRITE)\n"
+ "@item flags\n(logior MAP_ANONYMOUS MAP_PRIVATE)\n"
+ "@item fd\n-1\n"
+ "@item offset\n0\n"
+ "@end table")
+#define FUNC_NAME s_scm_mmap
+{
+ void *c_mem;
+ size_t c_len;
+ SCM bvec;
+
+ bvec = scm_mmap_search(addr, len, prot, flags, fd, offset);
+ c_mem = SCM_BYTEVECTOR_CONTENTS(bvec);
+ c_len = SCM_BYTEVECTOR_LENGTH(bvec);
+
+ /* Tell GC not to scan for pointers. */
+ GC_exclude_static_roots(c_mem, (char*) c_mem + c_len);
+
+ return bvec;
+}
+#undef FUNC_NAME
+
+/* The following copied from bytevectors.c. Kludge? */
+#define SCM_BYTEVECTOR_SET_LENGTH(_bv, _len) \
+ SCM_SET_CELL_WORD_1 ((_bv), (scm_t_bits) (_len))
+#define SCM_BYTEVECTOR_SET_CONTENTS(_bv, _contents) \
+ SCM_SET_CELL_WORD_2 ((_bv), (scm_t_bits) (_contents))
+
+SCM_DEFINE (scm_munmap, "munmap", 1, 0, 0,
+ (SCM bvec),
+ "Given bytevector generated by mmap or mmap/search, unmap the\n"
+ "the associated memory. The argument will be modified to \n"
+ "reflect a zero length bv. The return value is unspecified.\n"
+ "Note that munmap is called by finalizer associated with\n"
+ "bytevectors returned from mmap and mmap/search.\n")
+#define FUNC_NAME s_scm_munmap
+{
+ void *addr;
+ size_t len;
+ int rv;
+
+ SCM_VALIDATE_BYTEVECTOR (1, bvec);
+
+ addr = (void *) SCM_BYTEVECTOR_CONTENTS (bvec);
+ len = SCM_BYTEVECTOR_LENGTH (bvec);
+
+ /* Invalidate further work on this bytevector. */
+ SCM_BYTEVECTOR_SET_LENGTH (bvec, 0);
+ SCM_BYTEVECTOR_SET_CONTENTS (bvec, NULL);
+
+ SCM_SYSCALL (rv = munmap(addr, len));
+ if (rv == -1)
+ SCM_SYSERROR; /* errno set */
+
+ return SCM_UNSPECIFIED;
+}
+#undef FUNC_NAME
+
+#ifdef HAVE_MSYNC
+SCM_DEFINE (scm_msync, "msync", 2, 0, 0,
+ (SCM bvec, SCM flags),
+ "Flush changes made to the in-core copy of a file mapped using\n"
+ "mmap or mmap/search. This should be executed on modified memory\n"
+ "before calling munmap. The @var{flags} argument must be exactly\n"
+ "one of the following:\n"
+ "@table @code\n"
+ "@item MS_ASYNC\n"
+ "Initiate update, return immediately.\n"
+ "@item MS_SYNC\n"
+ "Initiate update, block until complete.\n"
+ "@item MS_INVALIDATE\n"
+ "Invalidate other mappings of the same file.\n"
+ "@end table\n"
+ "The return value is unspecified.")
+#define FUNC_NAME s_scm_msync
+{
+ void *addr;
+ size_t len;
+ int c_flags, rv;
+
+ SCM_VALIDATE_BYTEVECTOR (1, bvec);
+ addr = (void *) SCM_BYTEVECTOR_CONTENTS (bvec);
+ len = SCM_BYTEVECTOR_LENGTH (bvec);
+
+ c_flags = scm_to_int (flags);
+
+ SCM_SYSCALL (rv = msync(addr, len, c_flags));
+ if (rv == -1)
+ SCM_SYSERROR; /* errno set */
+
+ return SCM_UNSPECIFIED;
+}
+#undef FUNC_NAME
+#endif /* HAVE_MSYNC */
+
+static void init_mman(void) {
+ scm_add_feature("mman");
+
+#ifdef PROT_NONE
+ scm_c_define ("PROT_NONE", scm_from_int (PROT_NONE));
+#endif
+#ifdef PROT_READ
+ scm_c_define ("PROT_READ", scm_from_int (PROT_READ));
+#endif
+#ifdef PROT_WRITE
+ scm_c_define ("PROT_WRITE", scm_from_int (PROT_WRITE));
+#endif
+#ifdef PROT_EXEC
+ scm_c_define ("PROT_EXEC", scm_from_int (PROT_EXEC));
+#endif
+
+#ifdef MAP_ANONYMOUS
+ scm_c_define ("MAP_ANONYMOUS", scm_from_int (MAP_ANONYMOUS));
+#endif
+#ifdef MAP_ANON
+ scm_c_define ("MAP_ANON", scm_from_int (MAP_ANON));
+#endif
+#ifdef MAP_FILE
+ scm_c_define ("MAP_FILE", scm_from_int (MAP_FILE));
+#endif
+#ifdef MAP_FIXED
+ scm_c_define ("MAP_FIXED", scm_from_int (MAP_FIXED));
+#endif
+#ifdef MAP_HASSEMAPHORE
+ scm_c_define ("MAP_HASSEMAPHORE", scm_from_int (MAP_HASSEMAPHORE));
+#endif
+#ifdef MAP_PRIVATE
+ scm_c_define ("MAP_PRIVATE", scm_from_int (MAP_PRIVATE));
+#endif
+#ifdef MAP_SHARED
+ scm_c_define ("MAP_SHARED", scm_from_int (MAP_SHARED));
+#endif
+#ifdef MAP_NOCACHE
+ scm_c_define ("MAP_NOCACHE", scm_from_int (MAP_NOCACHE));
+#endif
+ scm_c_define ("PAGE_SIZE", scm_from_int (getpagesize()));
+#ifdef MS_ASYNC
+ scm_c_define ("MS_ASYNC", scm_from_int (MS_ASYNC));
+#endif
+#ifdef MS_SYNC
+ scm_c_define ("MS_SYNC", scm_from_int (MS_SYNC));
+#endif
+#ifdef MS_INVALIDATE
+ scm_c_define ("MS_INVALIDATE", scm_from_int (MS_INVALIDATE));
+#endif
+}
+
+#endif /* HAVE_SYS_MMAN_H */
+
+\f
+
void
scm_init_filesys ()
{
@@ -2387,6 +2648,9 @@ scm_init_filesys ()
#ifdef HAVE_READLINKAT
scm_add_feature("readlink-port");
#endif
+#if defined(HAVE_SYS_MMAN_H)
+ init_mman();
+#endif
#include "filesys.x"
}
diff --git a/libguile/filesys.h b/libguile/filesys.h
index 1ce50d30e..fa40b484f 100644
--- a/libguile/filesys.h
+++ b/libguile/filesys.h
@@ -80,6 +80,10 @@ SCM_API SCM scm_dirname (SCM filename);
SCM_API SCM scm_basename (SCM filename, SCM suffix);
SCM_API SCM scm_canonicalize_path (SCM path);
SCM_API SCM scm_sendfile (SCM out, SCM in, SCM count, SCM offset);
+SCM_API SCM scm_mmap_search(SCM addr, SCM len, SCM prot, SCM flags, SCM fd, SCM offset);
+SCM_API SCM scm_mmap(SCM addr, SCM len, SCM prot, SCM flags, SCM fd, SCM offset);
+SCM_API SCM scm_msync(SCM bvec, SCM flags);
+SCM_API SCM scm_munmap(SCM bvec);
SCM_INTERNAL SCM scm_i_relativize_path (SCM path, SCM in_path);
SCM_INTERNAL void scm_init_filesys (void);
diff --git a/test-suite/Makefile.am b/test-suite/Makefile.am
index 16fa2e952..3785e2f85 100644
--- a/test-suite/Makefile.am
+++ b/test-suite/Makefile.am
@@ -77,6 +77,7 @@ SCM_TESTS = tests/00-initial-env.test \
tests/load.test \
tests/match.test \
tests/match.test.upstream \
+ tests/mman.test \
tests/modules.test \
tests/multilingual.nottest \
tests/net-db.test \
--
2.34.1
next prev parent reply other threads:[~2023-01-14 0:49 UTC|newest]
Thread overview: 19+ messages / expand[flat|nested] mbox.gz Atom feed top
2017-07-21 13:39 bug#27782: [wishlist] scheme level mmap Matt Wette
[not found] ` <handler.27782.B.150064439025677.ack@debbugs.gnu.org>
2017-07-21 14:35 ` bug#27782: Acknowledgement ([wishlist] scheme level mmap) Matt Wette
2017-10-28 15:25 ` bug#27782: mmap for guile 2.2.2 Matt Wette
2017-10-28 17:09 ` Matt Wette
2017-11-24 15:54 ` bug#27782: mmap for guile Matt Wette
2017-11-24 16:22 ` Nala Ginrut
2017-11-24 17:09 ` Matt Wette
2017-11-25 14:41 ` Matt Wette
2017-11-25 16:17 ` Nala Ginrut
2020-07-04 19:40 ` bug#27782: new patch for mma Matt Wette
2020-07-09 12:45 ` Ludovic Courtès
2022-12-21 1:21 ` bug#27782: patch to add support for mmap and friends Matt Wette
2022-12-22 18:49 ` Matt Wette
2023-01-14 0:49 ` Matt Wette [this message]
2023-02-14 14:50 ` bug#27782: mman patch for v3.0.9 Matt Wette
2023-03-01 13:31 ` Matt Wette
[not found] <1ee846ab-e9ce-d616-94dd-0056e4b840f9@gmail.com>
2023-01-14 1:00 ` bug#27782: patch for mmap and friends Matt Wette
2023-01-14 15:18 ` Maxime Devos
[not found] ` <445d3567-9bbf-487b-f338-8a16903e9e62@telenet.be>
[not found] ` <5fda49f2-6e23-9a53-85d2-c1cc38cf0cce@gmail.com>
[not found] ` <c7e20254-6387-00c4-ea15-f3ed64668923@gmail.com>
[not found] ` <23890f8a-8891-d888-b289-c0d06304fff1@telenet.be>
2023-01-14 23:46 ` Matt Wette
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: https://www.gnu.org/software/guile/
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to='1ee846ab-e9ce-d616-94dd-0056e4b840f9__15725.49773134$1673657490$gmane$org@gmail.com' \
--to=matt.wette@gmail.com \
--cc=27782@debbugs.gnu.org \
--cc=guile-devel@gnu.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).