[PATCH] Append effective version to GUILE_LOAD[_COMPILED]_PATH

[PATCH] Append effective version to GUILE_LOAD[_COMPILED]_PATH

[Jan Nieuwenhuizen]

[-- Attachment #1: Type: text/plain, Size: 3117 bytes --]

Hi,

I am running guile-2.0 and guile-2.2 alongside each other which is
causing me some pain*).

This is what bits of my GUILE_LOAD_COMPILED_PATH look like

    /gnu/store/7ml4psifv46pzxjxw56xfl7mwd47k277-profile/lib/guile/2.2/ccache
    --> /gnu/store/7ml4psifv46pzxjxw56xfl7mwd47k277-profile/lib/guile/2.2/ccache/ice-9/and-let-star.go

    /gnu/store/7ml4psifv46pzxjxw56xfl7mwd47k277-profile/share/guile/site/2.2/
    --> /gnu/store/7ml4psifv46pzxjxw56xfl7mwd47k277-profile/share/guile/site/2.2/os/process.go

If `/<effective-version>' is always used as the suffix of each path
element, and we/guix/packagers do not include that suffix in GUILE_*PATH
elements, then Guile can append effective-prefix and different major
Guile-versions can happily share the same GUILE_LOAD[_COMPILED]_PATH,
e.g., having

    GUILE_LOAD_COMPILED_PATH=/gnu/store/7ml4psifv46pzxjxw56xfl7mwd47k277-profile/share/guile/site

then guile-2.0 would get (os process) from

    /gnu/store/7ml4psifv46pzxjxw56xfl7mwd47k277-profile/share/guile/site + /2.0
    --> /gnu/store/7ml4psifv46pzxjxw56xfl7mwd47k277-profile/share/guile/site/2.0/os/process.go

and guile-2.2 would read

    /gnu/store/7ml4psifv46pzxjxw56xfl7mwd47k277-profile/share/guile/site + /2.2
    --> /gnu/store/7ml4psifv46pzxjxw56xfl7mwd47k277-profile/share/guile/site/2.2/os/process.go
    
What do you think?  No more pain!  Find patch attached.

Greetings,
Jan


*) Some of my pain

My Debian host system has guile-2.0, guix depends on guile-2.0, guix's
LD script depends on guile-2.0, the guile bits of my project depend on
guile-2.2.

I have some scripts to make this situation almost bearable, but still I
regularly

cannot find a basic library

    [1]13:53:25 janneke@janneke-ijzer:~
    $ guile --no-auto-compile
    GNU Guile 2.0.11
    Copyright (C) 1995-2014 Free Software Foundation, Inc.

    Guile comes with ABSOLUTELY NO WARRANTY; for details type `,show w'.
    This program is free software, and you are welcome to redistribute it
    under certain conditions; type `,show c' for details.

    Enter `,help' for help.
    scheme@(guile-user)> (use-modules (os proccess))
    While compiling expression:
    ERROR: no code for module (os proccess)

or some guile script (guix) aborts

    guix environment --ad-hoc ccache coreutils git guix emacs guile-next guile-next-lib
    Throw without catch before boot:
    Throw to key misc-error with args ("make_objcode_from_file" "bad header on object file: ~s" ("\x7fELF\x02\x01\x01\x00\x00\x00\x00\x00\x00\x00\x00") #f)Aborting.
    Aborted

or linking breaks

    g++ -Wall -std=c++11  -g -I. -I out/alarm.project/c++ -I check/alarm.project/   out/alarm.project/c++/main.o out/alarm.project/c++/Alarm.o out/alarm.project/c++/AlarmSystem.o out/alarm.project/c++/pump.o out/alarm.project/c++/runtime.o    -lboost_system -lboost_coroutine -lboost_thread -lboost_context -pthread -o out/alarm.project/c++/test
    collect2: error: ld terminated with signal 6 [Afgebroken]
    Throw without catch before boot:
    Aborting.

and then I juggle installed guile versions and/or manually modify
GUILE_LOAD_COMPILED_PATH.


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-Append-effective-version-to-GUILE_LOAD-_COMPILED-_PA.patch --]
[-- Type: text/x-diff, Size: 3192 bytes --]

From 81c8b35752235aeb26058c0baadd0ea58046f640 Mon Sep 17 00:00:00 2001
From: Jan Nieuwenhuizen <janneke@gnu.org>
Date: Thu, 3 Mar 2016 18:18:55 +0100
Subject: [PATCH] Append effective version to GUILE_LOAD[_COMPILED]_PATH
 elements.

* libguile/load.c (scm_path_append_effective_version): New function
  (scm_init_load_path): Use it.

* am/guilec (moddir,ccachedir): Use GUILE_EFFECTIVE_VERSION as prefix.
---
 am/guilec       |  4 ++--
 libguile/load.c | 27 ++++++++++++++++++++++++++-
 2 files changed, 28 insertions(+), 3 deletions(-)

diff --git a/am/guilec b/am/guilec
index 5ef07fa..fa2b5ba 100644
--- a/am/guilec
+++ b/am/guilec
@@ -3,9 +3,9 @@ GOBJECTS = $(SOURCES:%.scm=%.go) $(ELISP_SOURCES:%.el=%.go)
 
 GUILE_WARNINGS = -Wunbound-variable -Warity-mismatch -Wformat
 
-moddir = $(pkgdatadir)/$(GUILE_EFFECTIVE_VERSION)/$(modpath)
+moddir = $(pkgdatadir)/$(modpath)/$(GUILE_EFFECTIVE_VERSION)
 nobase_mod_DATA = $(SOURCES) $(ELISP_SOURCES) $(NOCOMP_SOURCES)
-ccachedir = $(pkglibdir)/$(GUILE_EFFECTIVE_VERSION)/ccache/$(modpath)
+ccachedir = $(pkglibdir)/ccache/$(GUILE_EFFECTIVE_VERSION)/$(modpath)
 nobase_ccache_DATA = $(GOBJECTS)
 EXTRA_DIST = $(SOURCES) $(ELISP_SOURCES) $(NOCOMP_SOURCES)
 ETAGS_ARGS = $(SOURCES) $(ELISP_SOURCES) $(NOCOMP_SOURCES)
diff --git a/libguile/load.c b/libguile/load.c
index d26f9fc..055f3c4 100644
--- a/libguile/load.c
+++ b/libguile/load.c
@@ -1,5 +1,6 @@
 /* Copyright (C) 1995, 1996, 1998, 1999, 2000, 2001, 2004, 2006, 2008,
- *   2009, 2010, 2011, 2012, 2013, 2014 Free Software Foundation, Inc.
+ *   2009, 2010, 2011, 2012, 2013, 2014, 2016 Free Software Foundation,
+ *   Inc.
  * 
  * This library is free software; you can redistribute it and/or
  * modify it under the terms of the GNU Lesser General Public License
@@ -312,6 +313,28 @@ scm_i_mirror_backslashes (char *path)
   return path;
 }
 
+/* Append effective_version to each element in PATH.  This allows using
+   one single heterogeneous path for different effective versions of
+   Guile, i.e., running different versions of guile alongside each other
+   from one shell, without juggling GUILE_LOAD_COMPILED_PATH.  */
+static SCM
+scm_path_append_effective_version (SCM path)
+{
+  SCM vpath = SCM_EOL;
+  SCM version = scm_effective_version ();
+  SCM slash = scm_from_utf8_string ("/");
+  for (path = scm_reverse (path); scm_is_pair (path);
+       path = SCM_CDR (path))
+    {
+      SCM dir = SCM_CAR (path);
+      SCM vdir = scm_string_append (scm_list_3 (dir, slash, version));
+      /* keep non-M.M suffixed dir as fallback for compatibility.  */
+      vpath = scm_cons (dir, vpath);
+      vpath = scm_cons (vdir, vpath);
+    }
+  return vpath;
+}
+
 /* Initialize the global variable %load-path, given the value of the
    SCM_SITE_DIR and SCM_LIBRARY_DIR preprocessor symbols and the
    GUILE_LOAD_PATH environment variable.  */
@@ -393,7 +416,9 @@ scm_init_load_path ()
   if (env)
     cpath = scm_parse_path_with_ellipsis (scm_from_locale_string (env), cpath);
 
+  path = scm_path_append_effective_version (path);
   *scm_loc_load_path = path;
+  cpath = scm_path_append_effective_version (cpath);
   *scm_loc_load_compiled_path = cpath;
 }
 
-- 
2.6.3


[-- Attachment #3: Type: text/plain, Size: 154 bytes --]


-- 
Jan Nieuwenhuizen <janneke@gnu.org> | GNU LilyPond http://lilypond.org
Freelance IT http://JoyofSource.com | Avatar®  http://AvatarAcademy.nl  

[PATCH] Skip invalid compiled file found, continue searching path.

[Jan Nieuwenhuizen]

[-- Attachment #1: Type: text/plain, Size: 281 bytes --]

Hi,

As per chat with Ludovic on #guile (thanks!) find attached another
approach to allow switching incrementally from guile-2.0 to guile-2.2:
any invalid compiled files are skipped and we continue searching the
GUILE_LOAD_COMPILED_PATH instead of throwing early.

Greetings,
Jan


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-Skip-invalid-compiled-file-found-continue-searching-.patch --]
[-- Type: text/x-diff, Size: 9303 bytes --]

From f4f53b48c1d5ff42ecc66279c3b1cfcfb09d6757 Mon Sep 17 00:00:00 2001
From: Jan Nieuwenhuizen <janneke@gnu.org>
Date: Fri, 11 Mar 2016 14:58:09 +0100
Subject: [PATCH] Skip invalid compiled file found, continue searching path.

* libguile/vm.c (load_compiled_with_vm_catch_handler,
  do_try_scm_call_0): New static function.
  (scm_load_compiled_with_vm): Use them to implement not throwing
  if new argument EXCEPTION_ON_NOT_FOUND_P is not SCM_BOOL_TRUE.
* libguile/load.c (search_path): Take optional output argument
  PATH_REMAINING.
  (scm_primitive_load_path): Use it.  Take optional argument
  LOAD_COMPILED_PATH.  Skip any invalid compiled file found and
  continue searching scm_loc_load_compiled_path.
---
 libguile/load.c | 55 ++++++++++++++++++++++++++++++++++++++++---------------
 libguile/vm.c   | 27 ++++++++++++++++++++++++---
 libguile/vm.h   |  2 +-
 3 files changed, 65 insertions(+), 19 deletions(-)

diff --git a/libguile/load.c b/libguile/load.c
index d26f9fc..a6b87cf 100644
--- a/libguile/load.c
+++ b/libguile/load.c
@@ -582,12 +582,16 @@ compiled_is_fresh (SCM full_filename, SCM compiled_filename,
    file name that we find in the path.  Otherwise only return a file if
    it is newer than SOURCE_STAT_BUF, otherwise issuing a warning if we
    see a stale file earlier in the path, setting *FOUND_STALE_FILE to 1.
+
+   If PATH_REMAINING is not NULL, it is set to the part of PATH that was
+   not yet searched.
   */
 static SCM
 search_path (SCM path, SCM filename, SCM extensions, SCM require_exts,
              struct stat *stat_buf,
              SCM source_file_name, struct stat *source_stat_buf,
-             int *found_stale_file)
+             int *found_stale_file,
+             SCM *path_remaining)
 {
   struct stringbuf buf;
   char *filename_chars;
@@ -724,6 +728,8 @@ search_path (SCM path, SCM filename, SCM extensions, SCM require_exts,
 
  end:
   scm_dynwind_end ();
+  if (path_remaining)
+    path_remaining = &path;
   return result;
 }
 
@@ -781,7 +787,7 @@ SCM_DEFINE (scm_search_path, "search-path", 2, 0, 1,
     require_exts = SCM_BOOL_F;
 
   return search_path (path, filename, extensions, require_exts, &stat_buf,
-                      SCM_BOOL_F, NULL, NULL);
+                      SCM_BOOL_F, NULL, NULL, NULL);
 }
 #undef FUNC_NAME
 
@@ -806,7 +812,7 @@ SCM_DEFINE (scm_sys_search_load_path, "%search-load-path", 1, 0, 0,
   SCM_VALIDATE_STRING (1, filename);
 
   return search_path (*scm_loc_load_path, filename, *scm_loc_load_extensions,
-                      SCM_BOOL_F, &stat_buf, SCM_BOOL_F, NULL, NULL);
+                      SCM_BOOL_F, &stat_buf, SCM_BOOL_F, NULL, NULL, NULL);
 }
 #undef FUNC_NAME
 
@@ -969,14 +975,19 @@ SCM_DEFINE (scm_primitive_load_path, "primitive-load-path", 0, 0, 1,
             "depending on the optional second argument,\n"
             "@var{exception_on_not_found}.  If it is @code{#f}, @code{#f}\n"
             "will be returned.  If it is a procedure, it will be called\n"
-            "with no arguments.  Otherwise an error is signalled.")
+            "with no arguments.  Otherwise an error is signalled."
+            "If the optional third argument,\n"
+            "@var{load_compiled_path} is given, use it to search for compiled files\n"
+            "instead of @var{*scm_loc_load_compiled_path}.")
 #define FUNC_NAME s_scm_primitive_load_path
 {
   SCM filename, exception_on_not_found;
   SCM full_filename, compiled_filename;
   SCM hook = *scm_loc_load_hook;
   struct stat stat_source, stat_compiled;
+  SCM load_compiled_path;
   int found_stale_compiled_file = 0;
+  SCM load_compiled_path_remaining = SCM_EOL;
 
   if (scm_is_true (hook) && scm_is_false (scm_procedure_p (hook)))
     SCM_MISC_ERROR ("value of %load-hook is neither a procedure nor #f",
@@ -988,21 +999,27 @@ SCM_DEFINE (scm_primitive_load_path, "primitive-load-path", 0, 0, 1,
 	 single argument (the file name).  */
       filename = args;
       exception_on_not_found = SCM_UNDEFINED;
+      load_compiled_path = *scm_loc_load_compiled_path;
     }
   else
     {
-      /* Starting from 1.9, this function takes 1 required and 1 optional
-	 argument.  */
+      /* Starting from 1.9, this function takes 1 required and 1
+	 optional argument.
+
+         Starting from 2.1.2, this function takes 1 required and 2
+	 optional arguments.  */
       long len;
 
       SCM_VALIDATE_LIST_COPYLEN (SCM_ARG1, args, len);
-      if (len < 1 || len > 2)
+      if (len < 1 || len > 3)
 	scm_error_num_args_subr (FUNC_NAME);
 
       filename = SCM_CAR (args);
       SCM_VALIDATE_STRING (SCM_ARG1, filename);
 
       exception_on_not_found = len > 1 ? SCM_CADR (args) : SCM_UNDEFINED;
+      load_compiled_path = len < 3 ? *scm_loc_load_compiled_path
+        : SCM_CADDR (args);
     }
 
   if (SCM_UNBNDP (exception_on_not_found))
@@ -1010,13 +1027,13 @@ SCM_DEFINE (scm_primitive_load_path, "primitive-load-path", 0, 0, 1,
 
   full_filename = search_path (*scm_loc_load_path, filename,
                                *scm_loc_load_extensions, SCM_BOOL_F,
-                               &stat_source, SCM_BOOL_F, NULL, NULL);
+                               &stat_source, SCM_BOOL_F, NULL, NULL, NULL);
 
   compiled_filename =
-    search_path (*scm_loc_load_compiled_path, filename,
+    search_path (load_compiled_path, filename,
                  *scm_loc_load_compiled_extensions, SCM_BOOL_T,
                  &stat_compiled, full_filename, &stat_source,
-                 &found_stale_compiled_file);
+                 &found_stale_compiled_file, &load_compiled_path_remaining);
 
   if (scm_is_false (compiled_filename)
       && scm_is_true (full_filename)
@@ -1066,13 +1083,21 @@ SCM_DEFINE (scm_primitive_load_path, "primitive-load-path", 0, 0, 1,
                        ? full_filename : compiled_filename));
 
   if (scm_is_true (compiled_filename))
-    return scm_load_compiled_with_vm (compiled_filename);
+    {
+      SCM module = scm_load_compiled_with_vm (compiled_filename, SCM_BOOL_T);
+      if (module != SCM_BOOL_F || exception_on_not_found != SCM_BOOL_T)
+        return module;
+      return scm_primitive_load_path (scm_list_3
+                                      (filename,
+                                       exception_on_not_found,
+                                       load_compiled_path_remaining));
+    }
   else
     {
       SCM freshly_compiled = scm_try_auto_compile (full_filename);
 
       if (scm_is_true (freshly_compiled))
-        return scm_load_compiled_with_vm (freshly_compiled);
+        return scm_load_compiled_with_vm (freshly_compiled, SCM_BOOL_T);
       else
         return scm_primitive_load (full_filename);
     }
@@ -1095,14 +1120,14 @@ scm_init_eval_in_scheme (void)
   eval_scm = search_path (*scm_loc_load_path,
                           scm_from_locale_string ("ice-9/eval.scm"),
                           SCM_EOL, SCM_BOOL_F, &stat_source,
-                          SCM_BOOL_F, NULL, NULL);
+                          SCM_BOOL_F, NULL, NULL, NULL);
   eval_go = search_path (*scm_loc_load_compiled_path,
                          scm_from_locale_string ("ice-9/eval.go"),
                          SCM_EOL, SCM_BOOL_F, &stat_compiled,
-                         eval_scm, &stat_source, &found_stale_eval_go);
+                         eval_scm, &stat_source, &found_stale_eval_go, NULL);
   
   if (scm_is_true (eval_go))
-    scm_load_compiled_with_vm (eval_go);
+    scm_load_compiled_with_vm (eval_go, SCM_BOOL_T);
   else
     /* If we have no eval.go, we shouldn't load any compiled code at all
        because we can't guarantee that tail calls will work.  */
diff --git a/libguile/vm.c b/libguile/vm.c
index 33f12b4..b560ee5 100644
--- a/libguile/vm.c
+++ b/libguile/vm.c
@@ -1500,10 +1500,31 @@ SCM_DEFINE (scm_call_with_stack_overflow_handler,
  * Initialize
  */
 
-SCM
-scm_load_compiled_with_vm (SCM file)
+static SCM
+load_compiled_with_vm_catch_handler (void *data, SCM tag, SCM throw_args)
+{
+  return SCM_BOOL_F;
+}
+
+static SCM
+do_try_scm_call_0 (void *data)
 {
-  return scm_call_0 (scm_load_thunk_from_file (file));
+  SCM thunk = SCM_PACK_POINTER (data);
+  return scm_call_0 (thunk);
+}
+
+SCM
+scm_load_compiled_with_vm (SCM file, SCM exception_on_not_found_p)
+{
+  SCM thunk = scm_load_thunk_from_file (file);
+  if (exception_on_not_found_p == SCM_BOOL_F)
+    return scm_c_catch (SCM_BOOL_T,
+                        do_try_scm_call_0,
+                        SCM_UNPACK_POINTER (thunk),
+                        load_compiled_with_vm_catch_handler,
+                        SCM_UNPACK_POINTER (thunk),
+                        NULL, NULL);
+  return scm_call_0 (thunk);
 }
 
   
diff --git a/libguile/vm.h b/libguile/vm.h
index 2ca4f2a..d76ac31 100644
--- a/libguile/vm.h
+++ b/libguile/vm.h
@@ -93,7 +93,7 @@ struct scm_vm_cont {
 #define SCM_VM_CONT_PARTIAL_P(CONT) (SCM_VM_CONT_DATA (CONT)->flags & SCM_F_VM_CONT_PARTIAL)
 #define SCM_VM_CONT_REWINDABLE_P(CONT) (SCM_VM_CONT_DATA (CONT)->flags & SCM_F_VM_CONT_REWINDABLE)
 
-SCM_API SCM scm_load_compiled_with_vm (SCM file);
+SCM_API SCM scm_load_compiled_with_vm (SCM file, SCM exception_on_not_found_p);
 
 SCM_INTERNAL SCM scm_i_call_with_current_continuation (SCM proc);
 SCM_INTERNAL SCM scm_i_capture_current_stack (void);
-- 
2.6.3


[-- Attachment #3: Type: text/plain, Size: 154 bytes --]


-- 
Jan Nieuwenhuizen <janneke@gnu.org> | GNU LilyPond http://lilypond.org
Freelance IT http://JoyofSource.com | Avatar®  http://AvatarAcademy.nl  

Re: [PATCH] Skip invalid compiled file found, continue searching path.

[David Kastrup]

Jan Nieuwenhuizen <janneke@gnu.org> writes:

> Hi,
>
> As per chat with Ludovic on #guile (thanks!) find attached another
> approach to allow switching incrementally from guile-2.0 to guile-2.2:
> any invalid compiled files are skipped and we continue searching the
> GUILE_LOAD_COMPILED_PATH instead of throwing early.

Shouldn't we worry about switching _to_ guile-2.0 first?

-- 
David Kastrup

Re: [PATCH] Skip invalid compiled file found, continue searching path.

[Jan Nieuwenhuizen]

David Kastrup writes:

Hi David,

> Shouldn't we worry about switching _to_ guile-2.0 first?

Are you asking for help on this?

In any case, I am sure that running guix with nicely coexisting
guile-2.0, guile-2.0++ and guile-next versions can be of great help
here.

Greetings,
Jan

-- 
Jan Nieuwenhuizen <janneke@gnu.org> | GNU LilyPond http://lilypond.org
Freelance IT http://JoyofSource.com | Avatar®  http://AvatarAcademy.nl  

Re: [PATCH] Skip invalid compiled file found, continue searching path.

[David Kastrup]

Jan Nieuwenhuizen <janneke@gnu.org> writes:

> David Kastrup writes:
>
> Hi David,
>
>> Shouldn't we worry about switching _to_ guile-2.0 first?
>
> Are you asking for help on this?

Is there any to be had?  I got a whole lot of promises from Guile
developers over the years.  Basically, whenever you write up the status
quo and directions how to work with the current state, possibly
preparing a branch with ongoing work for them, that's the last you hear
of them.

The last round of "this is really important, let's do it" even included
Richard Stallman and Andy Wingo.  It went dead like all the previous
ones.  Everybody is willing to state how important it is but nobody ever
does anything except me.  The only thing that ever gets progress are
very distilled bug reports not involving any LilyPond code.  But since
the last Guile-2.0 release was 2.0.11 in March 2014, bug fixes don't
make it into usable releases anyway, and if we have a turnaround time of
2 years for any serious bug, we will not be getting anywhere soon.

-- 
David Kastrup

Re: [PATCH] Skip invalid compiled file found, continue searching path.

[David Kastrup]

David Kastrup <dak@gnu.org> writes:

[...]

I apologize for sending this and the previous mail to the Guile-devel
list in violation of my ban here.  I had been of the impression that
this was copied to the LilyPond developer list rather than the Guile
developer list.

Feel free to delete it.

-- 
David Kastrup

[PATCH v3] Skip invalid compiled file found, continue searching path.

[Jan Nieuwenhuizen]

[-- Attachment #1: Type: text/plain, Size: 1189 bytes --]

Hi,

Only after rebuilding world, I found this morning that the previous
patch did not exactly work as intended.  Find version 3 of this patch
attached.

Instead of having scm_call_0 possibly throw and catching that in
vm.c:(scm_load_compiled_with_vm), now the FALSE_ON_ERROR flag is
propagated down to loader.c: (scm_load_thunk_from_file,
scm_load_thunk_from_memory, map_file_contents).

To test this, do something like

    mkdir -p ~/broken/srfi
    cp /gnu/store/b44k6wx7nxn0kh2kchlazgas2ybc576v-guile-2.0.11/lib/guile/2.0/ccache/srfi/srfi-98.go ~/broken/srfi
    touch ~/broken/srfi/srfi-111.go
    export GUILE_DEBUG_GO=1
    touch ~/broken/srfi/*.go

meta/guile --no-auto-compile -s <(echo '
(set! %load-compiled-path
  (cons (string-append (getenv "HOME") "/broken") %load-compiled-path))
(use-modules (srfi srfi-98))
(display (get-environment-variable "GUILE_DEBUG_GO"))
(newline)
')

    meta/guile --no-auto-compile -s <(echo '
(set! %load-compiled-path
  (cons (string-append (getenv "HOME") "/broken") %load-compiled-path))
(use-modules (srfi srfi-111))
(display box)
(newline)
')

Now, on to backporting this to stable-2.0 and rebuilding world again,
twice.

Greetings,
Jan


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-Skip-invalid-compiled-file-found-continue-searching-.patch --]
[-- Type: text/x-diff, Size: 9303 bytes --]

From f4f53b48c1d5ff42ecc66279c3b1cfcfb09d6757 Mon Sep 17 00:00:00 2001
From: Jan Nieuwenhuizen <janneke@gnu.org>
Date: Fri, 11 Mar 2016 14:58:09 +0100
Subject: [PATCH] Skip invalid compiled file found, continue searching path.

* libguile/vm.c (load_compiled_with_vm_catch_handler,
  do_try_scm_call_0): New static function.
  (scm_load_compiled_with_vm): Use them to implement not throwing
  if new argument EXCEPTION_ON_NOT_FOUND_P is not SCM_BOOL_TRUE.
* libguile/load.c (search_path): Take optional output argument
  PATH_REMAINING.
  (scm_primitive_load_path): Use it.  Take optional argument
  LOAD_COMPILED_PATH.  Skip any invalid compiled file found and
  continue searching scm_loc_load_compiled_path.
---
 libguile/load.c | 55 ++++++++++++++++++++++++++++++++++++++++---------------
 libguile/vm.c   | 27 ++++++++++++++++++++++++---
 libguile/vm.h   |  2 +-
 3 files changed, 65 insertions(+), 19 deletions(-)

diff --git a/libguile/load.c b/libguile/load.c
index d26f9fc..a6b87cf 100644
--- a/libguile/load.c
+++ b/libguile/load.c
@@ -582,12 +582,16 @@ compiled_is_fresh (SCM full_filename, SCM compiled_filename,
    file name that we find in the path.  Otherwise only return a file if
    it is newer than SOURCE_STAT_BUF, otherwise issuing a warning if we
    see a stale file earlier in the path, setting *FOUND_STALE_FILE to 1.
+
+   If PATH_REMAINING is not NULL, it is set to the part of PATH that was
+   not yet searched.
   */
 static SCM
 search_path (SCM path, SCM filename, SCM extensions, SCM require_exts,
              struct stat *stat_buf,
              SCM source_file_name, struct stat *source_stat_buf,
-             int *found_stale_file)
+             int *found_stale_file,
+             SCM *path_remaining)
 {
   struct stringbuf buf;
   char *filename_chars;
@@ -724,6 +728,8 @@ search_path (SCM path, SCM filename, SCM extensions, SCM require_exts,
 
  end:
   scm_dynwind_end ();
+  if (path_remaining)
+    path_remaining = &path;
   return result;
 }
 
@@ -781,7 +787,7 @@ SCM_DEFINE (scm_search_path, "search-path", 2, 0, 1,
     require_exts = SCM_BOOL_F;
 
   return search_path (path, filename, extensions, require_exts, &stat_buf,
-                      SCM_BOOL_F, NULL, NULL);
+                      SCM_BOOL_F, NULL, NULL, NULL);
 }
 #undef FUNC_NAME
 
@@ -806,7 +812,7 @@ SCM_DEFINE (scm_sys_search_load_path, "%search-load-path", 1, 0, 0,
   SCM_VALIDATE_STRING (1, filename);
 
   return search_path (*scm_loc_load_path, filename, *scm_loc_load_extensions,
-                      SCM_BOOL_F, &stat_buf, SCM_BOOL_F, NULL, NULL);
+                      SCM_BOOL_F, &stat_buf, SCM_BOOL_F, NULL, NULL, NULL);
 }
 #undef FUNC_NAME
 
@@ -969,14 +975,19 @@ SCM_DEFINE (scm_primitive_load_path, "primitive-load-path", 0, 0, 1,
             "depending on the optional second argument,\n"
             "@var{exception_on_not_found}.  If it is @code{#f}, @code{#f}\n"
             "will be returned.  If it is a procedure, it will be called\n"
-            "with no arguments.  Otherwise an error is signalled.")
+            "with no arguments.  Otherwise an error is signalled."
+            "If the optional third argument,\n"
+            "@var{load_compiled_path} is given, use it to search for compiled files\n"
+            "instead of @var{*scm_loc_load_compiled_path}.")
 #define FUNC_NAME s_scm_primitive_load_path
 {
   SCM filename, exception_on_not_found;
   SCM full_filename, compiled_filename;
   SCM hook = *scm_loc_load_hook;
   struct stat stat_source, stat_compiled;
+  SCM load_compiled_path;
   int found_stale_compiled_file = 0;
+  SCM load_compiled_path_remaining = SCM_EOL;
 
   if (scm_is_true (hook) && scm_is_false (scm_procedure_p (hook)))
     SCM_MISC_ERROR ("value of %load-hook is neither a procedure nor #f",
@@ -988,21 +999,27 @@ SCM_DEFINE (scm_primitive_load_path, "primitive-load-path", 0, 0, 1,
 	 single argument (the file name).  */
       filename = args;
       exception_on_not_found = SCM_UNDEFINED;
+      load_compiled_path = *scm_loc_load_compiled_path;
     }
   else
     {
-      /* Starting from 1.9, this function takes 1 required and 1 optional
-	 argument.  */
+      /* Starting from 1.9, this function takes 1 required and 1
+	 optional argument.
+
+         Starting from 2.1.2, this function takes 1 required and 2
+	 optional arguments.  */
       long len;
 
       SCM_VALIDATE_LIST_COPYLEN (SCM_ARG1, args, len);
-      if (len < 1 || len > 2)
+      if (len < 1 || len > 3)
 	scm_error_num_args_subr (FUNC_NAME);
 
       filename = SCM_CAR (args);
       SCM_VALIDATE_STRING (SCM_ARG1, filename);
 
       exception_on_not_found = len > 1 ? SCM_CADR (args) : SCM_UNDEFINED;
+      load_compiled_path = len < 3 ? *scm_loc_load_compiled_path
+        : SCM_CADDR (args);
     }
 
   if (SCM_UNBNDP (exception_on_not_found))
@@ -1010,13 +1027,13 @@ SCM_DEFINE (scm_primitive_load_path, "primitive-load-path", 0, 0, 1,
 
   full_filename = search_path (*scm_loc_load_path, filename,
                                *scm_loc_load_extensions, SCM_BOOL_F,
-                               &stat_source, SCM_BOOL_F, NULL, NULL);
+                               &stat_source, SCM_BOOL_F, NULL, NULL, NULL);
 
   compiled_filename =
-    search_path (*scm_loc_load_compiled_path, filename,
+    search_path (load_compiled_path, filename,
                  *scm_loc_load_compiled_extensions, SCM_BOOL_T,
                  &stat_compiled, full_filename, &stat_source,
-                 &found_stale_compiled_file);
+                 &found_stale_compiled_file, &load_compiled_path_remaining);
 
   if (scm_is_false (compiled_filename)
       && scm_is_true (full_filename)
@@ -1066,13 +1083,21 @@ SCM_DEFINE (scm_primitive_load_path, "primitive-load-path", 0, 0, 1,
                        ? full_filename : compiled_filename));
 
   if (scm_is_true (compiled_filename))
-    return scm_load_compiled_with_vm (compiled_filename);
+    {
+      SCM module = scm_load_compiled_with_vm (compiled_filename, SCM_BOOL_T);
+      if (module != SCM_BOOL_F || exception_on_not_found != SCM_BOOL_T)
+        return module;
+      return scm_primitive_load_path (scm_list_3
+                                      (filename,
+                                       exception_on_not_found,
+                                       load_compiled_path_remaining));
+    }
   else
     {
       SCM freshly_compiled = scm_try_auto_compile (full_filename);
 
       if (scm_is_true (freshly_compiled))
-        return scm_load_compiled_with_vm (freshly_compiled);
+        return scm_load_compiled_with_vm (freshly_compiled, SCM_BOOL_T);
       else
         return scm_primitive_load (full_filename);
     }
@@ -1095,14 +1120,14 @@ scm_init_eval_in_scheme (void)
   eval_scm = search_path (*scm_loc_load_path,
                           scm_from_locale_string ("ice-9/eval.scm"),
                           SCM_EOL, SCM_BOOL_F, &stat_source,
-                          SCM_BOOL_F, NULL, NULL);
+                          SCM_BOOL_F, NULL, NULL, NULL);
   eval_go = search_path (*scm_loc_load_compiled_path,
                          scm_from_locale_string ("ice-9/eval.go"),
                          SCM_EOL, SCM_BOOL_F, &stat_compiled,
-                         eval_scm, &stat_source, &found_stale_eval_go);
+                         eval_scm, &stat_source, &found_stale_eval_go, NULL);
   
   if (scm_is_true (eval_go))
-    scm_load_compiled_with_vm (eval_go);
+    scm_load_compiled_with_vm (eval_go, SCM_BOOL_T);
   else
     /* If we have no eval.go, we shouldn't load any compiled code at all
        because we can't guarantee that tail calls will work.  */
diff --git a/libguile/vm.c b/libguile/vm.c
index 33f12b4..b560ee5 100644
--- a/libguile/vm.c
+++ b/libguile/vm.c
@@ -1500,10 +1500,31 @@ SCM_DEFINE (scm_call_with_stack_overflow_handler,
  * Initialize
  */
 
-SCM
-scm_load_compiled_with_vm (SCM file)
+static SCM
+load_compiled_with_vm_catch_handler (void *data, SCM tag, SCM throw_args)
+{
+  return SCM_BOOL_F;
+}
+
+static SCM
+do_try_scm_call_0 (void *data)
 {
-  return scm_call_0 (scm_load_thunk_from_file (file));
+  SCM thunk = SCM_PACK_POINTER (data);
+  return scm_call_0 (thunk);
+}
+
+SCM
+scm_load_compiled_with_vm (SCM file, SCM exception_on_not_found_p)
+{
+  SCM thunk = scm_load_thunk_from_file (file);
+  if (exception_on_not_found_p == SCM_BOOL_F)
+    return scm_c_catch (SCM_BOOL_T,
+                        do_try_scm_call_0,
+                        SCM_UNPACK_POINTER (thunk),
+                        load_compiled_with_vm_catch_handler,
+                        SCM_UNPACK_POINTER (thunk),
+                        NULL, NULL);
+  return scm_call_0 (thunk);
 }
 
   
diff --git a/libguile/vm.h b/libguile/vm.h
index 2ca4f2a..d76ac31 100644
--- a/libguile/vm.h
+++ b/libguile/vm.h
@@ -93,7 +93,7 @@ struct scm_vm_cont {
 #define SCM_VM_CONT_PARTIAL_P(CONT) (SCM_VM_CONT_DATA (CONT)->flags & SCM_F_VM_CONT_PARTIAL)
 #define SCM_VM_CONT_REWINDABLE_P(CONT) (SCM_VM_CONT_DATA (CONT)->flags & SCM_F_VM_CONT_REWINDABLE)
 
-SCM_API SCM scm_load_compiled_with_vm (SCM file);
+SCM_API SCM scm_load_compiled_with_vm (SCM file, SCM exception_on_not_found_p);
 
 SCM_INTERNAL SCM scm_i_call_with_current_continuation (SCM proc);
 SCM_INTERNAL SCM scm_i_capture_current_stack (void);
-- 
2.6.3


[-- Attachment #3: Type: text/plain, Size: 154 bytes --]


-- 
Jan Nieuwenhuizen <janneke@gnu.org> | GNU LilyPond http://lilypond.org
Freelance IT http://JoyofSource.com | Avatar®  http://AvatarAcademy.nl  

Re: [PATCH v3] Skip invalid compiled file found, continue searching path.

[Jan Nieuwenhuizen]

[-- Attachment #1: Type: text/plain, Size: 1234 bytes --]

Jan Nieuwenhuizen writes:

That was a copy of the old one again, here is the updated v3.

Jan


Only after rebuilding world, I found this morning that the previous
patch did not exactly work as intended.  Find version 3 of this patch
attached.

Instead of having scm_call_0 possibly throw and catching that in
vm.c:(scm_load_compiled_with_vm), now the FALSE_ON_ERROR flag is
propagated down to loader.c: (scm_load_thunk_from_file,
scm_load_thunk_from_memory, map_file_contents).

To test this, do something like

    mkdir -p ~/broken/srfi
    cp /gnu/store/b44k6wx7nxn0kh2kchlazgas2ybc576v-guile-2.0.11/lib/guile/2.0/ccache/srfi/srfi-98.go ~/broken/srfi
    touch ~/broken/srfi/srfi-111.go
    export GUILE_DEBUG_GO=1
    touch ~/broken/srfi/*.go

    meta/guile --no-auto-compile -s <(echo '
    (set! %load-compiled-path
    (cons (string-append (getenv "HOME") "/broken") %load-compiled-path))
    (use-modules (srfi srfi-98))
    (display (get-environment-variable "GUILE_DEBUG_GO"))
    (newline)
    ')

    meta/guile --no-auto-compile -s <(echo '
    (set! %load-compiled-path
    (cons (string-append (getenv "HOME") "/broken") %load-compiled-path))
    (use-modules (srfi srfi-111))
    (display box)
    (newline)
    ')


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-Skip-invalid-compiled-file-found-continue-searching-.patch --]
[-- Type: text/x-diff, Size: 13930 bytes --]

From f356b4dc8feef19fcab66967cd99575e6e96a4d5 Mon Sep 17 00:00:00 2001
From: Jan Nieuwenhuizen <janneke@gnu.org>
Date: Fri, 11 Mar 2016 14:58:09 +0100
Subject: [PATCH] Skip invalid compiled file found, continue searching path.

* libguile/vm.c (scm_load_compiled_with_vm): Add argument
  FALSE_ON_ERROR.
* libguile/loader.c (load_thunk_from_memory, scm_load_thunk_from_file,
  map_file_contents): Idem.
* libguile/load.c (search_path): Take optional output argument
  PATH_REMAINING.
  (scm_primitive_load_path): Use it.  Take optional argument
  LOAD_COMPILED_PATH.  Skip any invalid compiled file found and
  continue searching scm_loc_load_compiled_path.
---
 libguile/load.c   | 72 ++++++++++++++++++++++++++++++++++++++++++-------------
 libguile/loader.c | 42 ++++++++++++++++++++------------
 libguile/loader.h |  2 +-
 libguile/vm.c     |  8 ++++---
 libguile/vm.h     |  2 +-
 5 files changed, 90 insertions(+), 36 deletions(-)

diff --git a/libguile/load.c b/libguile/load.c
index d26f9fc..4c3ae86 100644
--- a/libguile/load.c
+++ b/libguile/load.c
@@ -582,12 +582,15 @@ compiled_is_fresh (SCM full_filename, SCM compiled_filename,
    file name that we find in the path.  Otherwise only return a file if
    it is newer than SOURCE_STAT_BUF, otherwise issuing a warning if we
    see a stale file earlier in the path, setting *FOUND_STALE_FILE to 1.
-  */
+
+   If PATH_REMAINING is not NULL, it is set to the tail of PATH that was
+   not skipped.  */
 static SCM
 search_path (SCM path, SCM filename, SCM extensions, SCM require_exts,
              struct stat *stat_buf,
              SCM source_file_name, struct stat *source_stat_buf,
-             int *found_stale_file)
+             int *found_stale_file,
+             SCM *path_remaining)
 {
   struct stringbuf buf;
   char *filename_chars;
@@ -724,6 +727,8 @@ search_path (SCM path, SCM filename, SCM extensions, SCM require_exts,
 
  end:
   scm_dynwind_end ();
+  if (path_remaining)
+    *path_remaining = path;
   return result;
 }
 
@@ -781,7 +786,7 @@ SCM_DEFINE (scm_search_path, "search-path", 2, 0, 1,
     require_exts = SCM_BOOL_F;
 
   return search_path (path, filename, extensions, require_exts, &stat_buf,
-                      SCM_BOOL_F, NULL, NULL);
+                      SCM_BOOL_F, NULL, NULL, NULL);
 }
 #undef FUNC_NAME
 
@@ -806,7 +811,7 @@ SCM_DEFINE (scm_sys_search_load_path, "%search-load-path", 1, 0, 0,
   SCM_VALIDATE_STRING (1, filename);
 
   return search_path (*scm_loc_load_path, filename, *scm_loc_load_extensions,
-                      SCM_BOOL_F, &stat_buf, SCM_BOOL_F, NULL, NULL);
+                      SCM_BOOL_F, &stat_buf, SCM_BOOL_F, NULL, NULL, NULL);
 }
 #undef FUNC_NAME
 
@@ -969,14 +974,19 @@ SCM_DEFINE (scm_primitive_load_path, "primitive-load-path", 0, 0, 1,
             "depending on the optional second argument,\n"
             "@var{exception_on_not_found}.  If it is @code{#f}, @code{#f}\n"
             "will be returned.  If it is a procedure, it will be called\n"
-            "with no arguments.  Otherwise an error is signalled.")
+            "with no arguments.  Otherwise an error is signalled."
+            "If the optional third argument,\n"
+            "@var{load_compiled_path} is given, use it to search for compiled files\n"
+            "instead of @var{*scm_loc_load_compiled_path}.")
 #define FUNC_NAME s_scm_primitive_load_path
 {
   SCM filename, exception_on_not_found;
   SCM full_filename, compiled_filename;
   SCM hook = *scm_loc_load_hook;
   struct stat stat_source, stat_compiled;
+  SCM load_compiled_path;
   int found_stale_compiled_file = 0;
+  SCM load_compiled_path_remaining = SCM_EOL;
 
   if (scm_is_true (hook) && scm_is_false (scm_procedure_p (hook)))
     SCM_MISC_ERROR ("value of %load-hook is neither a procedure nor #f",
@@ -988,21 +998,27 @@ SCM_DEFINE (scm_primitive_load_path, "primitive-load-path", 0, 0, 1,
 	 single argument (the file name).  */
       filename = args;
       exception_on_not_found = SCM_UNDEFINED;
+      load_compiled_path = *scm_loc_load_compiled_path;
     }
   else
     {
-      /* Starting from 1.9, this function takes 1 required and 1 optional
-	 argument.  */
+      /* Starting from 1.9, this function takes 1 required and 1
+	 optional argument.
+
+         Starting from 2.1.2, this function takes 1 required and 2
+	 optional arguments.  */
       long len;
 
       SCM_VALIDATE_LIST_COPYLEN (SCM_ARG1, args, len);
-      if (len < 1 || len > 2)
+      if (len < 1 || len > 3)
 	scm_error_num_args_subr (FUNC_NAME);
 
       filename = SCM_CAR (args);
       SCM_VALIDATE_STRING (SCM_ARG1, filename);
 
       exception_on_not_found = len > 1 ? SCM_CADR (args) : SCM_UNDEFINED;
+      load_compiled_path = len < 3 ? *scm_loc_load_compiled_path
+        : SCM_CADDR (args);
     }
 
   if (SCM_UNBNDP (exception_on_not_found))
@@ -1010,13 +1026,13 @@ SCM_DEFINE (scm_primitive_load_path, "primitive-load-path", 0, 0, 1,
 
   full_filename = search_path (*scm_loc_load_path, filename,
                                *scm_loc_load_extensions, SCM_BOOL_F,
-                               &stat_source, SCM_BOOL_F, NULL, NULL);
+                               &stat_source, SCM_BOOL_F, NULL, NULL, NULL);
 
   compiled_filename =
-    search_path (*scm_loc_load_compiled_path, filename,
+    search_path (load_compiled_path, filename,
                  *scm_loc_load_compiled_extensions, SCM_BOOL_T,
                  &stat_compiled, full_filename, &stat_source,
-                 &found_stale_compiled_file);
+                 &found_stale_compiled_file, &load_compiled_path_remaining);
 
   if (scm_is_false (compiled_filename)
       && scm_is_true (full_filename)
@@ -1066,13 +1082,37 @@ SCM_DEFINE (scm_primitive_load_path, "primitive-load-path", 0, 0, 1,
                        ? full_filename : compiled_filename));
 
   if (scm_is_true (compiled_filename))
-    return scm_load_compiled_with_vm (compiled_filename);
+    {
+      SCM module = scm_load_compiled_with_vm (compiled_filename, SCM_BOOL_T);
+      if (module == SCM_BOOL_F && getenv ("GUILE_DEBUG_GO"))
+        {
+          scm_puts_unlocked (";;; note: found broken .go ",
+                             scm_current_warning_port ());
+          scm_display (compiled_filename, scm_current_warning_port ());
+          scm_newline (scm_current_warning_port ());
+        }
+      if (module != SCM_BOOL_F || scm_is_false (exception_on_not_found))
+        return module;
+      if (scm_is_pair (load_compiled_path_remaining))
+        load_compiled_path_remaining = SCM_CDR (load_compiled_path_remaining);
+      if (module == SCM_BOOL_F && getenv ("GUILE_DEBUG_GO"))
+        {
+          scm_puts_unlocked (";;; skipping, continue on path ",
+                             scm_current_warning_port ());
+          scm_display (load_compiled_path_remaining, scm_current_warning_port ());
+          scm_newline (scm_current_warning_port ());
+        }
+      return scm_primitive_load_path (scm_list_3
+                                      (filename,
+                                       exception_on_not_found,
+                                       load_compiled_path_remaining));
+    }
   else
     {
       SCM freshly_compiled = scm_try_auto_compile (full_filename);
 
       if (scm_is_true (freshly_compiled))
-        return scm_load_compiled_with_vm (freshly_compiled);
+        return scm_load_compiled_with_vm (freshly_compiled, SCM_BOOL_F);
       else
         return scm_primitive_load (full_filename);
     }
@@ -1095,14 +1135,14 @@ scm_init_eval_in_scheme (void)
   eval_scm = search_path (*scm_loc_load_path,
                           scm_from_locale_string ("ice-9/eval.scm"),
                           SCM_EOL, SCM_BOOL_F, &stat_source,
-                          SCM_BOOL_F, NULL, NULL);
+                          SCM_BOOL_F, NULL, NULL, NULL);
   eval_go = search_path (*scm_loc_load_compiled_path,
                          scm_from_locale_string ("ice-9/eval.go"),
                          SCM_EOL, SCM_BOOL_F, &stat_compiled,
-                         eval_scm, &stat_source, &found_stale_eval_go);
+                         eval_scm, &stat_source, &found_stale_eval_go, NULL);
   
   if (scm_is_true (eval_go))
-    scm_load_compiled_with_vm (eval_go);
+    scm_load_compiled_with_vm (eval_go, SCM_BOOL_F);
   else
     /* If we have no eval.go, we shouldn't load any compiled code at all
        because we can't guarantee that tail calls will work.  */
diff --git a/libguile/loader.c b/libguile/loader.c
index 97effb3..bf72805 100644
--- a/libguile/loader.c
+++ b/libguile/loader.c
@@ -340,9 +340,12 @@ process_dynamic_segment (char *base, Elf_Phdr *dyn_phdr,
 }
 
 #define ABORT(msg) do { err_msg = msg; goto cleanup; } while (0)
+#define NULL_ELSE_SCM_SYSERROR(skip_p) {if (skip_p) {errno = 0; return NULL;} SCM_SYSERROR;}
+#define SCM_BOOL_F_ELSE_SCM_SYSERROR(skip_p) {if (skip_p) {errno = 0; return SCM_BOOL_F;} SCM_SYSERROR;}
 
 static SCM
-load_thunk_from_memory (char *data, size_t len, int is_read_only)
+load_thunk_from_memory (char *data, size_t len, int is_read_only,
+                        int false_on_error)
 #define FUNC_NAME "load-thunk-from-memory"
 {
   Elf_Ehdr *header;
@@ -456,10 +459,10 @@ load_thunk_from_memory (char *data, size_t len, int is_read_only)
 
  cleanup:
   {
-    if (errno)
-      SCM_SYSERROR;
-    scm_misc_error (FUNC_NAME, err_msg ? err_msg : "error loading ELF file",
-                    SCM_EOL);
+    if (!errno)
+      scm_misc_error (FUNC_NAME, err_msg ? err_msg : "error loading ELF file",
+                      SCM_EOL);
+    SCM_BOOL_F_ELSE_SCM_SYSERROR (false_on_error);
   }
 }
 #undef FUNC_NAME
@@ -467,7 +470,7 @@ load_thunk_from_memory (char *data, size_t len, int is_read_only)
 #define SCM_PAGE_SIZE 4096
 
 static char*
-map_file_contents (int fd, size_t len, int *is_read_only)
+map_file_contents (int fd, size_t len, int *is_read_only, int false_on_error)
 #define FUNC_NAME "load-thunk-from-file"
 {
   char *data;
@@ -475,7 +478,7 @@ map_file_contents (int fd, size_t len, int *is_read_only)
 #ifdef HAVE_SYS_MMAN_H
   data = mmap (NULL, len, PROT_READ, MAP_PRIVATE, fd, 0);
   if (data == MAP_FAILED)
-    SCM_SYSERROR;
+    NULL_ELSE_SCM_SYSERROR (false_on_error);
   *is_read_only = 1;
 #else
   if (lseek (fd, 0, SEEK_START) < 0)
@@ -483,7 +486,7 @@ map_file_contents (int fd, size_t len, int *is_read_only)
       int errno_save = errno;
       (void) close (fd);
       errno = errno_save;
-      SCM_SYSERROR;
+      NULL_ELSE_SCM_SYSERROR (false_on_error);
     }
 
   /* Given that we are using the read fallback, optimistically assume
@@ -527,8 +530,8 @@ map_file_contents (int fd, size_t len, int *is_read_only)
 }
 #undef FUNC_NAME
 
-SCM_DEFINE (scm_load_thunk_from_file, "load-thunk-from-file", 1, 0, 0,
-	    (SCM filename),
+SCM_DEFINE (scm_load_thunk_from_file, "load-thunk-from-file", 1, 1, 0,
+	    (SCM filename, SCM false_on_error),
 	    "")
 #define FUNC_NAME s_scm_load_thunk_from_file
 {
@@ -539,20 +542,29 @@ SCM_DEFINE (scm_load_thunk_from_file, "load-thunk-from-file", 1, 0, 0,
 
   SCM_VALIDATE_STRING (1, filename);
 
+  if (SCM_UNBNDP (false_on_error))
+    false_on_error = SCM_BOOL_F;
+
   c_filename = scm_to_locale_string (filename);
   fd = open (c_filename, O_RDONLY | O_BINARY | O_CLOEXEC);
   free (c_filename);
-  if (fd < 0) SCM_SYSERROR;
+  if (fd < 0)
+    SCM_BOOL_F_ELSE_SCM_SYSERROR (scm_is_true (false_on_error));
 
   end = lseek (fd, 0, SEEK_END);
   if (end < 0)
-    SCM_SYSERROR;
+    SCM_BOOL_F_ELSE_SCM_SYSERROR (scm_is_true (false_on_error));
 
-  data = map_file_contents (fd, end, &is_read_only);
+  data = map_file_contents (fd, end, &is_read_only,
+                            scm_is_true (false_on_error));
 
   (void) close (fd);
 
-  return load_thunk_from_memory (data, end, is_read_only);
+  if (data == NULL && scm_is_true (false_on_error))
+    return SCM_BOOL_F;
+
+  return load_thunk_from_memory (data, end, is_read_only,
+                                 scm_is_true (false_on_error));
 }
 #undef FUNC_NAME
 
@@ -574,7 +586,7 @@ SCM_DEFINE (scm_load_thunk_from_memory, "load-thunk-from-memory", 1, 0, 0,
 
   data = copy_and_align_elf_data (data, len);
 
-  return load_thunk_from_memory (data, len, 0);
+  return load_thunk_from_memory (data, len, 0, 0);
 }
 #undef FUNC_NAME
 
diff --git a/libguile/loader.h b/libguile/loader.h
index 5c719cb..e332abc 100644
--- a/libguile/loader.h
+++ b/libguile/loader.h
@@ -21,7 +21,7 @@
 
 #include <libguile.h>
 
-SCM_API SCM scm_load_thunk_from_file (SCM filename);
+SCM_API SCM scm_load_thunk_from_file (SCM filename, SCM exception_on_not_found_p);
 SCM_API SCM scm_load_thunk_from_memory (SCM bv);
 
 SCM_INTERNAL const scm_t_uint8 *
diff --git a/libguile/vm.c b/libguile/vm.c
index 33f12b4..d22990d 100644
--- a/libguile/vm.c
+++ b/libguile/vm.c
@@ -1501,12 +1501,14 @@ SCM_DEFINE (scm_call_with_stack_overflow_handler,
  */
 
 SCM
-scm_load_compiled_with_vm (SCM file)
+scm_load_compiled_with_vm (SCM file, SCM false_on_error)
 {
-  return scm_call_0 (scm_load_thunk_from_file (file));
+  SCM thunk = scm_load_thunk_from_file (file, false_on_error);
+  if (scm_is_false (thunk) && scm_is_true (false_on_error))
+    return SCM_BOOL_F;
+  return scm_call_0 (thunk);
 }
 
-  
 void
 scm_init_vm_builtin_properties (void)
 {
diff --git a/libguile/vm.h b/libguile/vm.h
index 2ca4f2a..037b1cb 100644
--- a/libguile/vm.h
+++ b/libguile/vm.h
@@ -93,7 +93,7 @@ struct scm_vm_cont {
 #define SCM_VM_CONT_PARTIAL_P(CONT) (SCM_VM_CONT_DATA (CONT)->flags & SCM_F_VM_CONT_PARTIAL)
 #define SCM_VM_CONT_REWINDABLE_P(CONT) (SCM_VM_CONT_DATA (CONT)->flags & SCM_F_VM_CONT_REWINDABLE)
 
-SCM_API SCM scm_load_compiled_with_vm (SCM file);
+SCM_API SCM scm_load_compiled_with_vm (SCM file, SCM false_on_error);
 
 SCM_INTERNAL SCM scm_i_call_with_current_continuation (SCM proc);
 SCM_INTERNAL SCM scm_i_capture_current_stack (void);
-- 
2.6.3


[-- Attachment #3: Type: text/plain, Size: 154 bytes --]


-- 
Jan Nieuwenhuizen <janneke@gnu.org> | GNU LilyPond http://lilypond.org
Freelance IT http://JoyofSource.com | Avatar®  http://AvatarAcademy.nl  

Re: [PATCH v4] Skip invalid compiled file found, continue searching path.

[Jan Nieuwenhuizen]

[-- Attachment #1: Type: text/plain, Size: 328 bytes --]

Jan Nieuwenhuizen writes:

The previous version v3 works for skipping modules after guile
has booted, this v4 also handles skipping any wrong `eval.go'.

I have included some error feedback when you set

    GUILE_DEBUG_GO=1

mainly to inspect and review the functionality, I suggest to get all
that out again.

Greetings,
Jan


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-Skip-invalid-compiled-file-found-continue-searching-.patch --]
[-- Type: text/x-diff, Size: 15760 bytes --]

From 515d23b52baacc62bebdf5986292303596674d16 Mon Sep 17 00:00:00 2001
From: Jan Nieuwenhuizen <janneke@gnu.org>
Date: Fri, 11 Mar 2016 14:58:09 +0100
Subject: [PATCH] Skip invalid compiled file found, continue searching path.

* libguile/vm.c (scm_load_compiled_with_vm): Add argument
  FALSE_ON_ERROR.
* libguile/loader.c (load_thunk_from_memory, scm_load_thunk_from_file,
  map_file_contents): Idem.
* libguile/load.c (search_path): Take optional output argument
  PATH_REMAINING.
  (scm_primitive_load_path): Use it.  Take optional argument
  LOAD_COMPILED_PATH.  Skip any invalid compiled file found and
  continue searching scm_loc_load_compiled_path.
  (internal_scm_init_eval_in_scheme): New function.  Implementation
  of scm_init_eval_in_scheme, taking path parameter to implement
  skipping any invalid eval.go's in path.
  (scm_init_eval_in_scheme): Use it.
---
 libguile/load.c   | 107 ++++++++++++++++++++++++++++++++++++++++++++----------
 libguile/loader.c |  42 +++++++++++++--------
 libguile/loader.h |   2 +-
 libguile/vm.c     |   8 ++--
 libguile/vm.h     |   2 +-
 5 files changed, 122 insertions(+), 39 deletions(-)

diff --git a/libguile/load.c b/libguile/load.c
index d26f9fc..b9db988 100644
--- a/libguile/load.c
+++ b/libguile/load.c
@@ -582,12 +582,15 @@ compiled_is_fresh (SCM full_filename, SCM compiled_filename,
    file name that we find in the path.  Otherwise only return a file if
    it is newer than SOURCE_STAT_BUF, otherwise issuing a warning if we
    see a stale file earlier in the path, setting *FOUND_STALE_FILE to 1.
-  */
+
+   If PATH_REMAINING is not NULL, it is set to the tail of PATH that was
+   not skipped.  */
 static SCM
 search_path (SCM path, SCM filename, SCM extensions, SCM require_exts,
              struct stat *stat_buf,
              SCM source_file_name, struct stat *source_stat_buf,
-             int *found_stale_file)
+             int *found_stale_file,
+             SCM *path_remaining)
 {
   struct stringbuf buf;
   char *filename_chars;
@@ -724,6 +727,8 @@ search_path (SCM path, SCM filename, SCM extensions, SCM require_exts,
 
  end:
   scm_dynwind_end ();
+  if (path_remaining)
+    *path_remaining = path;
   return result;
 }
 
@@ -781,7 +786,7 @@ SCM_DEFINE (scm_search_path, "search-path", 2, 0, 1,
     require_exts = SCM_BOOL_F;
 
   return search_path (path, filename, extensions, require_exts, &stat_buf,
-                      SCM_BOOL_F, NULL, NULL);
+                      SCM_BOOL_F, NULL, NULL, NULL);
 }
 #undef FUNC_NAME
 
@@ -806,7 +811,7 @@ SCM_DEFINE (scm_sys_search_load_path, "%search-load-path", 1, 0, 0,
   SCM_VALIDATE_STRING (1, filename);
 
   return search_path (*scm_loc_load_path, filename, *scm_loc_load_extensions,
-                      SCM_BOOL_F, &stat_buf, SCM_BOOL_F, NULL, NULL);
+                      SCM_BOOL_F, &stat_buf, SCM_BOOL_F, NULL, NULL, NULL);
 }
 #undef FUNC_NAME
 
@@ -969,14 +974,19 @@ SCM_DEFINE (scm_primitive_load_path, "primitive-load-path", 0, 0, 1,
             "depending on the optional second argument,\n"
             "@var{exception_on_not_found}.  If it is @code{#f}, @code{#f}\n"
             "will be returned.  If it is a procedure, it will be called\n"
-            "with no arguments.  Otherwise an error is signalled.")
+            "with no arguments.  Otherwise an error is signalled."
+            "If the optional third argument,\n"
+            "@var{load_compiled_path} is given, use it to search for compiled files\n"
+            "instead of @var{*scm_loc_load_compiled_path}.")
 #define FUNC_NAME s_scm_primitive_load_path
 {
   SCM filename, exception_on_not_found;
   SCM full_filename, compiled_filename;
   SCM hook = *scm_loc_load_hook;
   struct stat stat_source, stat_compiled;
+  SCM load_compiled_path;
   int found_stale_compiled_file = 0;
+  SCM load_compiled_path_remaining = SCM_EOL;
 
   if (scm_is_true (hook) && scm_is_false (scm_procedure_p (hook)))
     SCM_MISC_ERROR ("value of %load-hook is neither a procedure nor #f",
@@ -988,21 +998,27 @@ SCM_DEFINE (scm_primitive_load_path, "primitive-load-path", 0, 0, 1,
 	 single argument (the file name).  */
       filename = args;
       exception_on_not_found = SCM_UNDEFINED;
+      load_compiled_path = *scm_loc_load_compiled_path;
     }
   else
     {
-      /* Starting from 1.9, this function takes 1 required and 1 optional
-	 argument.  */
+      /* Starting from 1.9, this function takes 1 required and 1
+	 optional argument.
+
+         Starting from 2.1.2, this function takes 1 required and 2
+	 optional arguments.  */
       long len;
 
       SCM_VALIDATE_LIST_COPYLEN (SCM_ARG1, args, len);
-      if (len < 1 || len > 2)
+      if (len < 1 || len > 3)
 	scm_error_num_args_subr (FUNC_NAME);
 
       filename = SCM_CAR (args);
       SCM_VALIDATE_STRING (SCM_ARG1, filename);
 
       exception_on_not_found = len > 1 ? SCM_CADR (args) : SCM_UNDEFINED;
+      load_compiled_path = len < 3 ? *scm_loc_load_compiled_path
+        : SCM_CADDR (args);
     }
 
   if (SCM_UNBNDP (exception_on_not_found))
@@ -1010,13 +1026,13 @@ SCM_DEFINE (scm_primitive_load_path, "primitive-load-path", 0, 0, 1,
 
   full_filename = search_path (*scm_loc_load_path, filename,
                                *scm_loc_load_extensions, SCM_BOOL_F,
-                               &stat_source, SCM_BOOL_F, NULL, NULL);
+                               &stat_source, SCM_BOOL_F, NULL, NULL, NULL);
 
   compiled_filename =
-    search_path (*scm_loc_load_compiled_path, filename,
+    search_path (load_compiled_path, filename,
                  *scm_loc_load_compiled_extensions, SCM_BOOL_T,
                  &stat_compiled, full_filename, &stat_source,
-                 &found_stale_compiled_file);
+                 &found_stale_compiled_file, &load_compiled_path_remaining);
 
   if (scm_is_false (compiled_filename)
       && scm_is_true (full_filename)
@@ -1066,13 +1082,38 @@ SCM_DEFINE (scm_primitive_load_path, "primitive-load-path", 0, 0, 1,
                        ? full_filename : compiled_filename));
 
   if (scm_is_true (compiled_filename))
-    return scm_load_compiled_with_vm (compiled_filename);
+    {
+      SCM module = scm_load_compiled_with_vm (compiled_filename, SCM_BOOL_T);
+      if (scm_is_false (module) && getenv ("GUILE_DEBUG_GO"))
+        {
+          scm_puts_unlocked (";;; note: found broken .go ",
+                             scm_current_warning_port ());
+          scm_display (compiled_filename, scm_current_warning_port ());
+          scm_newline (scm_current_warning_port ());
+        }
+      if (!scm_is_false (module) || scm_is_false (exception_on_not_found))
+        return module;
+      if (scm_is_pair (load_compiled_path_remaining))
+        load_compiled_path_remaining = SCM_CDR (load_compiled_path_remaining);
+      if (scm_is_false (module) && getenv ("GUILE_DEBUG_GO"))
+        {
+          scm_puts_unlocked (";;; skipping, continue on path ",
+                             scm_current_warning_port ());
+          scm_display (load_compiled_path_remaining,
+                       scm_current_warning_port ());
+          scm_newline (scm_current_warning_port ());
+        }
+      return scm_primitive_load_path (scm_list_3
+                                      (filename,
+                                       exception_on_not_found,
+                                       load_compiled_path_remaining));
+    }
   else
     {
       SCM freshly_compiled = scm_try_auto_compile (full_filename);
 
       if (scm_is_true (freshly_compiled))
-        return scm_load_compiled_with_vm (freshly_compiled);
+        return scm_load_compiled_with_vm (freshly_compiled, SCM_BOOL_F);
       else
         return scm_primitive_load (full_filename);
     }
@@ -1085,30 +1126,58 @@ scm_c_primitive_load_path (const char *filename)
   return scm_primitive_load_path (scm_from_locale_string (filename));
 }
 
-void
-scm_init_eval_in_scheme (void)
+static void
+internal_scm_init_eval_in_scheme (SCM load_compiled_path)
 {
   SCM eval_scm, eval_go;
   struct stat stat_source, stat_compiled;
   int found_stale_eval_go = 0;
+  SCM load_compiled_path_remaining = SCM_EOL;
 
   eval_scm = search_path (*scm_loc_load_path,
                           scm_from_locale_string ("ice-9/eval.scm"),
                           SCM_EOL, SCM_BOOL_F, &stat_source,
-                          SCM_BOOL_F, NULL, NULL);
-  eval_go = search_path (*scm_loc_load_compiled_path,
+                          SCM_BOOL_F, NULL, NULL, NULL);
+  eval_go = search_path (load_compiled_path,
                          scm_from_locale_string ("ice-9/eval.go"),
                          SCM_EOL, SCM_BOOL_F, &stat_compiled,
-                         eval_scm, &stat_source, &found_stale_eval_go);
+                         eval_scm, &stat_source, &found_stale_eval_go,
+                         &load_compiled_path_remaining);
   
   if (scm_is_true (eval_go))
-    scm_load_compiled_with_vm (eval_go);
+    {
+      SCM module = scm_load_compiled_with_vm (eval_go, SCM_BOOL_T);
+      if (scm_is_pair (load_compiled_path_remaining))
+        load_compiled_path_remaining = SCM_CDR (load_compiled_path_remaining);
+      if (scm_is_false (module))
+        {
+          if (getenv ("GUILE_DEBUG_GO"))
+            {
+              scm_puts_unlocked (";;; note: found broken .go ",
+                                 scm_current_warning_port ());
+              scm_display (eval_go, scm_current_warning_port ());
+              scm_newline (scm_current_warning_port ());
+              scm_puts_unlocked (";;; skipping, continue on path ",
+                                 scm_current_warning_port ());
+              scm_display (load_compiled_path_remaining,
+                           scm_current_warning_port ());
+              scm_newline (scm_current_warning_port ());
+            }
+          internal_scm_init_eval_in_scheme (load_compiled_path_remaining);
+        }
+    }
   else
     /* If we have no eval.go, we shouldn't load any compiled code at all
        because we can't guarantee that tail calls will work.  */
     *scm_loc_load_compiled_path = SCM_EOL;
 }
 
+void
+scm_init_eval_in_scheme (void)
+{
+  internal_scm_init_eval_in_scheme (*scm_loc_load_compiled_path);
+}
+
 \f
 /* Information about the build environment.  */
 
diff --git a/libguile/loader.c b/libguile/loader.c
index 97effb3..bf72805 100644
--- a/libguile/loader.c
+++ b/libguile/loader.c
@@ -340,9 +340,12 @@ process_dynamic_segment (char *base, Elf_Phdr *dyn_phdr,
 }
 
 #define ABORT(msg) do { err_msg = msg; goto cleanup; } while (0)
+#define NULL_ELSE_SCM_SYSERROR(skip_p) {if (skip_p) {errno = 0; return NULL;} SCM_SYSERROR;}
+#define SCM_BOOL_F_ELSE_SCM_SYSERROR(skip_p) {if (skip_p) {errno = 0; return SCM_BOOL_F;} SCM_SYSERROR;}
 
 static SCM
-load_thunk_from_memory (char *data, size_t len, int is_read_only)
+load_thunk_from_memory (char *data, size_t len, int is_read_only,
+                        int false_on_error)
 #define FUNC_NAME "load-thunk-from-memory"
 {
   Elf_Ehdr *header;
@@ -456,10 +459,10 @@ load_thunk_from_memory (char *data, size_t len, int is_read_only)
 
  cleanup:
   {
-    if (errno)
-      SCM_SYSERROR;
-    scm_misc_error (FUNC_NAME, err_msg ? err_msg : "error loading ELF file",
-                    SCM_EOL);
+    if (!errno)
+      scm_misc_error (FUNC_NAME, err_msg ? err_msg : "error loading ELF file",
+                      SCM_EOL);
+    SCM_BOOL_F_ELSE_SCM_SYSERROR (false_on_error);
   }
 }
 #undef FUNC_NAME
@@ -467,7 +470,7 @@ load_thunk_from_memory (char *data, size_t len, int is_read_only)
 #define SCM_PAGE_SIZE 4096
 
 static char*
-map_file_contents (int fd, size_t len, int *is_read_only)
+map_file_contents (int fd, size_t len, int *is_read_only, int false_on_error)
 #define FUNC_NAME "load-thunk-from-file"
 {
   char *data;
@@ -475,7 +478,7 @@ map_file_contents (int fd, size_t len, int *is_read_only)
 #ifdef HAVE_SYS_MMAN_H
   data = mmap (NULL, len, PROT_READ, MAP_PRIVATE, fd, 0);
   if (data == MAP_FAILED)
-    SCM_SYSERROR;
+    NULL_ELSE_SCM_SYSERROR (false_on_error);
   *is_read_only = 1;
 #else
   if (lseek (fd, 0, SEEK_START) < 0)
@@ -483,7 +486,7 @@ map_file_contents (int fd, size_t len, int *is_read_only)
       int errno_save = errno;
       (void) close (fd);
       errno = errno_save;
-      SCM_SYSERROR;
+      NULL_ELSE_SCM_SYSERROR (false_on_error);
     }
 
   /* Given that we are using the read fallback, optimistically assume
@@ -527,8 +530,8 @@ map_file_contents (int fd, size_t len, int *is_read_only)
 }
 #undef FUNC_NAME
 
-SCM_DEFINE (scm_load_thunk_from_file, "load-thunk-from-file", 1, 0, 0,
-	    (SCM filename),
+SCM_DEFINE (scm_load_thunk_from_file, "load-thunk-from-file", 1, 1, 0,
+	    (SCM filename, SCM false_on_error),
 	    "")
 #define FUNC_NAME s_scm_load_thunk_from_file
 {
@@ -539,20 +542,29 @@ SCM_DEFINE (scm_load_thunk_from_file, "load-thunk-from-file", 1, 0, 0,
 
   SCM_VALIDATE_STRING (1, filename);
 
+  if (SCM_UNBNDP (false_on_error))
+    false_on_error = SCM_BOOL_F;
+
   c_filename = scm_to_locale_string (filename);
   fd = open (c_filename, O_RDONLY | O_BINARY | O_CLOEXEC);
   free (c_filename);
-  if (fd < 0) SCM_SYSERROR;
+  if (fd < 0)
+    SCM_BOOL_F_ELSE_SCM_SYSERROR (scm_is_true (false_on_error));
 
   end = lseek (fd, 0, SEEK_END);
   if (end < 0)
-    SCM_SYSERROR;
+    SCM_BOOL_F_ELSE_SCM_SYSERROR (scm_is_true (false_on_error));
 
-  data = map_file_contents (fd, end, &is_read_only);
+  data = map_file_contents (fd, end, &is_read_only,
+                            scm_is_true (false_on_error));
 
   (void) close (fd);
 
-  return load_thunk_from_memory (data, end, is_read_only);
+  if (data == NULL && scm_is_true (false_on_error))
+    return SCM_BOOL_F;
+
+  return load_thunk_from_memory (data, end, is_read_only,
+                                 scm_is_true (false_on_error));
 }
 #undef FUNC_NAME
 
@@ -574,7 +586,7 @@ SCM_DEFINE (scm_load_thunk_from_memory, "load-thunk-from-memory", 1, 0, 0,
 
   data = copy_and_align_elf_data (data, len);
 
-  return load_thunk_from_memory (data, len, 0);
+  return load_thunk_from_memory (data, len, 0, 0);
 }
 #undef FUNC_NAME
 
diff --git a/libguile/loader.h b/libguile/loader.h
index 5c719cb..e332abc 100644
--- a/libguile/loader.h
+++ b/libguile/loader.h
@@ -21,7 +21,7 @@
 
 #include <libguile.h>
 
-SCM_API SCM scm_load_thunk_from_file (SCM filename);
+SCM_API SCM scm_load_thunk_from_file (SCM filename, SCM exception_on_not_found_p);
 SCM_API SCM scm_load_thunk_from_memory (SCM bv);
 
 SCM_INTERNAL const scm_t_uint8 *
diff --git a/libguile/vm.c b/libguile/vm.c
index 33f12b4..d22990d 100644
--- a/libguile/vm.c
+++ b/libguile/vm.c
@@ -1501,12 +1501,14 @@ SCM_DEFINE (scm_call_with_stack_overflow_handler,
  */
 
 SCM
-scm_load_compiled_with_vm (SCM file)
+scm_load_compiled_with_vm (SCM file, SCM false_on_error)
 {
-  return scm_call_0 (scm_load_thunk_from_file (file));
+  SCM thunk = scm_load_thunk_from_file (file, false_on_error);
+  if (scm_is_false (thunk) && scm_is_true (false_on_error))
+    return SCM_BOOL_F;
+  return scm_call_0 (thunk);
 }
 
-  
 void
 scm_init_vm_builtin_properties (void)
 {
diff --git a/libguile/vm.h b/libguile/vm.h
index 2ca4f2a..037b1cb 100644
--- a/libguile/vm.h
+++ b/libguile/vm.h
@@ -93,7 +93,7 @@ struct scm_vm_cont {
 #define SCM_VM_CONT_PARTIAL_P(CONT) (SCM_VM_CONT_DATA (CONT)->flags & SCM_F_VM_CONT_PARTIAL)
 #define SCM_VM_CONT_REWINDABLE_P(CONT) (SCM_VM_CONT_DATA (CONT)->flags & SCM_F_VM_CONT_REWINDABLE)
 
-SCM_API SCM scm_load_compiled_with_vm (SCM file);
+SCM_API SCM scm_load_compiled_with_vm (SCM file, SCM false_on_error);
 
 SCM_INTERNAL SCM scm_i_call_with_current_continuation (SCM proc);
 SCM_INTERNAL SCM scm_i_capture_current_stack (void);
-- 
2.6.3


[-- Attachment #3: Type: text/plain, Size: 154 bytes --]


-- 
Jan Nieuwenhuizen <janneke@gnu.org> | GNU LilyPond http://lilypond.org
Freelance IT http://JoyofSource.com | Avatar®  http://AvatarAcademy.nl  

Re: [PATCH] Append effective version to GUILE_LOAD[_COMPILED]_PATH

[Mikael Djurfeldt]

[-- Attachment #1: Type: text/plain, Size: 3855 bytes --]

In python, the version number is higher up in the directory hierarchy,
which, hypothetically, allows newer versions to have "inventions" in the
more detailed directory structure:

/usr/lib/python2.6
/usr/lib/python2.7
etc

Just a thought.

On Fri, Mar 4, 2016 at 2:13 PM, Jan Nieuwenhuizen <janneke@gnu.org> wrote:

> Hi,
>
> I am running guile-2.0 and guile-2.2 alongside each other which is
> causing me some pain*).
>
> This is what bits of my GUILE_LOAD_COMPILED_PATH look like
>
>
> /gnu/store/7ml4psifv46pzxjxw56xfl7mwd47k277-profile/lib/guile/2.2/ccache
>     -->
> /gnu/store/7ml4psifv46pzxjxw56xfl7mwd47k277-profile/lib/guile/2.2/ccache/ice-9/and-let-star.go
>
>
> /gnu/store/7ml4psifv46pzxjxw56xfl7mwd47k277-profile/share/guile/site/2.2/
>     -->
> /gnu/store/7ml4psifv46pzxjxw56xfl7mwd47k277-profile/share/guile/site/2.2/os/process.go
>
> If `/<effective-version>' is always used as the suffix of each path
> element, and we/guix/packagers do not include that suffix in GUILE_*PATH
> elements, then Guile can append effective-prefix and different major
> Guile-versions can happily share the same GUILE_LOAD[_COMPILED]_PATH,
> e.g., having
>
>
> GUILE_LOAD_COMPILED_PATH=/gnu/store/7ml4psifv46pzxjxw56xfl7mwd47k277-profile/share/guile/site
>
> then guile-2.0 would get (os process) from
>
>     /gnu/store/7ml4psifv46pzxjxw56xfl7mwd47k277-profile/share/guile/site +
> /2.0
>     -->
> /gnu/store/7ml4psifv46pzxjxw56xfl7mwd47k277-profile/share/guile/site/2.0/os/process.go
>
> and guile-2.2 would read
>
>     /gnu/store/7ml4psifv46pzxjxw56xfl7mwd47k277-profile/share/guile/site +
> /2.2
>     -->
> /gnu/store/7ml4psifv46pzxjxw56xfl7mwd47k277-profile/share/guile/site/2.2/os/process.go
>
> What do you think?  No more pain!  Find patch attached.
>
> Greetings,
> Jan
>
>
> *) Some of my pain
>
> My Debian host system has guile-2.0, guix depends on guile-2.0, guix's
> LD script depends on guile-2.0, the guile bits of my project depend on
> guile-2.2.
>
> I have some scripts to make this situation almost bearable, but still I
> regularly
>
> cannot find a basic library
>
>     [1]13:53:25 janneke@janneke-ijzer:~
>     $ guile --no-auto-compile
>     GNU Guile 2.0.11
>     Copyright (C) 1995-2014 Free Software Foundation, Inc.
>
>     Guile comes with ABSOLUTELY NO WARRANTY; for details type `,show w'.
>     This program is free software, and you are welcome to redistribute it
>     under certain conditions; type `,show c' for details.
>
>     Enter `,help' for help.
>     scheme@(guile-user)> (use-modules (os proccess))
>     While compiling expression:
>     ERROR: no code for module (os proccess)
>
> or some guile script (guix) aborts
>
>     guix environment --ad-hoc ccache coreutils git guix emacs guile-next
> guile-next-lib
>     Throw without catch before boot:
>     Throw to key misc-error with args ("make_objcode_from_file" "bad
> header on object file: ~s"
> ("\x7fELF\x02\x01\x01\x00\x00\x00\x00\x00\x00\x00\x00") #f)Aborting.
>     Aborted
>
> or linking breaks
>
>     g++ -Wall -std=c++11  -g -I. -I out/alarm.project/c++ -I
> check/alarm.project/   out/alarm.project/c++/main.o
> out/alarm.project/c++/Alarm.o out/alarm.project/c++/AlarmSystem.o
> out/alarm.project/c++/pump.o out/alarm.project/c++/runtime.o
> -lboost_system -lboost_coroutine -lboost_thread -lboost_context -pthread -o
> out/alarm.project/c++/test
>     collect2: error: ld terminated with signal 6 [Afgebroken]
>     Throw without catch before boot:
>     Aborting.
>
> and then I juggle installed guile versions and/or manually modify
> GUILE_LOAD_COMPILED_PATH.
>
>
>
> --
> Jan Nieuwenhuizen <janneke@gnu.org> | GNU LilyPond http://lilypond.org
> Freelance IT http://JoyofSource.com | Avatar®  http://AvatarAcademy.nl
>
>

[-- Attachment #2: Type: text/html, Size: 4753 bytes --]

Re: [PATCH] Append effective version to GUILE_LOAD[_COMPILED]_PATH

[Jan Nieuwenhuizen]

Mikael Djurfeldt writes:

> In python, the version number is higher up in the directory hierarchy,
> which, hypothetically, allows newer versions to have "inventions" in
> the more detailed directory structure:
>
> /usr/lib/python2.6
> /usr/lib/python2.7
> etc
>
> Just a thought.

Python's case is a bit different

 * Python does not look at any environment variable to boot

in contrast, setting GUILE_LOAD_*PATH to the wrong place makes Guile
barf trying to load eval.* or boot-9.*.  Also, guix by default sets
GUILE_LOAD_PATH and GUILE_LOAD_COMPILED_PATH, thus "ensuring" that
different guile major versions will fail to boot.

Do we want Guile to look at environment variables to find its core
libraries?

 * Python has magic numbers and skip/recompiles if a .py[oc] does
   not match

This resembles what Ludovic suggested and what I have submitted a patch
for, to skip invalid .go files.

Greetings, Jan

--
Jan Nieuwenhuizen <janneke@gnu.org> | GNU LilyPond http://lilypond.org
Freelance IT http://JoyofSource.com | Avatar® http://AvatarAcademy.nl

Re: [PATCH] Append effective version to GUILE_LOAD[_COMPILED]_PATH

[Ludovic Courtès]

Jan Nieuwenhuizen <janneke@gnu.org> skribis:

> in contrast, setting GUILE_LOAD_*PATH to the wrong place makes Guile
> barf trying to load eval.* or boot-9.*. 

There’s also ‘GUILE_SYSTEM_COMPILED_PATH’ and ‘GUILE_SYSTEM_PATH’.

> Also, guix by default sets GUILE_LOAD_PATH and
> GUILE_LOAD_COMPILED_PATH, thus "ensuring" that different guile major
> versions will fail to boot.
>
> Do we want Guile to look at environment variables to find its core
> libraries?
>
>  * Python has magic numbers and skip/recompiles if a .py[oc] does
>    not match
>
> This resembles what Ludovic suggested and what I have submitted a patch
> for, to skip invalid .go files.

I think core libraries are not specifically the problem.  The problem is
that Guile can stumble upon incompatible .go files because
‘GUILE_LOAD_COMPILED_PATH’ is honored by both 2.0 and 2.2.

So I think the possible solutions are:

  1. Have 2.2 honor only GUILE_2_2_LOAD_COMPILED_PATH.  Obviously not
     realistic.

  2. Have Guile automatically append “/2.2” (or “/2.0”, etc.) to the
     GUILE_LOAD_COMPILED_PATH entries.  Would be nice, but too late.

  3. Change both 2.0 and 2.2 to skip invalid/incompatible .go files.
     Sounds like the only viable solution.

Thoughts?

Ludo’.