* bootstrapping SED-4.5, circular dependency? @ 2018-12-08 13:46 Jan Nieuwenhuizen 2018-12-08 18:59 ` Assaf Gordon 0 siblings, 1 reply; 6+ messages in thread From: Jan Nieuwenhuizen @ 2018-12-08 13:46 UTC (permalink / raw) To: sed-devel; +Cc: guix-devel Hi! Now that we have successfully bootstrapped GuixSD without using Gcc, Binutils, GNU C lib, I am working to remove other binary bootstrap seeds; such as SED and GZIP. I have bootstrapped GNU make and Bash and was looking into building GZIP, only to find out that it depends on a SED feature that our bootstrap-gash does not provide. So, I figured that SED should be built before GZIP, but I found that SED-4.5 depends on the same SED feature that GZIP needs...apparently a circular dependency? Here is what I get: --8<---------------cut here---------------start------------->8--- configure flags: ("CONFIG_SHELL=/gnu/store/md6pc5frcqnmndgwmnwk3xkmh4bx71jc-bash-mesboot-4.4.23/bin/bash" "SHELL=/gnu/store/md6pc5frcqnmndgwmnwk3xkmh4bx71jc-bash-mesboot-4.4.23/bin/bash" "--prefix=/gnu/store/hy9dq7jhmnjsgbaq9rkg59h9ssgqqzhw-sed-mesboot-4.5" "--enable-fast-install" "--build=x86_64-unknown-linux-gnu") Backtrace: 5 (apply-smob/1 #<catch-closure 9e5260>) In ice-9/boot-9.scm: 705:2 4 (call-with-prompt _ _ #<procedure default-prompt-handle?>) In ice-9/eval.scm: 619:8 3 (_ #(#(#<directory (guile-user) a670a0>))) In gash/commands/sed.scm: 132:30 2 (sed . _) In ice-9/boot-9.scm: 222:17 1 (map1 ("/^.*\\/\\([^/][^/]*\\)\\/*$/{\n\t s//\\1/\n?")) In unknown file: 0 (scm-error misc-error #f "~A" ("SED: command not supp?") ?) ERROR: In procedure scm-error: SED: command not supported: "/^.*\\/\\([^/][^/]*\\)\\/*$/{\n\t s//\\1/\n\t q\n\t }\n\t /^X\\/\\(\\/\\/\\)$/{\n\t s//\\1/\n\t q\n\t }\n\t /^X\\/\\(\\/\\).*/{\n\t s//\\1/\n\t q\n\t }\n\t s/.*/./; q" --8<---------------cut here---------------end--------------->8--- The SED throwing this error is the minimal SED from Gash, that I implemented in GNU Guile. Do you know whether bootstrapping of GZIP and/or SED is supported? Can you advise me on my next step (looking to bootstrap an earlier SED -- which version? or an earlier GZIP -- which version?) Thanks! janneke -- Jan Nieuwenhuizen <janneke@gnu.org> | GNU LilyPond http://lilypond.org Freelance IT http://JoyofSource.com | Avatar® http://AvatarAcademy.com ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: bootstrapping SED-4.5, circular dependency? 2018-12-08 13:46 bootstrapping SED-4.5, circular dependency? Jan Nieuwenhuizen @ 2018-12-08 18:59 ` Assaf Gordon 2018-12-08 23:26 ` Ludovic Courtès 2018-12-09 3:52 ` Jan Nieuwenhuizen 0 siblings, 2 replies; 6+ messages in thread From: Assaf Gordon @ 2018-12-08 18:59 UTC (permalink / raw) To: Jan Nieuwenhuizen, sed-devel; +Cc: guix-devel Hello, On 2018-12-08 6:46 a.m., Jan Nieuwenhuizen wrote: > Now that we have successfully bootstrapped GuixSD without using Gcc, > Binutils, GNU C lib, I am working to remove other binary bootstrap > seeds; such as SED and GZIP. > > I have bootstrapped GNU make and Bash and was looking into building > GZIP, only to find out that it depends on a SED feature that our > bootstrap-gash does not provide. > > So, I figured that SED should be built before GZIP, but I found that > SED-4.5 depends on the same SED feature that GZIP needs...apparently > a circular dependency? Technically speaking, these kind of dependencies (e.g. needing SED to build a package) arise from autotools, not directly from sed's build commands. > Here is what I get: [...] > ERROR: In procedure scm-error: > SED: command not supported: "/^.*\\/\\([^/][^/]*\\)\\/*$/{\n\t s//\\1/\n\t q\n\t }\n\t /^X\\/\\(\\/\\/\\)$/{\n\t s//\\1/\n\t q\n\t }\n\t /^X\\/\\(\\/\\).*/{\n\t s//\\1/\n\t q\n\t }\n\t s/.*/./; q" The failed sed program is (transcribed from the above error message) ==== /^.*\/\([^/][^/]*\)\/*$/{ s//\1/ q } /^X\/\(\/\/\)$/{ s//\1/ q } /^X\/\(\/\).*/{ s//\1/ q } s/.*/./; q ==== It is a very standard sed program, which should (and does) work on every existing SED implementation (posix-compliant of course). > The SED throwing this error is the minimal SED from Gash, that I > implemented in GNU Guile. I would recommend adjusting your sed implementation to support these commands, because you'll encounter them in many packages that use autotools. > Do you know whether bootstrapping of GZIP and/or SED is supported? It is supported in the sense that almost any old sed implementation can be used (e.g. you can build GNU sed if you only have BSD sed or AIX sed or busybox sed). > Can > you advise me on my next step (looking to bootstrap an earlier SED -- > which version? or an earlier GZIP -- which version?) Digging a bit further, I see this specific sed program originates from autoconf, from the M4 code used to implement basename and dirname, e.g. here: https://git.savannah.gnu.org/cgit/autoconf.git/tree/lib/m4sugar/m4sh.m4#n922 https://git.savannah.gnu.org/cgit/autoconf.git/tree/lib/m4sugar/m4sh.m4#n970 So it means every autotools package that uses these macros (and there are many) will cause the same issue. The best solution is likely to implement these sed commands in your sed implementation (if you need help in testing I'm happy to try, though I know very little about guile). Second best would likely be to investigate with autotools developers how to avoid this. Perhaps have a specific flag to autoconf to skip generation of these portability m4 macros, since you know guix's basename/dirname are good enough ? As a last resort, I can think of a really ugly hack: create a "fake sed" wrapper script in guile, that checks if the given SED script is one that you don't implement (e.g. the one above). If it is, since we know all it does is basename/dirname - just call a real dirname/basename instead of using sed's regexes. Otherwise, pass the parameters to the real guile-sed. Put this guile-sed-wrapper in $PATH, and it might just work... In a completely different approach, since you are trying to bootstrap: building OpenBSD's sed binary is very easy (needs just a C compiler and works fine on GNU/Linux) - perhaps building it would be faster as a first step to get a working sed, then use it to build other packages (I can help with that if you want, send me an email). regards, - assaf ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: bootstrapping SED-4.5, circular dependency? 2018-12-08 18:59 ` Assaf Gordon @ 2018-12-08 23:26 ` Ludovic Courtès 2018-12-09 3:59 ` Jan Nieuwenhuizen 2018-12-09 3:52 ` Jan Nieuwenhuizen 1 sibling, 1 reply; 6+ messages in thread From: Ludovic Courtès @ 2018-12-08 23:26 UTC (permalink / raw) To: Assaf Gordon; +Cc: guix-devel, sed-devel Hello, Assaf Gordon <assafgordon@gmail.com> skribis: > Digging a bit further, I see this specific sed program originates > from autoconf, from the M4 code used to implement basename and dirname, > e.g. here: > https://git.savannah.gnu.org/cgit/autoconf.git/tree/lib/m4sugar/m4sh.m4#n922 > https://git.savannah.gnu.org/cgit/autoconf.git/tree/lib/m4sugar/m4sh.m4#n970 IIUC, ‘configure’ scripts first try to determine whether they have a working ‘basename’ and ‘dirname’ with: --8<---------------cut here---------------start------------->8--- if (basename -- /) >/dev/null 2>&1 && test "X`basename -- / 2>&1`" = "X/"; then as_basename=basename else as_basename=false fi if (as_dir=`dirname -- /` && test "X$as_dir" = X/) >/dev/null 2>&1; then as_dirname=dirname else as_dirname=false fi --8<---------------cut here---------------end--------------->8--- Jan, could it be that the ‘basename’ and ‘dirname’ implementations in Gash fail this test? It might be that sed is not needed if you get these right. Thanks, Ludo’. ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: bootstrapping SED-4.5, circular dependency? 2018-12-08 23:26 ` Ludovic Courtès @ 2018-12-09 3:59 ` Jan Nieuwenhuizen 2018-12-09 13:42 ` Ludovic Courtès 0 siblings, 1 reply; 6+ messages in thread From: Jan Nieuwenhuizen @ 2018-12-09 3:59 UTC (permalink / raw) To: Ludovic Courtès; +Cc: guix-devel, Assaf Gordon, sed-devel Ludovic Courtès writes: Hello, > Jan, could it be that the ‘basename’ and ‘dirname’ implementations in > Gash fail this test? It might be that sed is not needed if you get > these right. Ah, yes! I have added some test to Gash and this basename -- / failed, producing the empty string, where coreutils' basename returns `/'. Gash "simply" (there is som if'ing involved) calls Guile's basename, which disagrees here and returns "" for "/". I'm assuming that this is expected behaviour for Guile?... so I special-cased this for Gash' basename. Greetings, janneke -- Jan Nieuwenhuizen <janneke@gnu.org> | GNU LilyPond http://lilypond.org Freelance IT http://JoyofSource.com | Avatar® http://AvatarAcademy.com ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: bootstrapping SED-4.5, circular dependency? 2018-12-09 3:59 ` Jan Nieuwenhuizen @ 2018-12-09 13:42 ` Ludovic Courtès 0 siblings, 0 replies; 6+ messages in thread From: Ludovic Courtès @ 2018-12-09 13:42 UTC (permalink / raw) To: Jan Nieuwenhuizen; +Cc: guix-devel, Assaf Gordon, sed-devel [-- Attachment #1: Type: text/plain, Size: 1005 bytes --] Hi! Jan Nieuwenhuizen <janneke@gnu.org> skribis: > Ludovic Courtès writes: > > Hello, > >> Jan, could it be that the ‘basename’ and ‘dirname’ implementations in >> Gash fail this test? It might be that sed is not needed if you get >> these right. > > Ah, yes! I have added some test to Gash and this > > basename -- / > > failed, producing the empty string, where coreutils' basename returns > `/'. > > Gash "simply" (there is som if'ing involved) calls Guile's basename, > which disagrees here and returns "" for "/". > > I'm assuming that this is expected behaviour for Guile?... so I > special-cased this for Gash' basename. Looking at <http://pubs.opengroup.org/onlinepubs/9699919799/functions/basename.html>, I think it’s a bug of Guile’s ‘basename’ (which is not implemented in terms of libc’s ‘basename’.) The fix might be as simple as the patch below, WDYT? In the meantime Gash should work around the bug. Thanks, Ludo’. [-- Warning: decoded text below may be mangled, UTF-8 assumed --] [-- Attachment #2: Type: text/x-patch, Size: 811 bytes --] diff --git a/libguile/filesys.c b/libguile/filesys.c index e1aeeed1b..c1cd50e21 100644 --- a/libguile/filesys.c +++ b/libguile/filesys.c @@ -1602,11 +1602,20 @@ SCM_DEFINE (scm_basename, "basename", 1, 1, 0, c_filename = scm_to_utf8_string (filename); scm_dynwind_free (c_filename); + if (strcmp (c_filename, "/") == 0 + || strcmp (c_filename, "//") == 0) + /* As per + <http://pubs.opengroup.org/onlinepubs/9699919799/functions/basename.html>, + "/" and "//" are treated specially. */ + res = scm_from_utf8_string ("/"); + else + { c_last_component = last_component (c_filename); if (!c_last_component) res = filename; else res = scm_from_utf8_string (c_last_component); + } scm_dynwind_end (); if (!SCM_UNBNDP (suffix) && ^ permalink raw reply related [flat|nested] 6+ messages in thread
* Re: bootstrapping SED-4.5, circular dependency? 2018-12-08 18:59 ` Assaf Gordon 2018-12-08 23:26 ` Ludovic Courtès @ 2018-12-09 3:52 ` Jan Nieuwenhuizen 1 sibling, 0 replies; 6+ messages in thread From: Jan Nieuwenhuizen @ 2018-12-09 3:52 UTC (permalink / raw) To: Assaf Gordon; +Cc: guix-devel, sed-devel Assaf Gordon writes: Hello Assaf, > Technically speaking, these kind of dependencies (e.g. needing SED > to build a package) arise from autotools, not directly from sed's > build commands. Ah yes, course. That is a big difference. >> Here is what I get: > [...] >> ERROR: In procedure scm-error: >> SED: command not supported: "/^.*\\/\\([^/][^/]*\\)\\/*$/{\n\t s//\\1/\n\t q\n\t }\n\t /^X\\/\\(\\/\\/\\)$/{\n\t s//\\1/\n\t q\n\t }\n\t /^X\\/\\(\\/\\).*/{\n\t s//\\1/\n\t q\n\t }\n\t s/.*/./; q" > > The failed sed program is (transcribed from the above error message) > ==== > /^.*\/\([^/][^/]*\)\/*$/{ > s//\1/ > q > } > /^X\/\(\/\/\)$/{ > s//\1/ > q > } > /^X\/\(\/\).*/{ > s//\1/ > q > } > s/.*/./; q > ==== > > It is a very standard sed program, which should (and does) work on every > existing SED implementation (posix-compliant of course). Right, now I see. I really only catered for plain `-e s,,,' commands. > I would recommend adjusting your sed implementation to support > these commands, because you'll encounter them in many packages > that use autotools. That's sounds like the right thing to do, thanks! > Digging a bit further, I see this specific sed program originates > from autoconf, from the M4 code used to implement basename and dirname, > e.g. here: > https://git.savannah.gnu.org/cgit/autoconf.git/tree/lib/m4sugar/m4sh.m4#n922 > https://git.savannah.gnu.org/cgit/autoconf.git/tree/lib/m4sugar/m4sh.m4#n970 > > So it means every autotools package that uses these macros (and there > are many) will cause the same issue. > > The best solution is likely to implement these sed commands in your > sed implementation (if you need help in testing I'm happy to try, > though I know very little about guile). That is even more helpful! While I still think we should support the simple SED script above, there could be a problem with basename/dirname. Solving that might even sidestep this SED feature. Many options now! > As a last resort, I can think of a really ugly hack: > create a "fake sed" wrapper script in guile, > that checks if the given SED script is one that you don't implement > (e.g. the one above). If it is, since we know all it does is > basename/dirname - just call a real dirname/basename instead of > using sed's regexes. Otherwise, pass the parameters to the real > guile-sed. > Put this guile-sed-wrapper in $PATH, and it might just work... > > > In a completely different approach, since you are trying to bootstrap: > building OpenBSD's sed binary is very easy (needs just a C compiler > and works fine on GNU/Linux) - perhaps building it > would be faster as a first step to get a working sed, then use it > to build other packages (I can help with that if you want, send me an > email). Many thanks, also for these two last resort options! I'm sure we'll manage to bootstrap SED. I'll let you know how it goes. Greetings, janneke -- Jan Nieuwenhuizen <janneke@gnu.org> | GNU LilyPond http://lilypond.org Freelance IT http://JoyofSource.com | Avatar® http://AvatarAcademy.com ^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2018-12-09 13:42 UTC | newest] Thread overview: 6+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2018-12-08 13:46 bootstrapping SED-4.5, circular dependency? Jan Nieuwenhuizen 2018-12-08 18:59 ` Assaf Gordon 2018-12-08 23:26 ` Ludovic Courtès 2018-12-09 3:59 ` Jan Nieuwenhuizen 2018-12-09 13:42 ` Ludovic Courtès 2018-12-09 3:52 ` Jan Nieuwenhuizen
Code repositories for project(s) associated with this external index https://git.savannah.gnu.org/cgit/guix.git This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.