bug#20081: patch-source-shebangs crashes on broken symlink

bug#20081: patch-source-shebangs crashes on broken symlink

[Tomáš Čech]

[-- Attachment #1: Type: text/plain, Size: 2442 bytes --]

I'm trying to create package for taskwarrior.

Source tarball contain symlinks to nonexisting file `task':

$ tar tvf /gnu/store/*-task*.tar.gz | grep -E '/src/(tw|cal |calendar|task)'
lrwxr-xr-x ultrafredde/staff      0 2015-02-16 23:45 task-2.4.1/src/cal -> task
lrwxr-xr-x ultrafredde/staff      0 2015-02-16 23:45 task-2.4.1/src/calendar -> task
lrwxr-xr-x ultrafredde/staff      0 2015-02-16 23:45 task-2.4.1/src/tw -> task

When I run build, I got this backtrace:

phase `unpack' succeeded after 0 seconds
starting phase `patch-usr-bin-file'
phase `patch-usr-bin-file' succeeded after 0 seconds
starting phase `patch-source-shebangs'
Backtrace:
In ice-9/boot-9.scm:
  157: 15 [catch #t #<catch-closure f65740> ...]
In unknown file:
    ?: 14 [apply-smob/1 #<catch-closure f65740>]
In ice-9/boot-9.scm:
   63: 13 [call-with-prompt prompt0 ...]
In ice-9/eval.scm:
  432: 12 [eval # #]
In ice-9/boot-9.scm:
2401: 11 [save-module-excursion #<procedure f82880 at ice-9/boot-9.scm:4045:3 ()>]
4050: 10 [#<procedure f82880 at ice-9/boot-9.scm:4045:3 ()>]
1724: 9 [%start-stack load-stack #<procedure f959a0 at ice-9/boot-9.scm:4041:10 ()>]
1729: 8 [#<procedure f97c60 ()>]
In unknown file:
    ?: 7 [primitive-load "/gnu/store/ix1s7q448frw02wy9xvzhd66vh08lxcw-taskwarrior-2.4.1-guile-builder"]
In ice-9/eval.scm:
  387: 6 [eval # ()]
In srfi/srfi-1.scm:
  830: 5 [every1 #<procedure 12eb680 at /gnu/store/dyv4k9p9na96q4yzahdlvij3nadaz65h-module-import/guix/build/gnu-build-system.scm:507:9 (expr)> ...]
In /gnu/store/dyv4k9p9na96q4yzahdlvij3nadaz65h-module-import/guix/build/gnu-build-system.scm:
  511: 4 [#<procedure 12eb680 at /gnu/store/dyv4k9p9na96q4yzahdlvij3nadaz65h-module-import/guix/build/gnu-build-system.scm:507:9 (expr)> #]
  164: 3 [patch-source-shebangs # ...]
In unknown file:
    ?: 2 [remove #<procedure ee4d60 at ice-9/boot-9.scm:1460:6 (str)> #]
In ice-9/boot-9.scm:
1461: 1 [#<procedure ee4d60 at ice-9/boot-9.scm:1460:6 (str)> "./src/cal"]
  In unknown file:
    ?: 0 [stat "./src/cal" #<undefined>]

ERROR: In procedure stat:
ERROR: In procedure stat: No such file or directory: "./src/cal"
builder for `/gnu/store/vr408ijifflkqjk9lgpj3sv469fj2pik-taskwarrior-2.4.1.drv' failed with exit code 1
cannot build derivation `/gnu/store/367g51d6vh8v5m1q58hls6bn40ha1262-profile.drv': 1 dependencies couldn't be built
guix package: error: build failed: build of `/gnu/store/367g51d6vh8v5m1q58hls6bn40ha1262-profile.drv' failed

[-- Attachment #2: Type: application/pgp-signature, Size: 181 bytes --]

bug#20081: patch-source-shebangs crashes on broken symlink

[Andreas Enge]

On Wed, Mar 11, 2015 at 04:02:11PM +0100, Tomáš Čech wrote:
> I'm trying to create package for taskwarrior.
> Source tarball contain symlinks to nonexisting file `task':

I would argue that this is not a bug in guix, but in the tarball.
You can remove the link with an additional phase before 'configure, see, for
instance, the dvdisaster package in cdrom.scm.

Andreas

bug#20081: patch-source-shebangs crashes on broken symlink

[Tomáš Čech]

On Wed, Mar 11, 2015 at 06:32:30PM +0100, Andreas Enge wrote:
>On Wed, Mar 11, 2015 at 04:02:11PM +0100, Tomáš Čech wrote:
>> I'm trying to create package for taskwarrior.
>> Source tarball contain symlinks to nonexisting file `task':
>
>I would argue that this is not a bug in guix, but in the tarball.
>You can remove the link with an additional phase before 'configure, see, for
>instance, the dvdisaster package in cdrom.scm.

I agree with you that the fishy part is in tarball, but we could make
build more robust. Getting backtrace is not nice way to end a build.

And we alone are getting into this problem it is not usual to
preprocess source files before compilation.

I already worked around the bug to confirm I correctly identified the cause.

S_W

bug#20081: patch-source-shebangs crashes on broken symlink

[Mark H Weaver]

Andreas Enge <andreas@enge.fr> writes:

> On Wed, Mar 11, 2015 at 04:02:11PM +0100, Tomáš Čech wrote:
>> I'm trying to create package for taskwarrior.
>> Source tarball contain symlinks to nonexisting file `task':
>
> I would argue that this is not a bug in guix, but in the tarball.
> You can remove the link with an additional phase before 'configure, see, for
> instance, the dvdisaster package in cdrom.scm.

The phase should go after the 'unpack' phase rather than before
configure, but otherwise I agree with Andreas.

       Mark

bug#20081: patch-source-shebangs crashes on broken symlink

[Ludovic Courtès]

[-- Attachment #1: Type: text/plain, Size: 744 bytes --]

Tomáš Čech <sleep_walker@suse.cz> skribis:

> On Wed, Mar 11, 2015 at 06:32:30PM +0100, Andreas Enge wrote:
>>On Wed, Mar 11, 2015 at 04:02:11PM +0100, Tomáš Čech wrote:
>>> I'm trying to create package for taskwarrior.
>>> Source tarball contain symlinks to nonexisting file `task':
>>
>>I would argue that this is not a bug in guix, but in the tarball.
>>You can remove the link with an additional phase before 'configure, see, for
>>instance, the dvdisaster package in cdrom.scm.
>
> I agree with you that the fishy part is in tarball, but we could make
> build more robust. Getting backtrace is not nice way to end a build.

I agree.  I think we should patch ‘find-files’ in core-updates to not
follow symlinks:


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: Type: text/x-patch, Size: 428 bytes --]

diff --git a/guix/build/utils.scm b/guix/build/utils.scm
index a5a6167..9cbddcd 100644
--- a/guix/build/utils.scm
+++ b/guix/build/utils.scm
@@ -288,7 +288,8 @@ matches REGEXP."
                                     file (strerror errno))
                             result)
                           '()
-                          dir)
+                          dir
+                          lstat)
         string<?))
 
 \f

[-- Attachment #3: Type: text/plain, Size: 34 bytes --]


Thoughts?

Thanks,
Ludo’.

bug#20081: patch-source-shebangs crashes on broken symlink

[Tomáš Čech]

[-- Attachment #1: Type: text/plain, Size: 1648 bytes --]

On Thu, Mar 12, 2015 at 10:30:55AM +0100, Ludovic Courtès wrote:
>Tomáš Čech <sleep_walker@suse.cz> skribis:
>
>> On Wed, Mar 11, 2015 at 06:32:30PM +0100, Andreas Enge wrote:
>>>On Wed, Mar 11, 2015 at 04:02:11PM +0100, Tomáš Čech wrote:
>>>> I'm trying to create package for taskwarrior.
>>>> Source tarball contain symlinks to nonexisting file `task':
>>>
>>>I would argue that this is not a bug in guix, but in the tarball.
>>>You can remove the link with an additional phase before 'configure, see, for
>>>instance, the dvdisaster package in cdrom.scm.
>>
>> I agree with you that the fishy part is in tarball, but we could make
>> build more robust. Getting backtrace is not nice way to end a build.
>
>I agree.  I think we should patch ‘find-files’ in core-updates to not
>follow symlinks:
>

>^[[1;32mdiff --git a/guix/build/utils.scm b/guix/build/utils.scm^[[0;0m
>^[[0;0mindex a5a6167..9cbddcd 100644^[[0;0m
>^[[1;31m--- a/guix/build/utils.scm^[[0;0m
>^[[1;34m+++ b/guix/build/utils.scm^[[0;0m
>^[[1;35m@@ -288,7 +288,8 @@ matches REGEXP."^[[0;0m
>^[[0;0m                                     file (strerror errno))^[[0;0m
>^[[0;0m                             result)^[[0;0m
>^[[0;0m                           '()^[[0;0m
>^[[1;31m-                          dir)^[[0;0m
>^[[1;34m+                          dir^[[0;0m
>^[[1;34m+                          lstat)^[[0;0m
>^[[0;0m         string<?))^[[0;0m
>^[[0;0m ^[[0;0m
>^[[0;0m \f^[[0;0m

>
>Thoughts?

Ignoring symlinks is nice solution.
I'd add comment:

We won't touch broken symlinks, symlinks pointing within the sources will be
fixed anyway.

Thanks!

S_W

[-- Attachment #2: Type: application/pgp-signature, Size: 181 bytes --]

bug#20081: patch-source-shebangs crashes on broken symlink

[Ludovic Courtès]

ludo@gnu.org (Ludovic Courtès) skribis:

> Tomáš Čech <sleep_walker@suse.cz> skribis:
>
>> On Wed, Mar 11, 2015 at 06:32:30PM +0100, Andreas Enge wrote:
>>>On Wed, Mar 11, 2015 at 04:02:11PM +0100, Tomáš Čech wrote:
>>>> I'm trying to create package for taskwarrior.
>>>> Source tarball contain symlinks to nonexisting file `task':
>>>
>>>I would argue that this is not a bug in guix, but in the tarball.
>>>You can remove the link with an additional phase before 'configure, see, for
>>>instance, the dvdisaster package in cdrom.scm.
>>
>> I agree with you that the fishy part is in tarball, but we could make
>> build more robust. Getting backtrace is not nice way to end a build.
>
> I agree.  I think we should patch ‘find-files’ in core-updates to not
> follow symlinks:

Done in 347f54e.

Thanks,
Ludo’.

bug#20081: issue is still not fixed

[Tomáš Čech]

[-- Attachment #1: Type: text/plain, Size: 1858 bytes --]

Hi,

during my clean-up of taskwarrior package I removed section which
removes broken symlinks. Even though this bug is considered as fixed, I
believe I reproduced it after 347f54e.

...
task-2.4.3/src/ViewText.cpp
task-2.4.3/src/ViewText.h
task-2.4.3/src/wcwidth6.cpp
phase `unpack' succeeded after 0 seconds
starting phase `patch-usr-bin-file'
phase `patch-usr-bin-file' succeeded after 0 seconds
starting phase `patch-source-shebangs'
Backtrace:
In ice-9/boot-9.scm:
 157: 15 [catch #t #<catch-closure 2357aa0> ...]
In unknown file:
   ?: 14 [apply-smob/1 #<catch-closure 2357aa0>]
In ice-9/boot-9.scm:
  63: 13 [call-with-prompt prompt0 ...]
In ice-9/eval.scm:
 432: 12 [eval # #]
In ice-9/boot-9.scm:
2401: 11 [save-module-excursion #<procedure 2374880 at ice-9/boot-9.scm:4045:3 ()>]
4050: 10 [#<procedure 2374880 at ice-9/boot-9.scm:4045:3 ()>]
1724: 9 [%start-stack load-stack ...]
1729: 8 [#<procedure 2389c60 ()>]
In unknown file:
   ?: 7 [primitive-load "/gnu/store/06l4f3al56cg49rfiy79risam0f8yb4y-taskwarrior-2.4.3-guile-builder"]
In ice-9/eval.scm:
 387: 6 [eval # ()]
In srfi/srfi-1.scm:
 830: 5 [every1 #<procedure 2590d00 at /gnu/store/j8w9vf3diyvkccsa21p9fqpwsl3wlsy3-module-import/guix/build/gnu-build-system.scm:580:9 (expr)> ...]
In /gnu/store/j8w9vf3diyvkccsa21p9fqpwsl3wlsy3-module-import/guix/build/gnu-build-system.scm:
 584: 4 [#<procedure 2590d00 at /gnu/store/j8w9vf3diyvkccsa21p9fqpwsl3wlsy3-module-import/guix/build/gnu-build-system.scm:580:9 (expr)> #]
 167: 3 [patch-source-shebangs # ...]
In unknown file:
   ?: 2 [remove #<procedure 22cd0e0 at ice-9/boot-9.scm:1460:6 (str)> #]
In ice-9/boot-9.scm:
1461: 1 [#<procedure 22cd0e0 at ice-9/boot-9.scm:1460:6 (str)> "./src/cal"]
In unknown file:
   ?: 0 [stat "./src/cal" #<undefined>]

ERROR: In procedure stat:
ERROR: In procedure stat: No such file or directory: "./src/cal"

[-- Attachment #2: Type: application/pgp-signature, Size: 181 bytes --]

bug#20081: Dangling symlinks gracelessly handled by ‘patch-shebangs’ & co.

[Ludovic Courtès]

Tomáš Čech <tcech@suse.cz> skribis:

> starting phase `patch-source-shebangs'
> Backtrace:
> In ice-9/boot-9.scm:
> 157: 15 [catch #t #<catch-closure 2357aa0> ...]
> In unknown file:
>   ?: 14 [apply-smob/1 #<catch-closure 2357aa0>]
> In ice-9/boot-9.scm:
>  63: 13 [call-with-prompt prompt0 ...]
> In ice-9/eval.scm:
> 432: 12 [eval # #]
> In ice-9/boot-9.scm:
> 2401: 11 [save-module-excursion #<procedure 2374880 at ice-9/boot-9.scm:4045:3 ()>]
> 4050: 10 [#<procedure 2374880 at ice-9/boot-9.scm:4045:3 ()>]
> 1724: 9 [%start-stack load-stack ...]
> 1729: 8 [#<procedure 2389c60 ()>]
> In unknown file:
>   ?: 7 [primitive-load "/gnu/store/06l4f3al56cg49rfiy79risam0f8yb4y-taskwarrior-2.4.3-guile-builder"]
> In ice-9/eval.scm:
> 387: 6 [eval # ()]
> In srfi/srfi-1.scm:
> 830: 5 [every1 #<procedure 2590d00 at /gnu/store/j8w9vf3diyvkccsa21p9fqpwsl3wlsy3-module-import/guix/build/gnu-build-system.scm:580:9 (expr)> ...]
> In /gnu/store/j8w9vf3diyvkccsa21p9fqpwsl3wlsy3-module-import/guix/build/gnu-build-system.scm:
> 584: 4 [#<procedure 2590d00 at /gnu/store/j8w9vf3diyvkccsa21p9fqpwsl3wlsy3-module-import/guix/build/gnu-build-system.scm:580:9 (expr)> #]
> 167: 3 [patch-source-shebangs # ...]
> In unknown file:
>   ?: 2 [remove #<procedure 22cd0e0 at ice-9/boot-9.scm:1460:6 (str)> #]
> In ice-9/boot-9.scm:
> 1461: 1 [#<procedure 22cd0e0 at ice-9/boot-9.scm:1460:6 (str)> "./src/cal"]
> In unknown file:
>   ?: 0 [stat "./src/cal" #<undefined>]

I believe this is Really Fixed™ by cb85eb5 (in core-updates.)

Thanks!

Ludo’.