unofficial mirror of bug-guix@gnu.org 
 help / color / mirror / Atom feed
* bug#50066: core-update: Raw origin tarballs are not handled correctly.
@ 2021-08-15  8:42 Mathieu Othacehe
  2021-08-27  4:19 ` Maxim Cournoyer
  0 siblings, 1 reply; 6+ messages in thread
From: Mathieu Othacehe @ 2021-08-15  8:42 UTC (permalink / raw)
  To: 50066; +Cc: Maxim Cournoyer


Hello,

When building "emacs-org-contrib" on core-updates-frozen, the following
message is printed:

--8<---------------cut here---------------start------------->8---
tar: Exiting with failure status due to previous errors
error: in phase 'unpack': uncaught exception:
%exception #<&invoke-error program: "tar" arguments: ("xvf" "/gnu/store/r3v4gkqqbxp2w739a0llvk9nrda9xn89-org-plus-contrib-20210809.tar") exit-status: 2 term-signal: #f stop-signal: #f> 
phase `unpack' failed after 0.0 seconds
command "tar" "xvf" "/gnu/store/r3v4gkqqbxp2w739a0llvk9nrda9xn89-org-plus-contrib-20210809.tar" failed with status 2
note: keeping build directory `/tmp/guix-build-emacs-org-contrib-20210809.drv-6'
builder for `/gnu/store/52jcx6rwnf0wrkrpcwkyz766fzg6asjy-emacs-org-contrib-20210809.drv' failed with exit code 1
build of /gnu/store/52jcx6rwnf0wrkrpcwkyz766fzg6asjy-emacs-org-contrib-20210809.drv failed
View build log at '/var/log/guix/drvs/52/jcx6rwnf0wrkrpcwkyz766fzg6asjy-emacs-org-contrib-20210809.drv.bz2'.
guix build: error: build of `/gnu/store/52jcx6rwnf0wrkrpcwkyz766fzg6asjy-emacs-org-contrib-20210809.drv' failed
--8<---------------cut here---------------end--------------->8---

That's because
"/gnu/store/r3v4gkqqbxp2w739a0llvk9nrda9xn89-org-plus-contrib-20210809.tar"
is empty. I believe this is a consequence of:
cfcead2e515c0dae02127e5a76496463898be6b6, and in particular this
section:

--8<---------------cut here---------------start------------->8---
              ;; If SOURCE is a directory (such as a checkout), return a
              ;; directory.  Otherwise create a tarball.
              (cond
               ((file-is-directory? #+source)
                (copy-recursively directory #$output
                                  #:log (%make-void-port "w")))
               ((not #+comp)
                (copy-file file #$output)) <---------- We should call repack instead here for raw tarballs?
               (else
                (repack directory #$output)))))))
--8<---------------cut here---------------end--------------->8---

Maxim, could you please have a look?

Thanks,

Mathieu




^ permalink raw reply	[flat|nested] 6+ messages in thread

* bug#50066: core-update: Raw origin tarballs are not handled correctly.
  2021-08-15  8:42 bug#50066: core-update: Raw origin tarballs are not handled correctly Mathieu Othacehe
@ 2021-08-27  4:19 ` Maxim Cournoyer
  2021-08-31  3:50   ` Maxim Cournoyer
  0 siblings, 1 reply; 6+ messages in thread
From: Maxim Cournoyer @ 2021-08-27  4:19 UTC (permalink / raw)
  To: Mathieu Othacehe; +Cc: 50066

Hello Mathieu,

Mathieu Othacehe <othacehe@gnu.org> writes:

> Hello,
>
> When building "emacs-org-contrib" on core-updates-frozen, the following
> message is printed:
>
> tar: Exiting with failure status due to previous errors
> error: in phase 'unpack': uncaught exception:
> %exception #<&invoke-error program: "tar" arguments: ("xvf" "/gnu/store/r3v4gkqqbxp2w739a0llvk9nrda9xn89-org-plus-contrib-20210809.tar") exit-status: 2 term-signal: #f stop-signal: #f> 
> phase `unpack' failed after 0.0 seconds
> command "tar" "xvf" "/gnu/store/r3v4gkqqbxp2w739a0llvk9nrda9xn89-org-plus-contrib-20210809.tar" failed with status 2
> note: keeping build directory `/tmp/guix-build-emacs-org-contrib-20210809.drv-6'
> builder for `/gnu/store/52jcx6rwnf0wrkrpcwkyz766fzg6asjy-emacs-org-contrib-20210809.drv' failed with exit code 1
> build of /gnu/store/52jcx6rwnf0wrkrpcwkyz766fzg6asjy-emacs-org-contrib-20210809.drv failed
> View build log at '/var/log/guix/drvs/52/jcx6rwnf0wrkrpcwkyz766fzg6asjy-emacs-org-contrib-20210809.drv.bz2'.
> guix build: error: build of `/gnu/store/52jcx6rwnf0wrkrpcwkyz766fzg6asjy-emacs-org-contrib-20210809.drv' failed
>
>
> That's because
> "/gnu/store/r3v4gkqqbxp2w739a0llvk9nrda9xn89-org-plus-contrib-20210809.tar"
> is empty. I believe this is a consequence of:
> cfcead2e515c0dae02127e5a76496463898be6b6, and in particular this
> section:
>
>               ;; If SOURCE is a directory (such as a checkout), return a
>               ;; directory.  Otherwise create a tarball.
>               (cond
>                ((file-is-directory? #+source)
>                 (copy-recursively directory #$output
>                                   #:log (%make-void-port "w")))
>                ((not #+comp)
>                 (copy-file file #$output)) <---------- We should call repack instead here for raw tarballs?
>                (else
>                 (repack directory #$output)))))))
>
> Maxim, could you please have a look?

Thanks for the report; it looks like a bug indeed.  I had probably
overlooked that case (where there is no compression but we are still
dealing with an archive format).  The annoying thing is that touching
this code will trigger a world rebuild; so unless we find other big
problems and batch the fixes, it may not be worth it.

The site.py problem you reported may be one, if I find a solution.

Other things on my mind:

1. rustc bootstrap from 1.39 (the itch to make it faster comes back
everytime I have to build it ;-)).

2. fontconfig update that should allow per-profile fonts management via
  XDG_DATA_DIRS (I have the commit ready, but failed to test it due to
  having to build rust and getting distracted by 1. :-))

Not a very useful reply in itself, but I felt that communicating what
I've been busy with was better than a longer silence :-).

Thank you,

Maxim




^ permalink raw reply	[flat|nested] 6+ messages in thread

* bug#50066: core-update: Raw origin tarballs are not handled correctly.
  2021-08-27  4:19 ` Maxim Cournoyer
@ 2021-08-31  3:50   ` Maxim Cournoyer
  2021-09-01  7:09     ` Mathieu Othacehe
  0 siblings, 1 reply; 6+ messages in thread
From: Maxim Cournoyer @ 2021-08-31  3:50 UTC (permalink / raw)
  To: Mathieu Othacehe; +Cc: 50066

Hello Mathieu,

[...]

> Thanks for the report; it looks like a bug indeed.  I had probably
> overlooked that case (where there is no compression but we are still
> dealing with an archive format).  The annoying thing is that touching
> this code will trigger a world rebuild; so unless we find other big
> problems and batch the fixes, it may not be worth it.
>
> The site.py problem you reported may be one, if I find a solution.
>
> Other things on my mind:
>
> 1. rustc bootstrap from 1.39 (the itch to make it faster comes back
> everytime I have to build it ;-)).

I'm nearing completion on this front.

> 2. fontconfig update that should allow per-profile fonts management via
>   XDG_DATA_DIRS (I have the commit ready, but failed to test it due to
>   having to build rust and getting distracted by 1. :-))

I'll propose a few patches to include in the 'frozen' core-updates
branch since we've found a world-rebuilding change worthy of addressing
anyway (the patch below).

1 file changed, 4 insertions(+), 4 deletions(-)
guix/packages.scm | 8 ++++----

modified   guix/packages.scm
@@ -836,10 +836,10 @@ specifies modules in scope when evaluating SNIPPET."
                ((file-is-directory? #+source)
                 (copy-recursively directory #$output
                                   #:log (%make-void-port "w")))
-               ((not #+comp)
-                (copy-file file #$output))
-               (else
-                (repack directory #$output)))))))
+               ((or #+comp (tarball? #+source))
+                (repack directory #$output))
+               (else                    ;single uncompressed file
+                (copy-file file #$output)))))))
 
     (let ((name (if (or (checkout? original-file-name)
                         (not (compressor original-file-name)))

Sadly the plain .tar file was/is not covered by tests (and not easily
so), but if his was my only omission, it should be correct (TM).
Attentive review needed :-).

Thank you!

Maxim




^ permalink raw reply	[flat|nested] 6+ messages in thread

* bug#50066: core-update: Raw origin tarballs are not handled correctly.
  2021-08-31  3:50   ` Maxim Cournoyer
@ 2021-09-01  7:09     ` Mathieu Othacehe
  2021-09-01 12:57       ` Maxim Cournoyer
  2021-09-03 13:30       ` Maxim Cournoyer
  0 siblings, 2 replies; 6+ messages in thread
From: Mathieu Othacehe @ 2021-09-01  7:09 UTC (permalink / raw)
  To: Maxim Cournoyer; +Cc: 50066


Hello Maxim,

>> 1. rustc bootstrap from 1.39 (the itch to make it faster comes back
>> everytime I have to build it ;-)).
>
> I'm nearing completion on this front.

That will be a welcomed improvement!

>> 2. fontconfig update that should allow per-profile fonts management via
>>   XDG_DATA_DIRS (I have the commit ready, but failed to test it due to
>>   having to build rust and getting distracted by 1. :-))
>
> I'll propose a few patches to include in the 'frozen' core-updates
> branch since we've found a world-rebuilding change worthy of addressing
> anyway (the patch below).

Great! As we are going for a full rebuild, maybe we should also consider
some of the Thiago fixes targeting core-updates-frozen such as
https://issues.guix.gnu.org/50239.

> Sadly the plain .tar file was/is not covered by tests (and not easily
> so), but if his was my only omission, it should be correct (TM).
> Attentive review needed :-).

This seems reasonable and I'm currently rebuilding the world to test
this fix and the Python fix. I should maybe apply your Rust bootstrap
patch if you have it around to gain some time.

Thanks for all those fixes!

Mathieu




^ permalink raw reply	[flat|nested] 6+ messages in thread

* bug#50066: core-update: Raw origin tarballs are not handled correctly.
  2021-09-01  7:09     ` Mathieu Othacehe
@ 2021-09-01 12:57       ` Maxim Cournoyer
  2021-09-03 13:30       ` Maxim Cournoyer
  1 sibling, 0 replies; 6+ messages in thread
From: Maxim Cournoyer @ 2021-09-01 12:57 UTC (permalink / raw)
  To: Mathieu Othacehe; +Cc: 50066

Hi Mathieu,

Mathieu Othacehe <othacehe@gnu.org> writes:

> Hello Maxim,
>
>>> 1. rustc bootstrap from 1.39 (the itch to make it faster comes back
>>> everytime I have to build it ;-)).
>>
>> I'm nearing completion on this front.
>
> That will be a welcomed improvement!
>
>>> 2. fontconfig update that should allow per-profile fonts management via
>>>   XDG_DATA_DIRS (I have the commit ready, but failed to test it due to
>>>   having to build rust and getting distracted by 1. :-))
>>
>> I'll propose a few patches to include in the 'frozen' core-updates
>> branch since we've found a world-rebuilding change worthy of addressing
>> anyway (the patch below).
>
> Great! As we are going for a full rebuild, maybe we should also consider
> some of the Thiago fixes targeting core-updates-frozen such as
> https://issues.guix.gnu.org/50239.

Yes, I think we should cherry-pick anything valuable before the big
rebuild of the frozen branch (perhaps we should simply sync with
core-updates?)

>> Sadly the plain .tar file was/is not covered by tests (and not easily
>> so), but if his was my only omission, it should be correct (TM).
>> Attentive review needed :-).
>
> This seems reasonable and I'm currently rebuilding the world to test
> this fix and the Python fix. I should maybe apply your Rust bootstrap
> patch if you have it around to gain some time.

It's currently building Rust 1.47 (it takes about 15 minutes per Rust).
I'll build it all until 1.54, then recheck a few ones for
reproducibility (at least the first and last ones).  If all is good I'll
send the patches I have.

> Thanks for all those fixes!

Thank you for the testing/reports!

Maxim




^ permalink raw reply	[flat|nested] 6+ messages in thread

* bug#50066: core-update: Raw origin tarballs are not handled correctly.
  2021-09-01  7:09     ` Mathieu Othacehe
  2021-09-01 12:57       ` Maxim Cournoyer
@ 2021-09-03 13:30       ` Maxim Cournoyer
  1 sibling, 0 replies; 6+ messages in thread
From: Maxim Cournoyer @ 2021-09-03 13:30 UTC (permalink / raw)
  To: Mathieu Othacehe; +Cc: 50066

Hi Mathieu,

Mathieu Othacehe <othacehe@gnu.org> writes:

> Hello Maxim,
>
>>> 1. rustc bootstrap from 1.39 (the itch to make it faster comes back
>>> everytime I have to build it ;-)).
>>
>> I'm nearing completion on this front.
>
> That will be a welcomed improvement!
>
>>> 2. fontconfig update that should allow per-profile fonts management via
>>>   XDG_DATA_DIRS (I have the commit ready, but failed to test it due to
>>>   having to build rust and getting distracted by 1. :-))
>>
>> I'll propose a few patches to include in the 'frozen' core-updates
>> branch since we've found a world-rebuilding change worthy of addressing
>> anyway (the patch below).
>
> Great! As we are going for a full rebuild, maybe we should also consider
> some of the Thiago fixes targeting core-updates-frozen such as
> https://issues.guix.gnu.org/50239.
>
>> Sadly the plain .tar file was/is not covered by tests (and not easily
>> so), but if his was my only omission, it should be correct (TM).
>> Attentive review needed :-).
>
> This seems reasonable and I'm currently rebuilding the world to test
> this fix and the Python fix. I should maybe apply your Rust bootstrap
> patch if you have it around to gain some time.

successfully built /gnu/store/vag2a53byhgrkpchviqz78lilall6412-emacs-org-contrib-20210519.drv
/gnu/store/40hl0dli745i2ggs1aqhn2drh0fkmrbx-emacs-org-contrib-20210519

:-).

I'm trying to fix one last thing with my local patch set (mozjs not
building with rust@1.54), but if takes too much time, I'll send what I
have and we can figure it out together.

Thanks,

Maxim




^ permalink raw reply	[flat|nested] 6+ messages in thread

end of thread, other threads:[~2021-09-03 13:47 UTC | newest]

Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-08-15  8:42 bug#50066: core-update: Raw origin tarballs are not handled correctly Mathieu Othacehe
2021-08-27  4:19 ` Maxim Cournoyer
2021-08-31  3:50   ` Maxim Cournoyer
2021-09-01  7:09     ` Mathieu Othacehe
2021-09-01 12:57       ` Maxim Cournoyer
2021-09-03 13:30       ` Maxim Cournoyer

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).