all messages for Guix-related lists mirrored at yhetil.org
 help / color / mirror / code / Atom feed
* [core-updates] issues with python's zip
@ 2023-04-14 22:47 Brian Cully via Development of GNU Guix and the GNU System distribution.
  2023-04-15  1:21 ` Felix Lechner via Development of GNU Guix and the GNU System distribution.
  2023-04-19 22:19 ` Andreas Enge
  0 siblings, 2 replies; 4+ messages in thread
From: Brian Cully via Development of GNU Guix and the GNU System distribution. @ 2023-04-14 22:47 UTC (permalink / raw)
  To: guix-devel


I've run into issues with two packages on core-updates that fail 
to build because the zip library being used can't handle dates 
before 1980. I assume there are more.

Do we have a way to handle this gracefully? In the mean time I've 
patched both packages (‘sssd’ and ‘criu’) with a new build phase:

--8<---------------cut here---------------start------------->8---
          (add-before 'install 'change-file-datetimes
            (lambda _
              (use-modules (ice-9 ftw))
              (let ((dir ".")
                    (time 315532800)) ; 00:00:00 1-jan-1980 UTC
                (let loop ((prefix "")
                           (node (file-system-tree (pk ".:" 
                           dir))))
                  ;; unqualified use of ‘match’ leads to a syntax 
                  error when
                  ;; using ‘...’, so specify the module 
                  explicitly.
                  ((@ (ice-9 match) match) node
                   ((name stat) ; flat file
                    (when (not (eq? (stat:type stat) 'symlink))
                      (utime (string-append prefix name) time 
                      time)))
                   ((name stat children ...) ; directory
                    (utime (string-append prefix name) time time)
                    (for-each (lambda (child)
                                (loop (string-append prefix name
                                                     file-name-separator-string)
                                      child))
                              children)))))))
--8<---------------cut here---------------end--------------->8---

With this, the only outstanding issue with core-updates is #62820, 
which I'm baffled by. It appears to me as though the substitute is 
issuing up a broken ‘rules’ file, but since this is used by the 
keymap in ‘operating-system’, I would imagine a lot of people 
would have hit this bug, so why seemingly just me?

-bjc


^ permalink raw reply	[flat|nested] 4+ messages in thread

* Re: [core-updates] issues with python's zip
  2023-04-14 22:47 [core-updates] issues with python's zip Brian Cully via Development of GNU Guix and the GNU System distribution.
@ 2023-04-15  1:21 ` Felix Lechner via Development of GNU Guix and the GNU System distribution.
  2023-04-19 22:19 ` Andreas Enge
  1 sibling, 0 replies; 4+ messages in thread
From: Felix Lechner via Development of GNU Guix and the GNU System distribution. @ 2023-04-15  1:21 UTC (permalink / raw)
  To: Brian Cully; +Cc: guix-devel

Hi Brian,

On Fri, Apr 14, 2023 at 3:57 PM Brian Cully wrote:
>
> With this, the only outstanding issue with core-updates is #62820

I believe that issue is potentially solved by my amendment to the
aforementioned bug. [1]

Thanks for your hard work on the core-updates branch.

Please have a good weekend!

Kind regards
Felix Lechner

[1] https://issues.guix.gnu.org/62820#1


^ permalink raw reply	[flat|nested] 4+ messages in thread

* Re: [core-updates] issues with python's zip
  2023-04-14 22:47 [core-updates] issues with python's zip Brian Cully via Development of GNU Guix and the GNU System distribution.
  2023-04-15  1:21 ` Felix Lechner via Development of GNU Guix and the GNU System distribution.
@ 2023-04-19 22:19 ` Andreas Enge
  1 sibling, 0 replies; 4+ messages in thread
From: Andreas Enge @ 2023-04-19 22:19 UTC (permalink / raw)
  To: Brian Cully; +Cc: guix-devel

Hello,

Am Fri, Apr 14, 2023 at 06:47:04PM -0400 schrieb Brian Cully via Development of GNU Guix and the GNU System distribution.:
> I've run into issues with two packages on core-updates that fail to build
> because the zip library being used can't handle dates before 1980. I assume
> there are more.
> Do we have a way to handle this gracefully? In the mean time I've patched
> both packages (‘sssd’ and ‘criu’) with a new build phase:

criu fails like this for me:
copying crit.egg-info/top_level.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
zip_safe flag not set; analyzing archive contents...
pycriu.images.__pycache__.__init__.cpython-310: module references __file__
creating dist
creating 'dist/crit-3.17.1-py3.10.egg' and adding 'build/bdist.linux-x86_64/egg' to it
Traceback (most recent call last):
  File "/tmp/guix-build-criu-3.17.1.drv-0/source/scripts/crit-setup.py", line 16, in <module>
    setup(name="crit",
  File "/gnu/store/dy3xh053ahkhrp2jamggq8cpsyvp8mg0-python-3.10.7/lib/python3.10/site-packages/setuptools/_distutils/core.py", line 177, in setup
    return run_commands(dist)
  File "/gnu/store/dy3xh053ahkhrp2jamggq8cpsyvp8mg0-python-3.10.7/lib/python3.10/site-packages/setuptools/_distutils/core.py", line 193, in run_commands
    dist.run_commands()
  File "/gnu/store/dy3xh053ahkhrp2jamggq8cpsyvp8mg0-python-3.10.7/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 968, in run_commands
    self.run_command(cmd)
  File "/gnu/store/dy3xh053ahkhrp2jamggq8cpsyvp8mg0-python-3.10.7/lib/python3.10/site-packages/setuptools/dist.py", line 1217, in run_command
    super().run_command(command)
  File "/gnu/store/dy3xh053ahkhrp2jamggq8cpsyvp8mg0-python-3.10.7/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 987, in run_command
    cmd_obj.run()
  File "/gnu/store/dy3xh053ahkhrp2jamggq8cpsyvp8mg0-python-3.10.7/lib/python3.10/site-packages/setuptools/command/install.py", line 74, in run
    self.do_egg_install()
  File "/gnu/store/dy3xh053ahkhrp2jamggq8cpsyvp8mg0-python-3.10.7/lib/python3.10/site-packages/setuptools/command/install.py", line 123, in do_egg_install
    self.run_command('bdist_egg')
  File "/gnu/store/dy3xh053ahkhrp2jamggq8cpsyvp8mg0-python-3.10.7/lib/python3.10/site-packages/setuptools/_distutils/cmd.py", line 317, in run_command
    self.distribution.run_command(command)
  File "/gnu/store/dy3xh053ahkhrp2jamggq8cpsyvp8mg0-python-3.10.7/lib/python3.10/site-packages/setuptools/dist.py", line 1217, in run_command
    super().run_command(command)
  File "/gnu/store/dy3xh053ahkhrp2jamggq8cpsyvp8mg0-python-3.10.7/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 987, in run_command
    cmd_obj.run()
  File "/gnu/store/dy3xh053ahkhrp2jamggq8cpsyvp8mg0-python-3.10.7/lib/python3.10/site-packages/setuptools/command/bdist_egg.py", line 226, in run
    make_zipfile(self.egg_output, archive_root, verbose=self.verbose,
  File "/gnu/store/dy3xh053ahkhrp2jamggq8cpsyvp8mg0-python-3.10.7/lib/python3.10/site-packages/setuptools/command/bdist_egg.py", line 452, in make_zipfile
    visit(z, dirname, files)
  File "/gnu/store/dy3xh053ahkhrp2jamggq8cpsyvp8mg0-python-3.10.7/lib/python3.10/site-packages/setuptools/command/bdist_egg.py", line 445, in visit
    z.write(path, p)
  File "/gnu/store/dy3xh053ahkhrp2jamggq8cpsyvp8mg0-python-3.10.7/lib/python3.10/zipfile.py", line 1739, in write
    zinfo = ZipInfo.from_file(filename, arcname,
  File "/gnu/store/dy3xh053ahkhrp2jamggq8cpsyvp8mg0-python-3.10.7/lib/python3.10/zipfile.py", line 518, in from_file
    zinfo = cls(arcname, date_time)
  File "/gnu/store/dy3xh053ahkhrp2jamggq8cpsyvp8mg0-python-3.10.7/lib/python3.10/zipfile.py", line 362, in __init__
    raise ValueError('ZIP does not support timestamps before 1980')
ValueError: ZIP does not support timestamps before 1980


I came across this:
   https://docs.python.org/3/library/zipfile.html#zipfile-objects
So apparently when creating a zipfile object (this is the constructor
we are looking at, right?), one can pass a parameter that tells it to
not bother about time stamps of old files.

Then it is a matter of finding out where it happens. I think it is
line 450 of (in my case)
/gnu/store/dy3xh053ahkhrp2jamggq8cpsyvp8mg0-python-3.10.7/lib/python3.10/site-packages/setuptools/command/bdist_egg.py
which reads
        z = zipfile.ZipFile(zip_filename, mode, compression=compression)
Maybe we should patch our Python so that it becomes
        z = zipfile.ZipFile(zip_filename, mode, compression=compression, strict_timestamps=False)
?
(not before the core-updates merge, however)

But is a bit surprising that it does not happen all the time;
or are only a few packages calling bdist_egg.py?

Andreas



^ permalink raw reply	[flat|nested] 4+ messages in thread

* Re: [core-updates] issues with python's zip
@ 2023-04-20  3:20 Brian Cully via Development of GNU Guix and the GNU System distribution.
  0 siblings, 0 replies; 4+ messages in thread
From: Brian Cully via Development of GNU Guix and the GNU System distribution. @ 2023-04-20  3:20 UTC (permalink / raw)
  To: Andreas Enge; +Cc: guix-devel


Andreas Enge <andreas@enge.fr> writes:

> I came across this:
>   https://docs.python.org/3/library/zipfile.html#zipfile-objects
> So apparently when creating a zipfile object (this is the constructor
> we are looking at, right?), one can pass a parameter that tells it to
> not bother about time stamps of old files.
> 
> Then it is a matter of finding out where it happens. I think it is
> line 450 of (in my case)
> /gnu/store/dy3xh053ahkhrp2jamggq8cpsyvp8mg0-python-3.10.7/lib/python3.10/site-packages/setuptools/command/bdist_egg.py
> which reads
>        z = zipfile.ZipFile(zip_filename, mode, compression=compression)
> Maybe we should patch our Python so that it becomes
>        z = zipfile.ZipFile(zip_filename, mode, compression=compression, strict_timestamps=False)
> ?
> (not before the core-updates merge, however)

Thanks for digging into this. Maybe it makes sense to patch bdist_egg.py in setuptools, then, rather than chase down every failed build?

> But is a bit surprising that it does not happen all the time;
> or are only a few packages calling bdist_egg.py?

I don't know. According to https://www.python101.pythonlibrary.org/chapter38_eggs.html it's an older format, which has been superseded by ‘wheel’. I know I see wheel a fair amount, but I'm not much of a Python hacker.

-bjc


^ permalink raw reply	[flat|nested] 4+ messages in thread

end of thread, other threads:[~2023-04-20  3:20 UTC | newest]

Thread overview: 4+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2023-04-14 22:47 [core-updates] issues with python's zip Brian Cully via Development of GNU Guix and the GNU System distribution.
2023-04-15  1:21 ` Felix Lechner via Development of GNU Guix and the GNU System distribution.
2023-04-19 22:19 ` Andreas Enge
  -- strict thread matches above, loose matches on Subject: below --
2023-04-20  3:20 Brian Cully via Development of GNU Guix and the GNU System distribution.

Code repositories for project(s) associated with this external index

	https://git.savannah.gnu.org/cgit/guix.git

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.