[core-updates] issues with python's zip

[core-updates] issues with python's zip

[Brian Cully via Development of GNU Guix and the GNU System distribution.]

I've run into issues with two packages on core-updates that fail 
to build because the zip library being used can't handle dates 
before 1980. I assume there are more.

Do we have a way to handle this gracefully? In the mean time I've 
patched both packages (‘sssd’ and ‘criu’) with a new build phase:

--8<---------------cut here---------------start------------->8---
          (add-before 'install 'change-file-datetimes
            (lambda _
              (use-modules (ice-9 ftw))
              (let ((dir ".")
                    (time 315532800)) ; 00:00:00 1-jan-1980 UTC
                (let loop ((prefix "")
                           (node (file-system-tree (pk ".:" 
                           dir))))
                  ;; unqualified use of ‘match’ leads to a syntax 
                  error when
                  ;; using ‘...’, so specify the module 
                  explicitly.
                  ((@ (ice-9 match) match) node
                   ((name stat) ; flat file
                    (when (not (eq? (stat:type stat) 'symlink))
                      (utime (string-append prefix name) time 
                      time)))
                   ((name stat children ...) ; directory
                    (utime (string-append prefix name) time time)
                    (for-each (lambda (child)
                                (loop (string-append prefix name
                                                     file-name-separator-string)
                                      child))
                              children)))))))
--8<---------------cut here---------------end--------------->8---

With this, the only outstanding issue with core-updates is #62820, 
which I'm baffled by. It appears to me as though the substitute is 
issuing up a broken ‘rules’ file, but since this is used by the 
keymap in ‘operating-system’, I would imagine a lot of people 
would have hit this bug, so why seemingly just me?

-bjc

Re: [core-updates] issues with python's zip

[Felix Lechner via Development of GNU Guix and the GNU System distribution.]

Hi Brian,

On Fri, Apr 14, 2023 at 3:57 PM Brian Cully wrote:
>
> With this, the only outstanding issue with core-updates is #62820

I believe that issue is potentially solved by my amendment to the
aforementioned bug. [1]

Thanks for your hard work on the core-updates branch.

Please have a good weekend!

Kind regards
Felix Lechner

[1] https://issues.guix.gnu.org/62820#1

Re: [core-updates] issues with python's zip

[Andreas Enge]

Hello,

Am Fri, Apr 14, 2023 at 06:47:04PM -0400 schrieb Brian Cully via Development of GNU Guix and the GNU System distribution.:
> I've run into issues with two packages on core-updates that fail to build
> because the zip library being used can't handle dates before 1980. I assume
> there are more.
> Do we have a way to handle this gracefully? In the mean time I've patched
> both packages (‘sssd’ and ‘criu’) with a new build phase:

criu fails like this for me:
copying crit.egg-info/top_level.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
zip_safe flag not set; analyzing archive contents...
pycriu.images.__pycache__.__init__.cpython-310: module references __file__
creating dist
creating 'dist/crit-3.17.1-py3.10.egg' and adding 'build/bdist.linux-x86_64/egg' to it
Traceback (most recent call last):
  File "/tmp/guix-build-criu-3.17.1.drv-0/source/scripts/crit-setup.py", line 16, in <module>
    setup(name="crit",
  File "/gnu/store/dy3xh053ahkhrp2jamggq8cpsyvp8mg0-python-3.10.7/lib/python3.10/site-packages/setuptools/_distutils/core.py", line 177, in setup
    return run_commands(dist)
  File "/gnu/store/dy3xh053ahkhrp2jamggq8cpsyvp8mg0-python-3.10.7/lib/python3.10/site-packages/setuptools/_distutils/core.py", line 193, in run_commands
    dist.run_commands()
  File "/gnu/store/dy3xh053ahkhrp2jamggq8cpsyvp8mg0-python-3.10.7/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 968, in run_commands
    self.run_command(cmd)
  File "/gnu/store/dy3xh053ahkhrp2jamggq8cpsyvp8mg0-python-3.10.7/lib/python3.10/site-packages/setuptools/dist.py", line 1217, in run_command
    super().run_command(command)
  File "/gnu/store/dy3xh053ahkhrp2jamggq8cpsyvp8mg0-python-3.10.7/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 987, in run_command
    cmd_obj.run()
  File "/gnu/store/dy3xh053ahkhrp2jamggq8cpsyvp8mg0-python-3.10.7/lib/python3.10/site-packages/setuptools/command/install.py", line 74, in run
    self.do_egg_install()
  File "/gnu/store/dy3xh053ahkhrp2jamggq8cpsyvp8mg0-python-3.10.7/lib/python3.10/site-packages/setuptools/command/install.py", line 123, in do_egg_install
    self.run_command('bdist_egg')
  File "/gnu/store/dy3xh053ahkhrp2jamggq8cpsyvp8mg0-python-3.10.7/lib/python3.10/site-packages/setuptools/_distutils/cmd.py", line 317, in run_command
    self.distribution.run_command(command)
  File "/gnu/store/dy3xh053ahkhrp2jamggq8cpsyvp8mg0-python-3.10.7/lib/python3.10/site-packages/setuptools/dist.py", line 1217, in run_command
    super().run_command(command)
  File "/gnu/store/dy3xh053ahkhrp2jamggq8cpsyvp8mg0-python-3.10.7/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 987, in run_command
    cmd_obj.run()
  File "/gnu/store/dy3xh053ahkhrp2jamggq8cpsyvp8mg0-python-3.10.7/lib/python3.10/site-packages/setuptools/command/bdist_egg.py", line 226, in run
    make_zipfile(self.egg_output, archive_root, verbose=self.verbose,
  File "/gnu/store/dy3xh053ahkhrp2jamggq8cpsyvp8mg0-python-3.10.7/lib/python3.10/site-packages/setuptools/command/bdist_egg.py", line 452, in make_zipfile
    visit(z, dirname, files)
  File "/gnu/store/dy3xh053ahkhrp2jamggq8cpsyvp8mg0-python-3.10.7/lib/python3.10/site-packages/setuptools/command/bdist_egg.py", line 445, in visit
    z.write(path, p)
  File "/gnu/store/dy3xh053ahkhrp2jamggq8cpsyvp8mg0-python-3.10.7/lib/python3.10/zipfile.py", line 1739, in write
    zinfo = ZipInfo.from_file(filename, arcname,
  File "/gnu/store/dy3xh053ahkhrp2jamggq8cpsyvp8mg0-python-3.10.7/lib/python3.10/zipfile.py", line 518, in from_file
    zinfo = cls(arcname, date_time)
  File "/gnu/store/dy3xh053ahkhrp2jamggq8cpsyvp8mg0-python-3.10.7/lib/python3.10/zipfile.py", line 362, in __init__
    raise ValueError('ZIP does not support timestamps before 1980')
ValueError: ZIP does not support timestamps before 1980


I came across this:
   https://docs.python.org/3/library/zipfile.html#zipfile-objects
So apparently when creating a zipfile object (this is the constructor
we are looking at, right?), one can pass a parameter that tells it to
not bother about time stamps of old files.

Then it is a matter of finding out where it happens. I think it is
line 450 of (in my case)
/gnu/store/dy3xh053ahkhrp2jamggq8cpsyvp8mg0-python-3.10.7/lib/python3.10/site-packages/setuptools/command/bdist_egg.py
which reads
        z = zipfile.ZipFile(zip_filename, mode, compression=compression)
Maybe we should patch our Python so that it becomes
        z = zipfile.ZipFile(zip_filename, mode, compression=compression, strict_timestamps=False)
?
(not before the core-updates merge, however)

But is a bit surprising that it does not happen all the time;
or are only a few packages calling bdist_egg.py?

Andreas

Re: [core-updates] issues with python's zip

[Brian Cully via Development of GNU Guix and the GNU System distribution.]


Andreas Enge <andreas@enge.fr> writes:

> I came across this:
>   https://docs.python.org/3/library/zipfile.html#zipfile-objects
> So apparently when creating a zipfile object (this is the constructor
> we are looking at, right?), one can pass a parameter that tells it to
> not bother about time stamps of old files.
> 
> Then it is a matter of finding out where it happens. I think it is
> line 450 of (in my case)
> /gnu/store/dy3xh053ahkhrp2jamggq8cpsyvp8mg0-python-3.10.7/lib/python3.10/site-packages/setuptools/command/bdist_egg.py
> which reads
>        z = zipfile.ZipFile(zip_filename, mode, compression=compression)
> Maybe we should patch our Python so that it becomes
>        z = zipfile.ZipFile(zip_filename, mode, compression=compression, strict_timestamps=False)
> ?
> (not before the core-updates merge, however)

Thanks for digging into this. Maybe it makes sense to patch bdist_egg.py in setuptools, then, rather than chase down every failed build?

> But is a bit surprising that it does not happen all the time;
> or are only a few packages calling bdist_egg.py?

I don't know. According to https://www.python101.pythonlibrary.org/chapter38_eggs.html it's an older format, which has been superseded by ‘wheel’. I know I see wheel a fair amount, but I'm not much of a Python hacker.

-bjc