bug#66647: Installation of RPMs produced by ‘guix pack’ is super slow

bug#66647: Installation of RPMs produced by ‘guix pack’ is super slow

[Ludovic Courtès]

Hello!

I made friends at work when I told them we could provide an RPM for any
modern package to install on their old RPM-based distro.  :-)

However, installing those RPMs takes a lot of time.  For example,
installing the RPM for ‘gmsh’ (closure: 596 MiB; thousands of files)
takes ~45mn.

Is there something about the metadata generated by (guix rpm) that could
be improved, or are we hitting some limitation of the RPM format or
implementation?

Thanks,
Ludo’.

bug#66647: Installation of RPMs produced by ‘guix pack’ is super slow

[Maxim Cournoyer]

Hi!

Ludovic Courtès <ludovic.courtes@inria.fr> writes:

> Hello!
>
> I made friends at work when I told them we could provide an RPM for any
> modern package to install on their old RPM-based distro.  :-)
>
> However, installing those RPMs takes a lot of time.  For example,
> installing the RPM for ‘gmsh’ (closure: 596 MiB; thousands of files)
> takes ~45mn.
>
> Is there something about the metadata generated by (guix rpm) that could
> be improved, or are we hitting some limitation of the RPM format or
> implementation?

What?  That's crazy.  I haven't experimented with it recently, but I
used to generate packages for Jami back when it had a closure of 2 GiB
and it took many seconds, but not that much.

Many RPM is doing extra checks now?

-- 
Thanks,
Maxim

bug#66647: Installation of RPMs produced by ‘guix pack’ is super slow

[Maxim Cournoyer]

Hi Ludo,

Ludovic Courtès <ludovic.courtes@inria.fr> writes:

> Hello!
>
> I made friends at work when I told them we could provide an RPM for any
> modern package to install on their old RPM-based distro.  :-)
>
> However, installing those RPMs takes a lot of time.  For example,
> installing the RPM for ‘gmsh’ (closure: 596 MiB; thousands of files)
> takes ~45mn.
>
> Is there something about the metadata generated by (guix rpm) that could
> be improved, or are we hitting some limitation of the RPM format or
> implementation?

What is the OS thy install on?  How do they generate the gmsh package
exactly?  Perhaps I still have a RPM-based distro VM to try it with.

-- 
Thanks,
Maxim

bug#66647: Installation of RPMs produced by ‘guix pack’ is super slow

[Ludovic Courtès]

Hi!

Maxim Cournoyer <maxim.cournoyer@gmail.com> skribis:

> Ludovic Courtès <ludovic.courtes@inria.fr> writes:

[...]

>> However, installing those RPMs takes a lot of time.  For example,
>> installing the RPM for ‘gmsh’ (closure: 596 MiB; thousands of files)
>> takes ~45mn.

[...]

> What is the OS thy install on?  How do they generate the gmsh package
> exactly?  Perhaps I still have a RPM-based distro VM to try it with.

They’re targeting CentOS 7 (!).  It’s built with:

  guix pack -f rpm -S /opt/bin=bin -R gmsh

Ludo’.

bug#66647: Installation of RPMs produced by ‘guix pack’ is super slow

[Maxim Cournoyer]

Hi,

Ludovic Courtès <ludo@gnu.org> writes:

> Hi!
>
> Maxim Cournoyer <maxim.cournoyer@gmail.com> skribis:
>
>> Ludovic Courtès <ludovic.courtes@inria.fr> writes:
>
> [...]
>
>>> However, installing those RPMs takes a lot of time.  For example,
>>> installing the RPM for ‘gmsh’ (closure: 596 MiB; thousands of files)
>>> takes ~45mn.
>
> [...]
>
>> What is the OS thy install on?  How do they generate the gmsh package
>> exactly?  Perhaps I still have a RPM-based distro VM to try it with.
>
> They’re targeting CentOS 7 (!).  It’s built with:
>
>   guix pack -f rpm -S /opt/bin=bin -R gmsh

I guess it has to do with that very dated version of rpm, because
testing from a Fedora 37 VM I had at hand, it's fast (42 seconds on my
17 years old Core 2 Duo desktop):

--8<---------------cut here---------------start------------->8---
$ cat /etc/os-release 
NAME="Fedora Linux"
VERSION="37 (Workstation Edition)"
ID=fedora
VERSION_ID=37
VERSION_CODENAME=""
PLATFORM_ID="platform:f37"
PRETTY_NAME="Fedora Linux 37 (Workstation Edition)"
ANSI_COLOR="0;38;2;60;110;180"
LOGO=fedora-logo-icon
CPE_NAME="cpe:/o:fedoraproject:fedora:37"
DEFAULT_HOSTNAME="fedora"
HOME_URL="https://fedoraproject.org/"
DOCUMENTATION_URL="https://docs.fedoraproject.org/en-US/fedora/f37/system-administrators-guide/"
SUPPORT_URL="https://ask.fedoraproject.org/"
BUG_REPORT_URL="https://bugzilla.redhat.com/"
REDHAT_BUGZILLA_PRODUCT="Fedora"
REDHAT_BUGZILLA_PRODUCT_VERSION=37
REDHAT_SUPPORT_PRODUCT="Fedora"
REDHAT_SUPPORT_PRODUCT_VERSION=37
VARIANT="Workstation Edition"
VARIANT_ID=workstation

[user@fedora Downloads]$ sudo time rpm -i 7m01b0308z5y2pmyn8ywzdj914dxawsl-gmsh-rpm-pack.rpm
17.26user 10.19system 0:42.31elapsed 64%CPU (0avgtext+0avgdata 24468maxresident)k
1481136inputs+2177344outputs (19major+6242minor)pagefaults 0swaps

[user@fedora Downloads]$ rpm --version
RPM version 4.18.0
--8<---------------cut here---------------end--------------->8---

Perhaps using guix time-machine to a commit where we had a RHEL 7 era
rpm version (4.11 according to [0]) would be faster than installing
Centos 7 in a VM... :-)  except, hm, no, that's way too old.  The oldest
we've got is:

e3e1ecf67c0 (Ludovic Courtès              2015-10-26 290)     (version
"4.12.0")

from 2015...

I don't think I'll be looking at fixing this use case; hopefully they
can retire their CentOS 7 soon (EOL: June 30th 2024) and use something
newer.

I'm tempted to close this as 'wontfix'.  What do you think?

[0]  https://rpmfind.net/linux/rpm2html/search.php?query=rpm&submit=Search+...

-- 
Thanks,
Maxim

bug#66647: Installation of RPMs produced by ‘guix pack’ is super slow

[Simon Tournier]

[-- Attachment #1: Type: text/plain, Size: 973 bytes --]

Hi,

On Sun, 22 Oct 2023 at 21:35, Maxim Cournoyer <maxim.cournoyer@gmail.com> wrote:

> --8<---------------cut here---------------start------------->8---

[...]

> [user@fedora Downloads]$ sudo time rpm -i 7m01b0308z5y2pmyn8ywzdj914dxawsl-gmsh-rpm-pack.rpm
> 17.26user 10.19system 0:42.31elapsed 64%CPU (0avgtext+0avgdata 24468maxresident)k
> 1481136inputs+2177344outputs (19major+6242minor)pagefaults 0swaps
>
> [user@fedora Downloads]$ rpm --version
> RPM version 4.18.0
> --8<---------------cut here---------------end--------------->8---

These days I am doing some experiments with CentOS7 (rpm 4.11.8), so I
have a virtualbox VM around.  Attached the view I get using the pack
generated with,

    guix pack -f rpm -S /opt/bin=bin -R gmsh

then copied with ’scp’.  Well, I do not have GUI but gmsh seems working.

For me, ’rpm -i’ needs less than 10 seconds.  It installs 88 items if I
read correctly.

Hope that helps,
simon



[-- Attachment #2: vm.png --]
[-- Type: image/png, Size: 10682 bytes --]

bug#66647: Installation of RPMs produced by ‘guix pack’ is super slow

[Maxim Cournoyer]

tags 66647 + moreinfo unreproducible
thanks

Hi,

Simon Tournier <zimon.toutoune@gmail.com> writes:

> Hi,
>
> On Sun, 22 Oct 2023 at 21:35, Maxim Cournoyer <maxim.cournoyer@gmail.com> wrote:
>
>> --8<---------------cut here---------------start------------->8---
>
> [...]
>
>> [user@fedora Downloads]$ sudo time rpm -i 7m01b0308z5y2pmyn8ywzdj914dxawsl-gmsh-rpm-pack.rpm
>> 17.26user 10.19system 0:42.31elapsed 64%CPU (0avgtext+0avgdata 24468maxresident)k
>> 1481136inputs+2177344outputs (19major+6242minor)pagefaults 0swaps
>>
>> [user@fedora Downloads]$ rpm --version
>> RPM version 4.18.0
>> --8<---------------cut here---------------end--------------->8---
>
> These days I am doing some experiments with CentOS7 (rpm 4.11.8), so I
> have a virtualbox VM around.  Attached the view I get using the pack
> generated with,
>
>     guix pack -f rpm -S /opt/bin=bin -R gmsh
>
> then copied with ’scp’.  Well, I do not have GUI but gmsh seems working.
>
> For me, ’rpm -i’ needs less than 10 seconds.  It installs 88 items if I
> read correctly.

Yeah, a ~600 MiB closure is by no means specially large, should it
shouldn't take 45 minutes unless the IO in the VM is pathologically
slow, or if the old RPM version was struggling with something in our
custom-generated RPMs.  You example shows this is not the case, so I'll
close this with 'moreinfo' and 'unreproducible' tags.

Ludo: feel free to reopen if you can gather more details that would
point at our RPMs being faulty.

-- 
Thanks,
Maxim

bug#66647: Installation of RPMs produced by ‘guix pack’ is super slow

[Loïc SIRVIN]

[-- Attachment #1.1: Type: text/plain, Size: 1631 bytes --]

Hello

Sorry for the late feedback

Installing the rpm with `rpm -i file.rpm` command does not take lot of time.

It takes very long time when we install it with `yum install file.rpm` 
command.

Some system informations :

# cat /etc/centos-release
CentOS Linux release 7.9.2009 (Core)
# yum --version
3.4.3
   Installés : rpm-4.11.3-48.el7_9.x86_64 à 2023-06-02 11:45
   Compilé   : CentOS BuildSystem<http://bugs.centos.org>  à 2021-11-24 16:33
   Commité   : Michal Domonkos<mdomonko@redhat.com>  à 2021-11-01

   Installés : subscription-manager-1.24.53-1.el7.centos.x86_64 à 2023-10-30 10:29
   Compilé   : CentOS BuildSystem<http://bugs.centos.org>  à 2023-10-16 13:21
   Commité   : Pino Toscano<ptoscano@redhat.com>  à 2023-09-22

   Installés : yum-3.4.3-168.el7.centos.noarch à 2023-06-02 11:45
   Compilé   : CentOS BuildSystem<http://bugs.centos.org>  à 2020-10-01 17:03
   Commité   : CentOS Sources<bugs@centos.org>  à 2020-09-29

   Installés : yum-plugin-fastestmirror-1.1.31-54.el7_8.noarch à 2023-06-02 11:45
   Compilé   : CentOS BuildSystem<http://bugs.centos.org>  à 2020-05-12 16:27
   Commité   : Michal Domonkos<mdomonko@redhat.com>  à 2020-03-12

   Installés : yum-rhn-plugin-2.7.7-1.el7.noarch à 2023-06-02 11:46
   Compilé   : Koji à 2017-08-01 10:56
   Commité   : Eric Herget<eherget@redhat.com>  à 2017-07-31
# rpm --version
RPM version 4.11.3

Regards

-- 
Loïc SIRVIN
DSI-SP-CENTRE
Tél. : +33 (0)5 24 57 40 90 (64090)

Centre Inria
de l’université de Bordeaux

200, Av de la vielle Tour
33 405 Talence CEDEX

www.inria.fr

[-- Attachment #1.2: Type: text/html, Size: 2728 bytes --]

[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 4772 bytes --]

bug#66647: Installation of RPMs produced by ‘guix pack’ is super slow

[Simon Tournier]

Hi,

On Mon, 13 Nov 2023 at 13:31, Loïc SIRVIN <loic.sirvin@inria.fr> wrote:

> It takes very long time when we install it with `yum install file.rpm` command.

Using CentOS 7 in VirtualBox, I confirm that the same pack as [1] takes
a looong time with “yum install” – hum, after 10 minutes I decided to
send this email. :-)  Well, indeed “yum install” takes many many more
time than “rpm -i”.


1: bug#66647: Installation of RPMs produced by ‘guix pack’ is super slow
Simon Tournier <zimon.toutoune@gmail.com>
Mon, 23 Oct 2023 14:16:28 +0200
id:87wmvdjzz7.fsf@gmail.com
https://issues.guix.gnu.org/66647
https://issues.guix.gnu.org/msgid/87wmvdjzz7.fsf@gmail.com
https://yhetil.org/guix/87wmvdjzz7.fsf@gmail.com

Cheers,
simon

bug#66647: Installation of RPMs produced by ‘guix pack’ is super slow

[Ludovic Courtès]

Hi,

Simon Tournier <zimon.toutoune@gmail.com> skribis:

> On Mon, 13 Nov 2023 at 13:31, Loïc SIRVIN <loic.sirvin@inria.fr> wrote:
>
>> It takes very long time when we install it with `yum install file.rpm` command.
>
> Using CentOS 7 in VirtualBox, I confirm that the same pack as [1] takes
> a looong time with “yum install” – hum, after 10 minutes I decided to
> send this email. :-)  Well, indeed “yum install” takes many many more
> time than “rpm -i”.

Maxim, should we explicitly recommend ‘rpm -i’ in the manual?

What can ‘yum install’ possibly do that takes so much time? 🤔

Ludo’.

bug#66647: Installation of RPMs produced by ‘guix pack’ is super slow

[Simon Tournier]

Hi,

On Sat, 25 Nov 2023 at 15:22, Ludovic Courtès <ludovic.courtes@inria.fr> wrote:

> Maxim, should we explicitly recommend ‘rpm -i’ in the manual?

I propose something like:

--8<---------------cut here---------------start------------->8---
1 file changed, 8 insertions(+)
doc/guix.texi | 8 ++++++++

modified   doc/guix.texi
@@ -7299,6 +7299,14 @@ Invoking guix pack
 sudo rpm --install --prefix=/opt /gnu/store/...-hello.rpm
 @end example
 
+@quotation Warning
+Rely on @command{rpm --install} for installing an RPM archive and avoid
+@command{yum install} or related.  The generated RPM archive will
+install faster when using @command{rpm} than when using @command{yum}.
+The performances of @command{yum} when installing generated RPM archive
+could be detrimental compared to installing using @command{rpm} tool.
+@end quotation
+
 @quotation Note
 Contrary to Debian packages, conflicting but @emph{identical} files in
 RPM packages can be installed simultaneously, which means multiple
--8<---------------cut here---------------end--------------->8---

WDYT?

Cheers,
simon

bug#66647: Installation of RPMs produced by ‘guix pack’ is super slow

[Maxim Cournoyer]

Hi,

Simon Tournier <zimon.toutoune@gmail.com> writes:

> Hi,
>
> On Sat, 25 Nov 2023 at 15:22, Ludovic Courtès <ludovic.courtes@inria.fr> wrote:
>
>> Maxim, should we explicitly recommend ‘rpm -i’ in the manual?

I'd rather we try it with a few more software such as 'dnf' to narrow it
down to just 'yum', or some other issues in our Guix-generated RPM.

If it'd down to 'yum', since it's being replaced by dnf on all
distributions (?), we could avoid the investigation and document it as
slow, best avoided as drafted by Simon.

-- 
Thanks,
Maxim

bug#66647: Installation of RPMs produced by ‘guix pack’ is super slow

[Simon Tournier]

Hi Inria’s folks, :-)

On Sat, 02 Dec 2023 at 18:13, Maxim Cournoyer <maxim.cournoyer@gmail.com> wrote:

> I'd rather we try it with a few more software such as 'dnf' to narrow it
> down to just 'yum', or some other issues in our Guix-generated RPM.

have you tried with ’dnf’?  Is it similarly slow as ’yum’?

Cheers,
simon

bug#66647: Installation of RPMs produced by ‘guix pack’ is super slow

[Maxim Cournoyer]

Hi,

Simon Tournier <zimon.toutoune@gmail.com> writes:

> Hi Inria’s folks, :-)
>
> On Sat, 02 Dec 2023 at 18:13, Maxim Cournoyer <maxim.cournoyer@gmail.com> wrote:
>
>> I'd rather we try it with a few more software such as 'dnf' to narrow it
>> down to just 'yum', or some other issues in our Guix-generated RPM.
>
> have you tried with ’dnf’?  Is it similarly slow as ’yum’?

I've tried it myself, and it was fast.  yum is an alias that invokes dnf
even on an old obsolete Fedora 37 VM I had available.

We could mention that other package managers than yum should be
preferred in a "@quotation Note", due to a performance problem when
handling modern RPMs as those made by Guix; or we could close this and
wait for yum to have become completely irrelevant (which seems like in a
year or so, last I checked the RHEL end-of-life dates).

Is someone volunteering to add the note?  Or should we close this?

-- 
Thanks,
Maxim

bug#66647: Installation of RPMs produced by ‘guix pack’ is super slow

[Ludovic Courtès]

Hi,

Maxim Cournoyer <maxim.cournoyer@gmail.com> skribis:

> Simon Tournier <zimon.toutoune@gmail.com> writes:
>
>> Hi Inria’s folks, :-)
>>
>> On Sat, 02 Dec 2023 at 18:13, Maxim Cournoyer <maxim.cournoyer@gmail.com> wrote:
>>
>>> I'd rather we try it with a few more software such as 'dnf' to narrow it
>>> down to just 'yum', or some other issues in our Guix-generated RPM.
>>
>> have you tried with ’dnf’?  Is it similarly slow as ’yum’?
>
> I've tried it myself, and it was fast.  yum is an alias that invokes dnf
> even on an old obsolete Fedora 37 VM I had available.
>
> We could mention that other package managers than yum should be
> preferred in a "@quotation Note", due to a performance problem when
> handling modern RPMs as those made by Guix; or we could close this and
> wait for yum to have become completely irrelevant (which seems like in a
> year or so, last I checked the RHEL end-of-life dates).
>
> Is someone volunteering to add the note?  Or should we close this?

Yeah maybe let’s just a short note warning against old versions of ‘yum’
and close this issue.

Thanks for following up!

Ludo’.

bug#66647: Installation of RPMs produced by ‘guix pack’ is super slow

[Simon Tournier]

Hi,

On Mon, 15 Jan 2024 at 10:12, Ludovic Courtès <ludovic.courtes@inria.fr> wrote:

>> We could mention that other package managers than yum should be
>> preferred in a "@quotation Note", due to a performance problem when
>> handling modern RPMs as those made by Guix; or we could close this and
>> wait for yum to have become completely irrelevant (which seems like in a
>> year or so, last I checked the RHEL end-of-life dates).
>>
>> Is someone volunteering to add the note?  Or should we close this?
>
> Yeah maybe let’s just a short note warning against old versions of ‘yum’
> and close this issue.

I am proposing [1]:

--8<---------------cut here---------------start------------->8---
1 file changed, 8 insertions(+)
doc/guix.texi | 8 ++++++++

modified   doc/guix.texi
@@ -7299,6 +7299,14 @@ Invoking guix pack
 sudo rpm --install --prefix=/opt /gnu/store/...-hello.rpm
 @end example

+@quotation Warning
+Rely on @command{rpm --install} for installing an RPM archive and avoid
+@command{yum install} or related.  The generated RPM archive will
+install faster when using @command{rpm} than when using @command{yum}.
+The performances of @command{yum} when installing generated RPM archive
+could be detrimental compared to installing using @command{rpm} tool.
+@end quotation
+
 @quotation Note
 Contrary to Debian packages, conflicting but @emph{identical} files in
 RPM packages can be installed simultaneously, which means multiple
--8<---------------cut here---------------end--------------->8---

WDYT?


Cheers,
simon


1: bug#66647: Installation of RPMs produced by ‘guix pack’ is super slow
Simon Tournier <zimon.toutoune@gmail.com>
Tue, 28 Nov 2023 14:31:20 +0100
id:87a5qyatsn.fsf@gmail.com
https://issues.guix.gnu.org/66647
https://issues.guix.gnu.org/msgid/87a5qyatsn.fsf@gmail.com
https://yhetil.org/guix/87a5qyatsn.fsf@gmail.com

bug#66647: Installation of RPMs produced by ‘guix pack’ is super slow

[Maxim Cournoyer]

Hello,

Simon Tournier <zimon.toutoune@gmail.com> writes:

> Hi,
>
> On Mon, 15 Jan 2024 at 10:12, Ludovic Courtès <ludovic.courtes@inria.fr> wrote:
>
>>> We could mention that other package managers than yum should be
>>> preferred in a "@quotation Note", due to a performance problem when
>>> handling modern RPMs as those made by Guix; or we could close this and
>>> wait for yum to have become completely irrelevant (which seems like in a
>>> year or so, last I checked the RHEL end-of-life dates).
>>>
>>> Is someone volunteering to add the note?  Or should we close this?
>>
>> Yeah maybe let’s just a short note warning against old versions of ‘yum’
>> and close this issue.
>
> I am proposing [1]:
>
> 1 file changed, 8 insertions(+)
> doc/guix.texi | 8 ++++++++
>
> modified   doc/guix.texi
> @@ -7299,6 +7299,14 @@ Invoking guix pack
>  sudo rpm --install --prefix=/opt /gnu/store/...-hello.rpm
>  @end example
>
> +@quotation Warning
> +Rely on @command{rpm --install} for installing an RPM archive and avoid
> +@command{yum install} or related.  The generated RPM archive will
> +install faster when using @command{rpm} than when using @command{yum}.
> +The performances of @command{yum} when installing generated RPM archive
> +could be detrimental compared to installing using @command{rpm} tool.
> +@end quotation
> +

Instead of wording it in a way that make it seems 'rpm' is the only tool
to be preferred, I'd say something like

--8<---------------cut here---------------start------------->8--- Older
versions of the @command{yum} command is known to have performance
problems when installing a Guix-generated RPM package.  Prefer to use
modern alternatives such as the @command{dnf} or the @command{rpm}
commands.
--8<---------------cut here---------------end--------------->8---

Otherwise, LGTM!  Thanks for volunteering to write it!

-- 
Thanks,
Maxim