Multiple versions

Multiple versions

[Dmitry Bogatov]

Hello!

In my attempt to understand Guix, get used to it and use
it's advantages, I got following considerations that I would like
to discuss with more experienced users:

  * Guix provides first-class support for multiple versions of packages.
    By first class I mean, that you don't need to do anything special
    to get this support, unlike Gentoo, which for example, supports
    multiple versions of Python and Ruby, but not Guile or GHC.

    But reading 'gnu/packages/haskell.scm' I see same, single-versioned
    packaging in style of Debian. Why? If we would provide package for
    every version of library 'foo' and every version of 'ghc', Guix
    would replace `haskell-stack' tool, and, eventually became The Ring
    to rule stack,virtualenv,bundler,...

    Or am I missing the point, and libraries are packaged only as long
    they are needed for some program?

  * By default, ~/.guix-profile/share/emacs/site-lisp/guix.d is not in
    load-path. `emacs-no-x` exports no variables. So, if I install
    some emacs library, like `emacs-f`, evaluating (require 'f) in emacs
    fails. It is... unexpected. Is it intended behaviour?

-- 
Accept: text/plain, text/x-diff
Accept-Language: eo,en,ru
X-Keep-In-CC: yes
X-Web-Site: sinsekvu.github.io

Re: Multiple versions

[Pjotr Prins]

Hi Dmitry,

On Sat, Dec 26, 2015 at 11:02:52PM +0000, Dmitry Bogatov wrote:
> Hello!
> 
> In my attempt to understand Guix, get used to it and use
> it's advantages, I got following considerations that I would like
> to discuss with more experienced users:
> 
>   * Guix provides first-class support for multiple versions of packages.
>     By first class I mean, that you don't need to do anything special
>     to get this support, unlike Gentoo, which for example, supports
>     multiple versions of Python and Ruby, but not Guile or GHC.
> 
>     But reading 'gnu/packages/haskell.scm' I see same, single-versioned
>     packaging in style of Debian. Why? If we would provide package for
>     every version of library 'foo' and every version of 'ghc', Guix
>     would replace `haskell-stack' tool, and, eventually became The Ring
>     to rule stack,virtualenv,bundler,...
> 
>     Or am I missing the point, and libraries are packaged only as long
>     they are needed for some program?

You can mix versions. Maybe this helps

  https://github.com/pjotrp/guix-notes/blob/master/REPRODUCIBLE.org

Re: Multiple versions

[Dmitry Bogatov]

[-- Attachment #1: Type: text/plain, Size: 1551 bytes --]

> > In my attempt to understand Guix, get used to it and use
> > it's advantages, I got following considerations that I would like
> > to discuss with more experienced users:
> > 
> >   * Guix provides first-class support for multiple versions of packages.
> >     By first class I mean, that you don't need to do anything special
> >     to get this support, unlike Gentoo, which for example, supports
> >     multiple versions of Python and Ruby, but not Guile or GHC.
> > 
> >     But reading 'gnu/packages/haskell.scm' I see same, single-versioned
> >     packaging in style of Debian. Why? If we would provide package for
> >     every version of library 'foo' and every version of 'ghc', Guix
> >     would replace `haskell-stack' tool, and, eventually became The Ring
> >     to rule stack,virtualenv,bundler,...
> > 
> >     Or am I missing the point, and libraries are packaged only as long
> >     they are needed for some program?
> You can mix versions. Maybe this helps
> 
>   https://github.com/pjotrp/guix-notes/blob/master/REPRODUCIBLE.org

Seems I failed to make myself clear. Let me try again.

Currently, I am at master branch. I want install parallel-20151122.
But it is gone since 0877e. I propose to keep *all* versions,
but just 'parallel' refer to latest.

In case of haskell/python/ruby libraries, I propose keep all versions
multiplied by every version of compiler/interpreter.

-- 
Accept: text/plain, text/x-diff
Accept-Language: eo,en,ru
X-Keep-In-CC: yes
X-Web-Site: sinsekvu.github.io

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: Multiple versions

[Andreas Enge]

On Sun, Dec 27, 2015 at 12:20:27PM +0300, Dmitry Bogatov wrote:
> Currently, I am at master branch. I want install parallel-20151122.
> But it is gone since 0877e. I propose to keep *all* versions,
> but just 'parallel' refer to latest.

This would be a nightmare to maintain. And what do you do about security
updates? If libfoo-1.1.7 fixes a critical security bug, who would backport
this to libfoo-1.0.x and libfoo-1.1.0 to libfoo-1.1.6?

Then there is the combinatorial explosion. If you have 20 libraries in
10 versions each that are needed to build a derived binary, then there
will be 10^20 possible combinations. Which of them would you like to
support?

Our general policy is to keep only the latest version, except for special
cases where people see a point in keeping older versions (script languages,
libraries like qt with two major versions supported in parallel, and so on).

What is your use case? If you want reproducibility, it could make sense
to simply stick to a given git commit. If you just need a particular older
version of some code, you could keep it in your separate tree.

Andreas

Re: Multiple versions

[Dmitry Bogatov]

[-- Attachment #1: Type: text/plain, Size: 2207 bytes --]

* Andreas Enge <andreas@enge.fr> [2015-12-27 10:48:32+0100]
> On Sun, Dec 27, 2015 at 12:20:27PM +0300, Dmitry Bogatov wrote:
> > Currently, I am at master branch. I want install parallel-20151122.
> > But it is gone since 0877e. I propose to keep *all* versions,
> > but just 'parallel' refer to latest.
> 
> This would be a nightmare to maintain. And what do you do about security
> updates? If libfoo-1.1.7 fixes a critical security bug, who would backport
> this to libfoo-1.0.x and libfoo-1.1.0 to libfoo-1.1.6?

Drop old versions, or provide a warning to user in such emergency case.
After all, user have right to believe, that this particular bug is
not relevant to him

> Then there is the combinatorial explosion. If you have 20 libraries in
> 10 versions each that are needed to build a derived binary, then there
> will be 10^20 possible combinations. Which of them would you like to
> support?

Build binaries with latest versions of libraries and compilers.

> Our general policy is to keep only the latest version, except for special
> cases where people see a point in keeping older versions (script languages,
> libraries like qt with two major versions supported in parallel, and so on).

> What is your use case? If you want reproducibility, it could make sense
> to simply stick to a given git commit. If you just need a particular older

My use case is that I want to be able to install every version of
any haskell library and every version of ghc for testing purposes.

For example, I write code, that uses ghc-7.10 with lens-4.13. Will it
work with ghc-7.6(Debian stable) and lens-4.1.2.1? This versions are
important to me, but some other person may have another reference.

Yes, there is combinatorics, but not too terrible. ~1000 packages, 4 ghc
versions -> 4000 <package> variables.

Ah, and I do not propose to actually support, for example,
ghc7.6-lens-4.13. If it fails to build, that it is dropped.
Some kind of automatization and integration with hydra would
be useful.

PS. Let's also discuss Emacs 'load-path'.

-- 
Accept: text/plain, text/x-diff
Accept-Language: eo,en,ru
X-Keep-In-CC: yes
X-Web-Site: sinsekvu.github.io

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: Multiple versions

[Ricardo Wurmus]

Dmitry Bogatov <KAction@gnu.org> writes:

>> Then there is the combinatorial explosion. If you have 20 libraries in
>> 10 versions each that are needed to build a derived binary, then there
>> will be 10^20 possible combinations. Which of them would you like to
>> support?
>
> Build binaries with latest versions of libraries and compilers.
>
>> Our general policy is to keep only the latest version, except for special
>> cases where people see a point in keeping older versions (script languages,
>> libraries like qt with two major versions supported in parallel, and so on).
>
>> What is your use case? If you want reproducibility, it could make sense
>> to simply stick to a given git commit. If you just need a particular older
>
> My use case is that I want to be able to install every version of
> any haskell library and every version of ghc for testing purposes.
>
> For example, I write code, that uses ghc-7.10 with lens-4.13. Will it
> work with ghc-7.6(Debian stable) and lens-4.1.2.1? This versions are
> important to me, but some other person may have another reference.

This doesn’t seem to be a useful way for the Guix project as such to use
our limited resources.

However, Guix is a library and packages are just Scheme values.  You can
write a procedure that recursively replaces the GHC value throughout the
graph of a given package.  Then you can build variants of the package
graph that use a different version of GHC.

> Yes, there is combinatorics, but not too terrible. ~1000 packages, 4 ghc
> versions -> 4000 <package> variables.

This is only for different versions of GHC, but not different versions
of packages such as lens.

> Ah, and I do not propose to actually support, for example,
> ghc7.6-lens-4.13. If it fails to build, that it is dropped.

Then it may be better to do this on a machine dedicated to the task of
testing Haskell package combinations.

> Some kind of automatization and integration with hydra would
> be useful.

Using our main Hydra instance for this seems like wasting resources that
could be used for other purposes more closely related to the goals of
the Guix project.

~~ Ricardo

Re: Multiple versions

[Pjotr Prins]

On Sun, Dec 27, 2015 at 12:20:27PM +0300, Dmitry Bogatov wrote:
> Currently, I am at master branch. I want install parallel-20151122.
> But it is gone since 0877e. I propose to keep *all* versions,
> but just 'parallel' refer to latest.

Use a combination of git hash values (to generate the versions) and
guix profiles (to manage them at the same time).

> In case of haskell/python/ruby libraries, I propose keep all versions
> multiplied by every version of compiler/interpreter.

I do the same with Ruby using profiles. I have any number of interpreters
installed for testing and any number of libraries using either guix or
the lib path with a profile in there.

It can all be done, but it is not the job of guix to to manage beyond
profiles. I do think we should have transparent git HASH tagged
installs, e.g., 

guix package -i ruby --git-hash HASH

which would checkout a git tree and run something like ./pre-inst-env
under the hood.  What we do by hand now can be done automatically. But
that is a wider subject.

With regard to your emacs question, better post it with a separate
subject title. It got lost now.

Pj.

Re: Multiple versions

[Dmitry Bogatov]

[-- Attachment #1: Type: text/plain, Size: 1155 bytes --]

* Pjotr Prins <pjotr.public12@thebird.nl> [2015-12-27 13:41:52+0100]
> On Sun, Dec 27, 2015 at 12:20:27PM +0300, Dmitry Bogatov wrote:
> > Currently, I am at master branch. I want install parallel-20151122.
> > But it is gone since 0877e. I propose to keep *all* versions,
> > but just 'parallel' refer to latest.
> 
> Use a combination of git hash values (to generate the versions) and
> guix profiles (to manage them at the same time).

It makes sence. The only problem is find required hash. Is it any idea
to automate it?

> > In case of haskell/python/ruby libraries, I propose keep all versions
> > multiplied by every version of compiler/interpreter.
> 
> I do the same with Ruby using profiles. I have any number of interpreters
> installed for testing and any number of libraries using either guix or
> the lib path with a profile in there.

But how do you solve problem, that for example you want library foo-999.very.new,
compiled with ruby-1.8, but they never existed at same time in guix
package tree?

-- 
Accept: text/plain, text/x-diff
Accept-Language: eo,en,ru
X-Keep-In-CC: yes
X-Web-Site: sinsekvu.github.io

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: Multiple versions

[Ricardo Wurmus]

Dmitry Bogatov <KAction@gnu.org> writes:

>> I do the same with Ruby using profiles. I have any number of interpreters
>> installed for testing and any number of libraries using either guix or
>> the lib path with a profile in there.
>
> But how do you solve problem, that for example you want library foo-999.very.new,
> compiled with ruby-1.8, but they never existed at same time in guix
> package tree?

Then you can either look up the recipe for ruby-1.8 in the repository
history and copy it, or you create a fresh variant of the “ruby” package
with something like this:

    (define-public my-particular-ruby
      (package (inherit ruby)
        (version "1.8")
        (source (origin ...) ...)))

Here you adjust the version and the source, and bind this variant to a
name “my-particular-ruby”.

You can either put this expression in the “ruby.scm” module (e.g. in a
local branch), or maintain your own package module.  If you choose the
latter you’d have to tell Guix about it by pointing the environment
variable “GUIX_PACKAGE_PATH” at the path.

~~ Ricardo

Re: Multiple versions

[Dmitry Bogatov]

[-- Attachment #1: Type: text/plain, Size: 1275 bytes --]

> >> I do the same with Ruby using profiles. I have any number of interpreters
> >> installed for testing and any number of libraries using either guix or
> >> the lib path with a profile in there.
> >
> > But how do you solve problem, that for example you want library foo-999.very.new,
> > compiled with ruby-1.8, but they never existed at same time in guix
> > package tree?
> 
> Then you can either look up the recipe for ruby-1.8 in the repository
> history and copy it, or you create a fresh variant of the “ruby” package
> with something like this:
> 
>     (define-public my-particular-ruby
>       (package (inherit ruby)
>         (version "1.8")
>         (source (origin ...) ...)))
> 
> Here you adjust the version and the source, and bind this variant to a
> name “my-particular-ruby”.
> 
> You can either put this expression in the “ruby.scm” module (e.g. in a
> local branch), or maintain your own package module.  If you choose the
> latter you’d have to tell Guix about it by pointing the environment
> variable “GUIX_PACKAGE_PATH” at the path.

Looks not so easy, but I got my answers. Thanks.

-- 
Accept: text/plain, text/x-diff
Accept-Language: eo,en,ru
X-Keep-In-CC: yes
X-Web-Site: sinsekvu.github.io

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: Multiple versions

[Pjotr Prins]

On Sun, Dec 27, 2015 at 05:40:02PM +0300, Dmitry Bogatov wrote:
> It makes sence. The only problem is find required hash. Is it any idea
> to automate it?

Not really.

> > > In case of haskell/python/ruby libraries, I propose keep all versions
> > > multiplied by every version of compiler/interpreter.
> > 
> > I do the same with Ruby using profiles. I have any number of interpreters
> > installed for testing and any number of libraries using either guix or
> > the lib path with a profile in there.
> 
> But how do you solve problem, that for example you want library foo-999.very.new,
> compiled with ruby-1.8, but they never existed at same time in guix
> package tree?

You can use source trees with GUIX_PACKAGE_PATH. Maybe you'll have to
move them to a source file. It is less hard than it looks if you are
committed to the idea.

Pj.

Re: Multiple versions

[Ludovic Courtès]

Dmitry Bogatov <KAction@gnu.org> skribis:

> Currently, I am at master branch. I want install parallel-20151122.
> But it is gone since 0877e. I propose to keep *all* versions,
> but just 'parallel' refer to latest.

First, I think it’s important to distinguish the source (the package
recipes in gnu/packages/*.scm) and the installed packages (stuff in
/gnu/store.)

Users can keep the versions they want in their store and in their
profiles.

For recipes, as Andreas noted, it would be terrible for Guix as a
project to provide and maintain zillions of different versions of each
package (keeping them all would be easy, but we want to provide packages
that actually build :-)).

However, users can write their own modules containing different versions
or variants of the packages.  For instance, you could write:

  (define-module (my-packages)
    #:use-module (gnu packages parallel))

  (define my-parallel
    ;; Inherit from the Parallel recipe that Guix provides, but override
    ;; the version being used.
    (package
      (inherit parallel)
      (version "x.y.z")
      (source (origin
                (method url-fetch)
                (uri (string-append "mirror://gnu/parallel/parallel-"
                                    version ".tar.bz2"))
                (sha256
                 (base32
                  "0phn9dlkqlq3cq468ypxbbn78bsjcin743pyvf8ip4qg6jz662jm"))))))

Then drop that in ‘GUIX_PACKAGE_PATH’, and run:

  guix package -i parallel-x.y.z

HTH,
Ludo’.

Re: Multiple versions

[Alex Kost]

Dmitry Bogatov (2015-12-27 02:02 +0300) wrote:

>   * By default, ~/.guix-profile/share/emacs/site-lisp/guix.d is not in
>     load-path. `emacs-no-x` exports no variables. So, if I install
>     some emacs library, like `emacs-f`, evaluating (require 'f) in emacs
>     fails. It is... unexpected. Is it intended behaviour?

Yes, Federico Beffa (the author of emacs-build-system) explained it here:
<http://lists.gnu.org/archive/html/guix-devel/2015-06/msg00398.html>

You either have to add guix.d directory manually or you can configure
emacs interface that comes with Guix, which means (require 'guix-init)
more or less.  See (info "(guix) Emacs Initial Setup") for details.

-- 
Alex

Emacs load path (was: Re: Multiple versions)

[Dmitry Bogatov]

[-- Attachment #1: Type: text/plain, Size: 1126 bytes --]

* Alex Kost <alezost@gmail.com> [2015-12-27 17:11:20+0300]
> Dmitry Bogatov (2015-12-27 02:02 +0300) wrote:
> 
> >   * By default, ~/.guix-profile/share/emacs/site-lisp/guix.d is not in
> >     load-path. `emacs-no-x` exports no variables. So, if I install
> >     some emacs library, like `emacs-f`, evaluating (require 'f) in emacs
> >     fails. It is... unexpected. Is it intended behaviour?
> 
> Yes, Federico Beffa (the author of emacs-build-system) explained it here:
> <http://lists.gnu.org/archive/html/guix-devel/2015-06/msg00398.html>
> 
> You either have to add guix.d directory manually or you can configure
> emacs interface that comes with Guix, which means (require 'guix-init)
> more or less.  See (info "(guix) Emacs Initial Setup") for details.

Read this email. It explain existance of guix.d subdirectory, but why
can't we force emacs to execute following pseudo-code at start:

(for dir in "${HOME}.guix-profile/share/emacs/site-lisp/guix.d"
  (push dir load-path))

-- 
Accept: text/plain, text/x-diff
Accept-Language: eo,en,ru
X-Keep-In-CC: yes
X-Web-Site: sinsekvu.github.io

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: Emacs load path

[Alex Kost]

Dmitry Bogatov (2015-12-27 19:47 +0300) wrote:

> * Alex Kost <alezost@gmail.com> [2015-12-27 17:11:20+0300]
>> Dmitry Bogatov (2015-12-27 02:02 +0300) wrote:
>>
>> >   * By default, ~/.guix-profile/share/emacs/site-lisp/guix.d is not in
>> >     load-path. `emacs-no-x` exports no variables. So, if I install
>> >     some emacs library, like `emacs-f`, evaluating (require 'f) in emacs
>> >     fails. It is... unexpected. Is it intended behaviour?
>>
>> Yes, Federico Beffa (the author of emacs-build-system) explained it here:
>> <http://lists.gnu.org/archive/html/guix-devel/2015-06/msg00398.html>
>>
>> You either have to add guix.d directory manually or you can configure
>> emacs interface that comes with Guix, which means (require 'guix-init)
>> more or less.  See (info "(guix) Emacs Initial Setup") for details.
>
> Read this email. It explain existance of guix.d subdirectory, but why
> can't we force emacs to execute following pseudo-code at start:
>
> (for dir in "${HOME}.guix-profile/share/emacs/site-lisp/guix.d"
>   (push dir load-path))

This is more or less what (require 'guix-init) does (among other
things).  And it works out of the box on GuixSD.

--
Alex