question re. installing software & init system

question re. installing software & init system

[Miles Fidelman]

Hi Folks,

One thing that is very murky in the documentation is how to install 
software that isn't packaged.  I can't seem to find any clear 
documentation on the process.

Several specific questions:

The recent PPT presentation on GUIX (to Inria) implies that packaging 
can be as simple as writing a config file that tells GUIX to do a 
"config; make; make install" on a source file built with the GNU tools
- do I NEED to package something or can I simply do a basic download, 
untar, config, make, make install on a running system?
- does dmd handle standard sysvinit files (as are still most commonly 
included in source packages)?
- and the there's the whole set of issues, recently raised, related to 
language systems that maintain their own repos and build systems (e.g., 
CPAN) -- somehow, partial import of dependencies into the GUIX 
environment does not seem usable

And a general comment on the documentation:  Given that this is a 
(sort-of) new distro, that does things VERY differently from previous 
distros - it sure would be helpful to have the install documentation 
provide both a very clear overview of the conceptual 
approach/architecture (vs. items spread around various ppts), AND very 
clear step-by-step instructions for:
- basic install & configuration
- installing & configuring packages (both those that run as services and 
those that don't; including later re-configuration)
- installing & configuring software that isn't packaged
- with particular attention to how the installer, package system, and 
init system work together
- and with attention to how these all work with other build systems
You know - something like the Debian install instructions or the FreeBSD 
handbook.

Granted that documentation generally follows code; when doing things 
radically differently, there's a lot to be said for writing 
documentation FIRST - doing so provides a really good check on 
conceptual clarity and usability.  (It kind of makes it hard to do any 
kind of testing, evaluating, or contributing without a good starting point.)

Thanks,

Miles Fidelman

-- 
In theory, there is no difference between theory and practice.
In practice, there is.  .... Yogi Berra

Re: question re. installing software & init system

[Thompson, David]

Hello Miles,

On Thu, Dec 31, 2015 at 12:26 PM, Miles Fidelman
<mfidelman@meetinghouse.net> wrote:
> Hi Folks,
>
> One thing that is very murky in the documentation is how to install software
> that isn't packaged.  I can't seem to find any clear documentation on the
> process.

Packaging instructions can be found here:

https://gnu.org/software/guix/manual/html_node/Defining-Packages.html#Defining-Packages

> Several specific questions:
>
> The recent PPT presentation on GUIX (to Inria) implies that packaging can be
> as simple as writing a config file that tells GUIX to do a "config; make;
> make install" on a source file built with the GNU tools
> - do I NEED to package something or can I simply do a basic download, untar,
> config, make, make install on a running system?

No, you don't have to make a package to use software.  GuixSD is,
after all, a GNU/Linux system where you can build and run custom
software without ever telling the package manager about it.  Do note
that we have no global /usr directory, so you'd want to choose where
'make install' will install things by running './configure
--prefix=/whatever' when building software that uses the GNU build
system.  Ideally, you'd want everything to be managed by Guix so that
you could benefit from all the nice features like transactional
upgrades and rollbacks, but for quick one-off hacks you can go rogue
just as you could on Debian or Trisquel or any other distro.

> - does dmd handle standard sysvinit files (as are still most commonly
> included in source packages)?

No, dmd services are Scheme expressions, rather than ad-hoc Bash
scripts.  For example, here's a custom dmd service I wrote awhile back
to manage my music player daemon:

    (make <service>
       #:provides '(mpd)
       #:requires '()
       #:start (lambda args
                 (define (scope file)
                   (string-append (getenv "HOME") "/.config/mpd/" file))

                 (unless (file-exists? (scope "playlists"))
                   (mkdir (scope "playlists")))
                 (touch-file (scope "database"))
                 (fork+exec-command
                  (list "mpd" "--no-daemon"(scope "mpd.conf"))))
       #:stop (make-kill-destructor))

The dmd manual can be found here:
http://www.gnu.org/software/dmd/manual/html_node/index.html

> - and the there's the whole set of issues, recently raised, related to
> language systems that maintain their own repos and build systems (e.g.,
> CPAN) -- somehow, partial import of dependencies into the GUIX environment
> does not seem usable

Like other distros, we work on providing a means to package any type
of software with Guix.  We have defined build systems to build
software in a variety of languages like Python, Ruby, Perl, R, and
Emacs Lisp.  Additionally, we have a tool called 'guix import' to help
automate some of the package recipe writing process by using the data
available on PyPI, RubyGems, CPAN, CRAN, ELPA, etc.

Why do you think that using Guix for languages beyond C isn't usable?

> And a general comment on the documentation:  Given that this is a (sort-of)
> new distro, that does things VERY differently from previous distros - it
> sure would be helpful to have the install documentation provide both a very
> clear overview of the conceptual approach/architecture (vs. items spread
> around various ppts)

We have a section dedicated to the features:
https://gnu.org/software/guix/manual/html_node/Features.html#Features

Are there any specific things that you would change about this section?

> , AND very clear step-by-step instructions for:
> - basic install & configuration

https://gnu.org/software/guix/manual/html_node/Binary-Installation.html#Binary-Installation

https://gnu.org/software/guix/manual/html_node/System-Installation.html#System-Installation

> - installing & configuring packages (both those that run as services and
> those that don't; including later re-configuration)

https://gnu.org/software/guix/manual/html_node/Invoking-guix-package.html#Invoking-guix-package

https://gnu.org/software/guix/manual/html_node/System-Configuration.html#System-Configuration

> - installing & configuring software that isn't packaged

Linked earlier in the message.

> - with particular attention to how the installer, package system, and init
> system work together
> - and with attention to how these all work with other build systems
> You know - something like the Debian install instructions or the FreeBSD
> handbook.
>
> Granted that documentation generally follows code; when doing things
> radically differently, there's a lot to be said for writing documentation
> FIRST - doing so provides a really good check on conceptual clarity and
> usability.  (It kind of makes it hard to do any kind of testing, evaluating,
> or contributing without a good starting point.)

In addition to all of the above-mentioned documentation sections, we
also have a contributing section:
https://gnu.org/software/guix/manual/html_node/Contributing.html#Contributing

Our documentation, like our code, is a work-in-progress.  We've been
building the manual alongside the code the entire time.  There is
certainly room for improvement, and we've been actively trying to
improve it, and we would *gladly* accept patches that clarify things
that tend to confuse new users.  That said, I think our documentation
is pretty decent and not so poor as you are describing.  We see more
and more contributors with each release, and the docs we do have
helped them get started.

Thanks for the feedback,

- Dave

Re: question re. installing software & init system

[Miles Fidelman]

Thanks!  Some follow-ups inline, below....

On 12/31/15 5:41 PM, Thompson, David wrote:
> Hello Miles,
>
> On Thu, Dec 31, 2015 at 12:26 PM, Miles Fidelman
> <mfidelman@meetinghouse.net> wrote:
>> Hi Folks,
>>
>> One thing that is very murky in the documentation is how to install software
>> that isn't packaged.  I can't seem to find any clear documentation on the
>> process.
> Packaging instructions can be found here:
>
> https://gnu.org/software/guix/manual/html_node/Defining-Packages.html#Defining-Packages
>
>> Several specific questions:
>>
>> The recent PPT presentation on GUIX (to Inria) implies that packaging can be
>> as simple as writing a config file that tells GUIX to do a "config; make;
>> make install" on a source file built with the GNU tools
>> - do I NEED to package something or can I simply do a basic download, untar,
>> config, make, make install on a running system?
> No, you don't have to make a package to use software.  GuixSD is,
> after all, a GNU/Linux system where you can build and run custom
> software without ever telling the package manager about it.  Do note
> that we have no global /usr directory, so you'd want to choose where
> 'make install' will install things by running './configure
> --prefix=/whatever' when building software that uses the GNU build
> system.  Ideally, you'd want everything to be managed by Guix so that
> you could benefit from all the nice features like transactional
> upgrades and rollbacks, but for quick one-off hacks you can go rogue
> just as you could on Debian or Trisquel or any other distro.

Does GuixSD respect the LSB, and in particular /usr/local/.... ?

>
>> - does dmd handle standard sysvinit files (as are still most commonly
>> included in source packages)?
> No, dmd services are Scheme expressions, rather than ad-hoc Bash
> scripts.  For example, here's a custom dmd service I wrote awhile back
> to manage my music player daemon:
>
>      (make <service>
>         #:provides '(mpd)
>         #:requires '()
>         #:start (lambda args
>                   (define (scope file)
>                     (string-append (getenv "HOME") "/.config/mpd/" file))
>
>                   (unless (file-exists? (scope "playlists"))
>                     (mkdir (scope "playlists")))
>                   (touch-file (scope "database"))
>                   (fork+exec-command
>                    (list "mpd" "--no-daemon"(scope "mpd.conf"))))
>         #:stop (make-kill-destructor))
>
> The dmd manual can be found here:
> http://www.gnu.org/software/dmd/manual/html_node/index.html

I was afraid of that.  It's pretty standard for source code to include 
sysvinit files; I doubt anybody includes dmd service files. Kind of 
breaks most standard service installs.

It doesn't look like sysvinit is packaged for GuixSD - it sure would be 
nice to support compatibility.  That would go a long way toward making 
GuixSD a usable distribution.  (One man's opinion.)



>
>> - and the there's the whole set of issues, recently raised, related to
>> language systems that maintain their own repos and build systems (e.g.,
>> CPAN) -- somehow, partial import of dependencies into the GUIX environment
>> does not seem usable
> Like other distros, we work on providing a means to package any type
> of software with Guix.  We have defined build systems to build
> software in a variety of languages like Python, Ruby, Perl, R, and
> Emacs Lisp.  Additionally, we have a tool called 'guix import' to help
> automate some of the package recipe writing process by using the data
> available on PyPI, RubyGems, CPAN, CRAN, ELPA, etc.
>
> Why do you think that using Guix for languages beyond C isn't usable?

Well... speaking just for CPAN - I use several perl-intensive packages 
on our production system.  I almost always find that a cpan-based 
install works.  Trying to install via apt, rarely works. It's a matter 
of keeping stuff up to date.
>
>> And a general comment on the documentation:  Given that this is a (sort-of)
>> new distro, that does things VERY differently from previous distros - it
>> sure would be helpful to have the install documentation provide both a very
>> clear overview of the conceptual approach/architecture (vs. items spread
>> around various ppts)
> We have a section dedicated to the features:
> https://gnu.org/software/guix/manual/html_node/Features.html#Features
>
> Are there any specific things that you would change about this section?

Well, yes.  Including the referenced introduction, there's no 
elucidation of how Guix works, what scripts look like, how to write 
them, etc.  Given that Guix is radically different than traditional 
package managers, a conceptual introduction seems important.
>
>> , AND very clear step-by-step instructions for:
>> - basic install & configuration
> https://gnu.org/software/guix/manual/html_node/Binary-Installation.html#Binary-Installation
>
> https://gnu.org/software/guix/manual/html_node/System-Installation.html#System-Installation

Very short, not a lot of details, particularly when compared to, say the 
Debian install instructions or the BSD handbook.

>
>> - installing & configuring packages (both those that run as services and
>> those that don't; including later re-configuration)
> https://gnu.org/software/guix/manual/html_node/Invoking-guix-package.html#Invoking-guix-package
>
> https://gnu.org/software/guix/manual/html_node/System-Configuration.html#System-Configuration
>
>> - installing & configuring software that isn't packaged
> Linked earlier in the message.
>
>> - with particular attention to how the installer, package system, and init
>> system work together
>> - and with attention to how these all work with other build systems
>> You know - something like the Debian install instructions or the FreeBSD
>> handbook.
>>
>> Granted that documentation generally follows code; when doing things
>> radically differently, there's a lot to be said for writing documentation
>> FIRST - doing so provides a really good check on conceptual clarity and
>> usability.  (It kind of makes it hard to do any kind of testing, evaluating,
>> or contributing without a good starting point.)
> In addition to all of the above-mentioned documentation sections, we
> also have a contributing section:
> https://gnu.org/software/guix/manual/html_node/Contributing.html#Contributing
>
> Our documentation, like our code, is a work-in-progress.  We've been
> building the manual alongside the code the entire time.  There is
> certainly room for improvement, and we've been actively trying to
> improve it, and we would *gladly* accept patches that clarify things
> that tend to confuse new users.  That said, I think our documentation
> is pretty decent and not so poor as you are describing.  We see more
> and more contributors with each release, and the docs we do have
> helped them get started.
>

I'll simply repeat what I said above - when doing things radically 
differently, writing the documentation first helps a lot.  When I need 
to dive into the Debian installer, I can generally find enough detail 
(not necessarily in one place).  (IMHO) Way too much of Guix and GuixSD 
seems to still be in key developers' heads.

Miles Fidelman


-- 
In theory, there is no difference between theory and practice.
In practice, there is.  .... Yogi Berra

Re: question re. installing software & init system

[Thompson, David]

On Fri, Jan 1, 2016 at 12:52 PM, Miles Fidelman
<mfidelman@meetinghouse.net> wrote:
> Thanks!  Some follow-ups inline, below....
>
>
> On 12/31/15 5:41 PM, Thompson, David wrote:
>>
>>> Several specific questions:
>>>
>>> The recent PPT presentation on GUIX (to Inria) implies that packaging can
>>> be
>>> as simple as writing a config file that tells GUIX to do a "config; make;
>>> make install" on a source file built with the GNU tools
>>> - do I NEED to package something or can I simply do a basic download,
>>> untar,
>>> config, make, make install on a running system?
>>
>> No, you don't have to make a package to use software.  GuixSD is,
>> after all, a GNU/Linux system where you can build and run custom
>> software without ever telling the package manager about it.  Do note
>> that we have no global /usr directory, so you'd want to choose where
>> 'make install' will install things by running './configure
>> --prefix=/whatever' when building software that uses the GNU build
>> system.  Ideally, you'd want everything to be managed by Guix so that
>> you could benefit from all the nice features like transactional
>> upgrades and rollbacks, but for quick one-off hacks you can go rogue
>> just as you could on Debian or Trisquel or any other distro.
>
>
> Does GuixSD respect the LSB, and in particular /usr/local/.... ?

No.  We don't use LSB, nor do we conform to the FHS, because they
conflict with functional, stateless system management.  Of course, you
could 'mkdir -p /usr/local' on a GuixSD system and do whatever you'd
like.  In Guix, all software builds are kept in /gnu/store, and each
user on the system maintains one or more "profiles" that reference
some subset of the software available in the store.  This gives each
user the freedom to use the software of their choosing, without
worrying about global conflicts with the system or other users.  Thus,
/usr is obsoleted.

For more information about profiles and other per-user package
management features, see:
https://gnu.org/software/guix/manual/html_node/Invoking-guix-package.html#Invoking-guix-package

>>> - does dmd handle standard sysvinit files (as are still most commonly
>>> included in source packages)?
>>
>> No, dmd services are Scheme expressions, rather than ad-hoc Bash
>> scripts.  For example, here's a custom dmd service I wrote awhile back
>> to manage my music player daemon:
>>
>>      (make <service>
>>         #:provides '(mpd)
>>         #:requires '()
>>         #:start (lambda args
>>                   (define (scope file)
>>                     (string-append (getenv "HOME") "/.config/mpd/" file))
>>
>>                   (unless (file-exists? (scope "playlists"))
>>                     (mkdir (scope "playlists")))
>>                   (touch-file (scope "database"))
>>                   (fork+exec-command
>>                    (list "mpd" "--no-daemon"(scope "mpd.conf"))))
>>         #:stop (make-kill-destructor))
>>
>> The dmd manual can be found here:
>> http://www.gnu.org/software/dmd/manual/html_node/index.html
>
>
> I was afraid of that.  It's pretty standard for source code to include
> sysvinit files; I doubt anybody includes dmd service files. Kind of breaks
> most standard service installs.

GuixSD has a service API built into the system configuration system.
The reason for using an init system written in Guile Scheme is that we
get a much better programming interface (Scheme vs. Bash) and can
share a great deal of code between Guix and the service layer.
Sysvinit, systemd, etc. are just not a good fit for us.

> It doesn't look like sysvinit is packaged for GuixSD - it sure would be nice
> to support compatibility.  That would go a long way toward making GuixSD a
> usable distribution.  (One man's opinion.)

Well, one should be able to easily package sysvinit for Guix, but
GuixSD will never use it as PID 1 because it doesn't meet our needs.
Sysvinit is, IMO, antiquated and undesirable.

>>> - and the there's the whole set of issues, recently raised, related to
>>> language systems that maintain their own repos and build systems (e.g.,
>>> CPAN) -- somehow, partial import of dependencies into the GUIX
>>> environment
>>> does not seem usable
>>
>> Like other distros, we work on providing a means to package any type
>> of software with Guix.  We have defined build systems to build
>> software in a variety of languages like Python, Ruby, Perl, R, and
>> Emacs Lisp.  Additionally, we have a tool called 'guix import' to help
>> automate some of the package recipe writing process by using the data
>> available on PyPI, RubyGems, CPAN, CRAN, ELPA, etc.
>>
>> Why do you think that using Guix for languages beyond C isn't usable?
>
> Well... speaking just for CPAN - I use several perl-intensive packages on
> our production system.  I almost always find that a cpan-based install
> works.  Trying to install via apt, rarely works. It's a matter of keeping
> stuff up to date.

This is why we have the 'guix import' and 'guix refresh' tools, to
make it easier to create the initial packages for these things, and
make updating them later easier.  There is, of course, much work to be
done, but it's very usable right now.  The pains of dealing with
language-specific package managers all day, every day is what lead me
to Guix in the first place.

Also, it is *much* easier to write Guix package recipes than it is to
deal with dpkg/yum/etc.  Typically a package is a very brief Scheme
expression.  Here's an example:

    (define-public perl-scalar-list-utils
      (package
        (name "perl-scalar-list-utils")
        (version "1.41")
        (source
         (origin
           (method url-fetch)
           (uri (string-append "mirror://cpan/authors/id/P/PE/PEVANS/"
                               "Scalar-List-Utils-" version ".tar.gz"))
           (sha256
            (base32
             "04l1q4hps9n8b1hk9kpgpc1cryim7pl9sfdyb7fz5nq4gmz307j7"))))
        (build-system perl-build-system)
        (home-page "http://search.cpan.org/dist/Scalar-List-Utils")
        (synopsis "Common Scalar and List utility subroutines")
        (description "This package contains a selection of subroutines
that people
    have expressed would be nice to have in the perl core, but the
usage would not
    really be high enough to warrant the use of a keyword, and the size so small
    such that being individual extensions would be wasteful.")
        (license (package-license perl))))

You can see more Perl package recipes here:
http://git.savannah.gnu.org/cgit/guix.git/tree/gnu/packages/perl.scm

> I'll simply repeat what I said above - when doing things radically
> differently, writing the documentation first helps a lot.  When I need to
> dive into the Debian installer, I can generally find enough detail (not
> necessarily in one place).  (IMHO) Way too much of Guix and GuixSD seems to
> still be in key developers' heads.

Noted.  We are actively working on improving our documentation and
looking for better ways of explaining Guix's core architecture.

Thanks,

- Dave

Re: question re. installing software & init system

[swedebugia]

On 2016-01-01 19:23, Thompson, David wrote:
> On Fri, Jan 1, 2016 at 12:52 PM, Miles Fidelman
> <mfidelman@meetinghouse.net> wrote:

snip

>> Does GuixSD respect the LSB, and in particular /usr/local/.... ?
> 
> No.  We don't use LSB, nor do we conform to the FHS, because they
> conflict with functional, stateless system management.  Of course, you
> could 'mkdir -p /usr/local' on a GuixSD system and do whatever you'd
> like.  In Guix, all software builds are kept in /gnu/store, and each
> user on the system maintains one or more "profiles" that reference
> some subset of the software available in the store.  This gives each
> user the freedom to use the software of their choosing, without
> worrying about global conflicts with the system or other users.  Thus,
> /usr is obsoleted.

I would like to improve the doc with a section which grafically and 
details explain where GuixSD differs from other systems people are used 
to.

Here is a start:
GuixSD implements symlinks on a wide scale to succeed in being 
functional and transactional.

/bin obsolete (contains only 1 file sh which symlinks to the /gnu/store 
(see below)
/boot contains grub without symlinks
/dev as usual
/etc mostly symlinks to files in /gnu/store or /run/current except for 
group, passwd, shadow and a few others
/gnu only contains the store which is READONLY to all other processes 
than guix-daemon
/home as usual with the differences introduces to make unpriveledged 
package management stored in .guix-profile
/mnt as usual
/proc as usual
/usr does not exist unless you create it
/root as usual and contains the system profile?
/run special to GuixSD - it contains a farm of symlinks according to the 
operating system description installed by the administrator of the 
system (requires root acces to change)
/sys no idea :p
/tmp as usual - guix-daemon uses this extensively when building 
derivation so ensure that you have >10 GB if you are going to build 
large packages as for example libreoffice
/var as usual - guix-daemon uses this to create the temporary 
build-chroots and a bunch of other stuff changed during runtime.

Was this clear enough?

cheers
swedebugia