Re: guile scheme tutorial

[rendaw <7e9wc56emjakcm@s.rendaw.me>]

On 5/4/19 5:57 PM, Ludovic Courtès wrote:
> Hi rendaw,
>
> rendaw <7e9wc56emjakcm@s.rendaw.me> skribis:
>
>> Oh, this is great!  I've actually been putting together a small Guix
>> guide here:
>> https://gitlab.com/rendaw/blog/blob/master/how_to_guix_for_those_who_dont.md
> Nice!
>
> I skimmed at
> <https://gitlab.com/rendaw/blog/blob/master/a_brief_guix_review.md> and
> I wonder if you could point out areas where documentation is severely
> lacking?  We’d all very much welcome improvements to the manual so your
> insight as a newcomer is valuable.
>
> (Besides I wouldn’t consider mailing lists as a “user hostile” tool, but
> that’s not another story…)
>
> Thanks,
> Ludo’.

Thanks for looking at that!  I appreciate you asking and I'd like to
provide a careful answer.


The documentation currently excels at:

1. Installation (straightforward, theory light steps for getting started)

2. Listing commands and arguments

3. Specific use cases - setting up an application, bootstrapping the
installer, creating graphs


The areas that were important for me but weren't well documented were:

1. Concepts

* What is functional package management?  What is garbage collection? 
How do these work?

What is a package?  How does Guix determine what makes a package
unique?  What's a derivation, and what's the relationship between
derivations and packages?  The best I could find is that a derivation
communicates an action to the store, but that uses vocabulary in ways
that 99% of readers won't be familiar with ("communicate", "action",
isn't the "store" a directory?) and is so abstract that it won't help
people make choices dependent on that.  What's a tangible example of a
derivation?  Where and why would someone use a derivation directly?

The only gc documentation says that anything reachable from a root won't
be deleted, but doesn't explain what a root is, how they're created, how
they're identified, how they're removed, etc.

What's a profile?  The documentation says where they're stored, but not
how they interact with garbage collection.  What's the relationship
between a profile and a system config scm file?

What is a system generation?  How does this relate to the above?  How
does it work, maybe with a tangible example.

* What happens when running guix system?

How is the config file used?  The documentation says that the config scm
has should contain the operating system definition, but that's wrong -
it should return the operating system definition.  The config itself is
essentially a function, and I've seen confusion due to that not being
clearly explained in other threads ("Installing a package using guile
script").

What does guix system do with the definition once it's returned?  How
does install happen on a live system?  What stages are there in the install?

How does this process change for the different install targets? (vm,
disk-image, docker image, etc)

* G-expressions

The analogy to quotes is useful, but how do these actually work? 
They're relocatable - how does one store them to disk?  Read them from
disk an execute them?  Or execute them from memory?  Quotes can be
converted back from data using `eval` - what's the g-expression
equivalent?  Where can they be used (maybe this is more a reference
thing - which functions take g-expression arguments)?

What is lowering, what can be lowered, and what's the relation between
lowering and g-expressions?

How do you run a program in a g-expression?  This seems like one of the
most fundamental things one would do in a g-expression (especially when
interfacing with a world of non-guile software) but AFAICT it's only
addressed indirectly in a blog post.  The blog post says use invoke, and
that invoke should be used instead of system*.  Why?  What does it do
differently?  If invoke is better, why does Guix source use system*?

* Package creation

Again, the documentation says "use these build systems" but not why, or
what they do.  How does a build system transform a bunch of files into
something in /gnu/store?  What are the expectations of a package (where
do binaries go, shared libraries, etc).  Kernel modules?  Plugins for
other software?  Things with a global registry file/directory?

How do dependencies between packages work?  How are packages exposed to
users?  How are packages exposed to services?  How are packages exposed
to other packages?  And I don't mean how as in "what's the canonical way
to access package X" but "all these packages are in different
directories - how does the build process know that this package needs to
access this directory instead of this other directory?"

IMO it would be better to start with the trivial build system, say "we
have these files, here's how we turn it into a package" then show how
the build systems make it much simpler but that they're just an
extension of first principles.

* Services

I've been discussing this in another thread, but it's the same thing -
what a service is is unclear because it hinges on the use of the word
"extend" which isn't defined.

2. Reference

* Specifications

The arguments to functions are defined but what the function/macro
returns is often missing or incompletely defined.  `service` returns a
record of type `<service>` with fields `type` and `value`.  Without
knowing this stuff it's impossible to effectively "program" around it -
ex: filtering services, doing match, introspecting packages, etc. 
`scheme-file` returns an object - what sort of object, how can it be
used, etc.

For arguments with enumerative or switch parameters, how the function
behaves with different values is only partially defined.  Ex:
needed-for-boot? only describes what happens when #t, not when #f.  Same
with #volatile-root? in initrd.

Function specifications should have a much more rigorous format.  Links
to related definitions would be awesome too.

* Current status

There's a lot of "this doesn't work yet" type comments in the code, but
isn't mentioned in the documentation.  Ex: disk-image with uefi
bootloaders.  Users will read about features in the documentation,
assume it works (it's documented and there's no caveats), and then dump
N hours into trying to figure out what they did wrong.

* Comprehensiveness

Many auxiliary methods are not defined.  For example,
`use-package-modules` and `use-service-modules`.  These appear
frequently in the documentation, but it's unclear why you'd use those
instead of `use-modules` and the syntax is subtly different.  `mkdir-p`,
`chmod`, `make-forkexec-constructor` (it's hard to create shepherd
services without this), `call-with-output-file`, ...


I listed a lot of specific points because I think it's the best way to
communicate how I got blocked, but I don't think these should be
explicitly answered in the documentation.  If the documentation starts
with theory/concepts, such as explaining that "invoke runs a program
with system* but raises an exception if the return code is nonzero",
then the explanation of why it should be preferred to system*, how it's
different to system*, and what situations each should be used in is
stuff the reader can easily reach on their own.