Sigil
*****

This manual is for Sigil version 0.1.0.

   Sigil is a package manager written for Guile Scheme.

   A guild is an association of independent craftspeople.  This sigil
marks you as a member of the guild of Guile wizards and journeyfolk.

   On a practical level, Sigil lets you share Scheme modules and
programs over the internet, and install code that has been shared by
others.  Sigil can handle dependencies, so when a program requires
several libraries, and each of those has further dependencies, all of
the prerequisites for the program can be installed in one go.

   Sigil is a port of Andreas Rottmann's Dorodango portable package
manager for R6RS Scheme, adapted to better fit in with Guile Scheme
environments.  It is compatible with the R6RS Scheme package archives
that Dorodango uses.

1 Getting Started
*****************

1.1 Installation
================

Sigil comes in a standard GNU tarball, and so the normal package
installation techniques apply:

     tar -xjf sigil-0.1.0.tar.bz2
     cd sigil-0.1.0/sigil
     ./configure
     make
     make check
     # become root
     make install

   Note that this assumes that you have installed Guile into the default
prefix for `./configure', which is usually `/usr/local'.  It is
recommended to install sigil into the same prefix as your Guile
installation, so if you installed Guile to `/usr', add `--prefix=/usr'
after to your `./configure' invocation.

   After running installing Sigil, you should now be able to run `guild
config --help', which displays a help message.

1.2 Quickstart
==============

For the impatient, this section presents the minimum you need to know
to install software using your new guild sigil.

Configuration
-------------

So, you've successfully installed Sigil, and were able to get the help
message via `guild config --help'? Then it's time to tell Sigil where
it can find software (*note Packages::) to install. Create the file
`~/.config/sigil/config.scm' and add this line:

     (repository experimental "http://rotty.yi.org/doro/experimental")

   This tells the guild the location of the Andreas Rottmann's
experimental repository, and gives it the name `experimental'. You
could add further repositories with different names and locations, and
Sigil will consider them all when installing packages.

Updating the package database
-----------------------------

This is all configuration that is needed; if you now run `guild
update', you should see something like the following:

$ guild update
Fetching http://rotty.yi.org/doro/experimental/available.scm
Loading available information for repository `experimental'

   Now you have obtained the information of available packages from the
hall. You can verify that by running `guild list-packages --all', which
should produce output resembling the following:

u conjure                    0-20091204
u dorodango                  0-20091204
u fidfw                      0-20091204
...

   The rightmost columns indicates the package state (`u' means
"uninstalled"), the other columns are the package name and version.

Installing software
-------------------

You can now install any of the listed packages, using `guild install
PACKAGE-NAME':

% guild install spells
The following NEW packages will be installed:
  spells srfi{a}
Do you want to continue? [Y/n]
Fetching http://rotty.yi.org/doro/experimental/srfi_0-20091204.zip
Installing srfi (0-20091204) ...
Fetching http://rotty.yi.org/doro/experimental/spells_0-20091204.zip
Installing spells (0-20091204) ...

   As demonstrated in the above listing, the guild knows that the
package `spells' depends on `srfi', and will automatically install that
package as well.

Other important commands
------------------------

Now you you know how to achieve the primary task of Sigil: installing
software.  There are a few other things you probably want to do at
times:

`guild upgrade'
     Attempts to upgrade each package to the newest available version.

`guild remove'
     Allows you to remove packages from your system.

Getting help
------------

For each command, you can invoke `guild help COMMAND', and it will show
you what options and argument that command requires:

% guild help remove
Usage: guild remove PACKAGE...
Remove installed packages.

Options:
      --no-depends     Ignore dependencies.

  -c, --config=FILE    Use configuration file FILE, instead of the
                       default.
      --no-config      Do not read a configuration file.
      --help           Print this help message.
      --version        Print version information.

2 Overview
**********

Sigil extends the `guild' program, provided by Guile, with a number of
sub-comands that let you download, install, remove and upgrade Scheme
modules and programs.

   The unit of installable software is the "package": a collection of
modules, possibly including programs and documentation, together with
some metadata.

2.1 Anatomy of a Package
========================

A package is the "unit of software" that Sigil works with. It has a
name and a version, which may be used to form dependency relationships
among packages.  It also may have other attributes, like a description
and a homepage URL.  Besides the metadata, a package also contains
files, which are grouped into categories.  Each category of a package
conceptionally contains a set of files, along with their desired
installation locations.  The categories currently handled by Sigil are:

`libraries'
     R6RS libraries, and files required by them (either at runtime or at
     expand-time).

`programs'
     R6RS programs.

`documentation'
     README files, HTML documentation, etc.

2.2 Bundles
===========

Packages are distributed in ZIP files referred to as "bundles"; each
bundle may contain one or more packages.

   If you already are familiar with other package managers, such as
Debian's APT, having more than one package bundled in the same file
might seem unusual to you, but don't worry: bundles are mostly
transparent to the user.  Most of the time, you will deal with
packages, and bundles are of concern mostly when packaging your or
other people's software for the guild.

2.3 Destinations
================

Now the files contained in these categories must be installed
somewhere, and usually into different locations. The rules that
describe where software is installed into are provided by a
_destination_.  For each destination, the guild maintains a database of
installed and available packages.

   Currently, all destinations have the same rules which should be
suitable for POSIXish platforms, and especially for FHS
(http://www.pathname.com/fhs/) platforms:

`libraries'
     Installed into `PREFIX/share/r6rs-libs'.

`programs'
     Installed into `PREFIX/share/libr6rs-PACKAGE-NAME/programs', and a
     shell wrapper in `PREFIX/bin' is created which starts the Scheme
     program via `r6rs-script', which is created automatically when the
     guild initializes a destination.

`documentation'
     Installed into PREFIX`/share/doc/libr6rs-PACKAGE-NAME'.

   For more on how to set the destination on the command line, *note
Common Options::.

2.4 Repositories
================

The bundles in which the packages are installed from are fetched from
repositories. A repository is accessed via HTTP and is essentially a
directory that contains bundles along with a file listing their
locations and the packages within them.

3 Reference
***********

Sigil doesn't actually install any programs.  Instead, it extends the
`guild' program that comes with Guile, adding new commands to it like
`guild install', `guild upgrade', and the like.  These sigil commands
allow you, the guild member, to download, install and inspect packages
containing Scheme code.

   The following subsections document the available commands, grouped by
related tasks.

3.1 Common Options
==================

Many commands take the same options, so we document them here in one
section.  In addition to the following, all commands take `--help' and
`--version' options, although it's usually easier to type `guild help
install' than `guild install --help'.

3.1.1 Config Options
--------------------

Commands that need to access the sigil configuration file accept
options to allow you choose a non-default file, or modify aspects of
the configuration while the command is running.

`--config=FILE'
`-c'
     Use configuration file FILE, instead of the default one. *Note
     Configuration File::, for configuration file syntax.

`--no-config'
     Do not read any configuration file. This option can be used to
     ensure that sigil commands will just use built-in defaults.

`--prefix=PREFIX'
     Use an FHS destination located below the directory PREFIX. This
     option has the same effect as adding an FHS destination to the
     configuration file and using it for this invocation of the sigil
     command.

3.1.2 Database Options
----------------------

Commands that query or modify the package database accept options to
allow you to choose particular destinations or enable certain
repositories.

`--repo=URI'
`-r URI'
     Add URI as an additional repository, during the extent of this
     command.

`--dest=DEST-NAME'
`-d DEST-NAME'
     Choose a specific destination, instead of the default one.  A
     destination specifies where to install software.  Destinations are
     associated with paths in the file system via entries in the
     configuration file.

3.1.3 UI Options
----------------

Some commands might need input from the user.  These options allow a
user to specifiy their choices in advance.

`--non-interactive'
`-n'
     Run non-interactively.  This assumes "yes" on most questions.
     Confirmation prompts for actions that are "risky" (in the sense
     that their effects may be potentially unexpected or unwanted)
     cause program termination with a non-zero exit code, after
     displaying an error message.

`--yes'
`-y'
     Like `--non-interactive', but silently assume "yes" on all
     questions, even risky ones.

3.2 Command Reference
=====================

3.2.1 Querying
--------------

The following commands gather information; either from the package
database, uninstalled bundles, or about the configuration.

 -- Command: list-packages
     Produces a list of packages, along with their installation status
     and version on standard output.

    `--all'
    `-a'
          Show all packages, including uninstalled, but available ones.
          By default only installed packages are listed.

    `--bundle=BUNDLE'
    `-b BUNDLE'
          Temporarily adds BUNDLE's contents to the package database.


     This command also accepts config and database options, as listed in
     the previous section.

 -- Command: show package ...
     Shows information about one or more packages.  This command lists
     package, name, version and dependencies in RFC822-like style on
     standard output.  Each PACKAGE may be either:

        * A package name; in that case, all versions of PACKAGE are
          shown.  It is not error when the package does not exist in
          the database or has no versions, but that package will be
          simply ignored.

        * A package name and version, separated by an equal sign, for
          example `foo=0.1.0'.  Only the specified version will be
          shown in this case.  It is not an error if the package or
          specified version cannot be found, but the package will
          simply be ignored.

    `--bundle=BUNDLE'
    `-b BUNDLE'
          Temporarily adds BUNDLE's contents to the package database.


     This command also accepts config and database options, as listed in
     the previous section.

 -- Command: show-bundle bundle ...
     Shows the contents of one or more bundles on standard output. The
     content listing consist of each package's information, as shown by
     the the `show' command, plus the package's the list of files in
     each category. *Note Packages::.


 -- Command: config
     Shows the current configuration in YAML-like style.

     This command takes config options.

3.2.2 Package managment
-----------------------

In addition to the options listed below, all of these commands take
config, database, and UI options.  *Note Common Options::.

 -- Command: update
     Download information about available packages from all
     repositories of the selected destination.

 -- Command: install package ...
     Install the listed PACKAGEs. Each PACKAGE argument can be a
     package name, in which case the newest available version is
     installed. If the package in question is already installed, it
     will be upgraded. One may also explicitly specify a specific
     version to be installed using the syntax `PACKAGE-NAME=VERSION'.

    `--bundle=BUNDLE'
    `-b BUNDLE'
          Temporarily adds BUNDLE's contents to the package database.

    `--no-depends'
          Disable dependency resolution. This option allows for
          installing packages with unresolved dependencies.


 -- Command: remove package ...
     Remove the listed PACKAGEs from the system.

    `--no-depends'
          Disable dependency resolution.  This option allows for
          removing packages that still have others depending on them.


 -- Command: upgrade
     Upgrade all packages to the newest available version.

3.2.3 Development
-----------------

The following commands are of use if you want to create your own
packages and repositories.

 -- Command: create-bundle directory ...
     Create a bundle from the directories given as arguments.

    `--output=FILENAME'
    `-o FILENAME'
          Output the bundle to FILENAME. When this option is not given,
          the guild will try to name the bundle based on the package
          contained in it. Should the bundle contain multiple packages,
          this option is mandatory.

    `--directory=DIRECTORY'
    `-d DIRECTORY'
          Output directory for the created bundle file. This option
          only has an effect when `--output' is _not_ provided.

    `--append-version=VERSION'
          Rewrite the versions of all packages in the created bundle by
          appending VERSION. This is useful, for e.g. creating
          "snapshot" bundles from a VCS, where one could append the
          current date to the upstream version.

 -- Command: scan-bundles directory ...
     Search the directories passed as arguments for bundles and produce
     an "available file" containing information about found bundles on
     standard output.

    `--output=FILENAME'
    `-o FILENAME'

 -- Command: symlink-bundle bundle-directory target-directory
     Create a symbolic link tree in TARGET-DIRECTORY, using the bundle
     at BUNDLE-DIRECTORY.

    `--force'
          Allow the command to operate even when TARGET-DIRECTORY
          already exists.

    `--deep'
          Create a symbolic link for every file. Without this option,
          `guild hall' will create symbolic links to directories when
          this doesn't change the created symlink tree.

    `--include=PACKAGES'
          Create symbolic links just for the packages listed in the
          comma- or space-seperated list PACKAGES.

    `--exclude=PACKAGES'
          Create symbolic links for all _but_ the packages listed in the
          comma- or space-seperated list PACKAGES.


3.3 Configuration File
======================

The configuration file stores permanent settings for the guild, and can
be selected with the `--config' option, *note Common Options::. Its
syntax is S-expression-based clauses. In the following, we will dissect
an example configuration file; note however, that for most users, a
much simpler configuration will suffice (*note Quickstart::). Also the
`--prefix' global option can be used to work with multiple destinations
without explicitly setting them up in the configuration file. Anyway,
without further ado, here's a configuration that uses all possible
clauses:

     (repository experimental "http://rotty.yi.org/doro/experimental")
     (repository unstable "http://rotty.yi.org/doro/unstable")
     (destination unstable
       (fhs "/home/alice/scheme")
       (repositories unstable))
     (destination experimental
       (fhs "/home/alice/scheme-experiments")
       (database "/home/alice/scheme-experiments/db"))
     (default-destination experimental)

3.3.1 Repositories
------------------

A `repository' clause defines a repository, which may be located on an
HTTP server or a local file system. The repository is given a name, and
a location is specified as an URI:

     (repository <NAME> <LOCATION-URI>)

   In the running example, <NAME> is `experimental', and <LOCATION-URI>
is the string `"http://rotty.yi.org/doro/experimental"', denoting an
HTTP repository at the apparent location.

3.3.2 Destinations
------------------

Destinations are where a package's files are installed to; they have an
associated package database that keeps track of installed packages. In
principle, destinations come in "flavors", but at the time of writing,
there's only a single flavor: `fhs', which puts the files in
subdirectories of the specified prefix directory that are (at least
roughly) in line with the FHS (http://www.pathname.com/fhs/), which
specifies the directory layout of UNIX-like systems.  This means one
can use an `fhs' destination to install to `/usr/local', and have files
ending up in familiar locations.

   In the configuration file, destinations are given a name so they can
be referred to by the `--dest' option.  *Note Common Options::.

   Unless specified otherwise via the `repositories' sub-clause, all
repositories listed up to the point of the destination's declaration
will be used with this destination. A repository must be declared before
being referenced in a destination's `repositories' clause.

   The `database' sub-clause allows to define the location of the
package database on disk; if it is left out, the guild will use a
default location, based on the destination's prefix.

3.3.3 Defaults
--------------

Using the `default-destination' clause one can specify which configured
destination will be used when none is explicitly specified via the
`--dest' option.  If there is no `default-destination' clause, the
first destination specified is considered the default.

3.3.4 Formal Grammar
--------------------

A complete BNF-style grammar for the configuration file:

<configuration> -> <clause>*
<clause> ->  <repository> | <destination>
   | <default-destination>

<repository> -> (repository <name> <uri>)

<destination> -> (destination <name> <destination-spec> <option>*)
<option> -> (database <directory>)
   | (repositories <name>*)
<destination-spec> -> (fhs <directory>)

<default-destination> -> (default-destination <name>)

<directory> -> <string>
<name> -> <symbol>
<uri> -> <string>

4 Packaging Guide
*****************

"Packaging" is the act of supplying Sigil metadata for a set of
libraries and/or programs.  You should make yourself familiar with the
overall concepts of Sigil (*note Overview::) as prerequiste to this
chapter.

4.1 A simple Example
====================

Let's say you want to package a program named `example', which contains
a few libraries and a program.  These files are present in its source
tree:

`README'
     A short note explaining usage of the software.

`foo.scm'
     Contains the library `(example foo)'.

`bar.scm'
     Contains the library `(example bar)'.

`programs/hello'
     A program that use the above libraries.

   For packaging this software, we add a file `pkg-list.scm' in the
top-level directory, which will contain the metadata in S-expression
syntax, a bit like the R6RS `library' form.  The following metadata can
be supplied:

   * The name of the package.

   * The version of the package.

   * Dependencies on other packages.

   * Rules describing wich files in the source tree belong to which
     categories (*note Packages::), and what filename they should have
     relative to that category.

   * An "installation hook", which is code run after the package is
     already unpacked in its destination, but before it is considered
     installed.  This hook may install additional files (probably ones
     that can be generated automatically).

   Of these, only the name and version are mandatory.  The rest are
optional.

   Hence a minimal `package' form listing package name looks like this:

     (package (example (0))
       (depends (srfi)))

   This already specifies the `srfi' package as the only dependency;
further dependencies may of course be listed.

   Inspecting our work with using the `guild show-bundle' command,
passing it the top-level directory where the `pkg-list.scm' file
resides in:

     % guild show-bundle example
     Package: example
     Version: 0
     Depends: (srfi)
     Category: documentation
      README

   The package is nearly empty, besides the `README' file, which was
added to the `documentation' category automatically.  We need rules
dealing with the libraries and the program, which are missing:

     (package (example (0))
       (depends (srfi))
       (libraries
         (scm -> "example"))
       (programs
         (("programs" "hello") -> "hello")))

   The rule inside the `library' form says "put all files with an `scm'
extension below the `example' directory", while the `programs' rule
tells Sigil to place the file `programs/hello' into the top-level
directory of the `programs' category as file `hello'.

   Now we have everything in its proper place, and we are done with
packaging this small example:

     % guild show-bundle example
     Package: example
     Version: 0
     Depends: (srfi)
     Category: libraries
      example/foo.scm
      example/bar.scm
     Category: programs
      hello
     Category: documentation
      README

   For good measure, we should also add the `synopsis', `description'
and `homepage' properties to the package:

     (package (example (0))
       (depends (srfi))
       (synopsis "'Hello World' example")
       (description "This package contains a program that displays"
                    "a familiar greeting.")
       (homepage "http://www.example.org/hello-world")
       (libraries
         (scm -> "example"))
       (programs
         (("programs" "hello") -> "hello")))

4.2 The Metadata File
=====================

A metadata file must be named `pkg-list.scm' and must be either in the
top-level directory or an immediate sub-directory of a bundle to be
considered by Sigil.  It contains one or more `package' forms, thus
declaring which packages are inside the bundle.

   The `package' form declares the name and version of the package, and
lists its properties, some of which are defined and used by Sigil, but
there also may be additional properties used by other software or
humans.

   This leads to the following BNF-style grammar for the file's
contents:

<pkg-list> -> <pkg-form>+
<pkg-form> -> (package (<name> <version>) <property>*)
<property> -> <synopsis> | <description> | <homepage>
   | <depends> | <category> | <hook> | <user-defined>
<synopsis> -> (synopsis <string>)
<description> -> (description <string>*)
<homepage> -> (homepage <string>)
<depends> -> (depends <pkg-reference>*)
<category> -> (<category-name> <file-rule>*)
<category-name> -> libraries | programs | documentation | man
<hook> -> (installation-hook (<hook-option>*) <hook-body>)
<user-defined> -> (<property-name> <property-value>)

   As should be recognizable from the above grammar, properties are
identified by a keyword, and have specific contents depending on that
keyword.  Currently, the following keywords are defined and used by
Sigil:

`synopsis'
     A short single line describing the package briefly.

`description'
     A longer, possibly multi-line description of the package.

`homepage'
     This should carry an URL on which information about the software
     contained in the package can be found.

`depends'
     Declares the package's dependencies.

`libraries'
`programs'
`documentation'
`man'
     These are used for categorizing files contained in the package, so
     they can be installed into the approriate place in the filesystem,
     depending on the destination.

`installation-hook'
     A package may specify actions to be executed during installation
     using this property.


4.2.1 Package Versions
----------------------

Each package has a version that can be ordered, so that a "newer"
relation can be established.  A version has two representations, one as
a text string (for use in file names and for human consumption in
general) and another one as S-expression, allowing easy embedding into
and manipulation in Scheme.

   The S-expression representation is a sequence of lists of integers,
for example `(1 2) (3)', which is textually rendered as `1.2-3' - each
list of integers is concatenated using a dot as separator, and those
groups are then concatenated with a dash separating them.

4.2.2 The package description
-----------------------------

The properties `synopsis' and `description' are used convey the purpose
of the package to the user.

   The value of the `synposis' property should be single string shortly
describing the package, such that it could be used in a sentence
starting with "PACKAGE is a(n) ...".

   The `description' property contains a longer description of the
package, specified via a sequence of strings.  These sequence is handled
like like this:

   * If an item is starting with a space, it is treated as a verbatim
     line.

   * If an item does _not_ start with a space, this item and all
     subsequent ones that don't start with a space are treated as a
     paragraph and are word-wrapped.

   * Paragraphs are separated by empty strings.

4.2.3 Dependencies
------------------

A package can declare its dependencies on other packages via the
`depends' clause of the `package' form.  A dependency names the package
depended upon and, optionally, constrains the version of that package.
In the following example, `foo' depends on `bar' and `baz'; the
dependency on `bar' is unconstrained, allowing for any version of `bar'
to fulfill the dependency, while only a version of `baz' higher than
1.2-3 will satisfy the second dependency.

     (package (foo (1 1))
       (depends (bar)
                (foo (>= (1 2) (3)))))

   This is the BNF grammar for the package references used in the
`depends' clause:

<pkg-reference> -> (<pkg-name>) | (<pkg-name> <version-constraint>)
<version-constraint> -> <version> | (<comparator> <version>)
   | (not <version-constraint>)
   | (or <version-contraint>*)
   | (and <version-constraint>*)
<comparator> -> <= | >= | < | >
<version> -> <part>+
<part> -> (<integer>+)

4.2.4 Categories
----------------

A package's files are grouped into categories to allow installation into
appropriate places in the filesystem.  Each category contains a
(possibly empty) subset of the files contained in or below the
directory the metadata file resides in.

   A file can belong into at most one category.  It can be mapped to any
location (relative filename) inside that category, regardless of its
physical location relative to the metadata file.  To specify this
mapping, a set of rules may be specified for each category known to
Sigil, using the following grammar (see the IrRegular expression
documentation (http://synthcode.com/scheme/irregex/) for more on SRE
syntax):

<file-rule> -> <source> | <source> -> <destination>
   | (exclude <source>*)
<source> -> <string> | <path-with-tail>
<path-with-tail> -> (<string>* <tail>) | <tail>
<tail> -> * | sls | (: <sre>*)

   For instance, the following rules are used for the `libraries'
category in the packaging of the SRFI collection:

     (libraries ((: "%3a" (* any)) -> "srfi")
                ("private" -> ("srfi" "private")))

   In plain English, this means "put any files and directories starting
with "%3a" below the `srfi' directory, and put the file (or, in this
case, directory) `private' in `srfi/private'.

4.2.5 Hooks
-----------

Hooks are a mechanism to perform actions during package installation.
To that end, the package maintainer writes Scheme code resembling an
R6RS program.  The code must satisfy certain properties, namely that
the last expression of the "program" must evaluate to a single-argument
procedure that is invoked when the package is installed.  The details of
that procedure will be discussed below, but first the grammar for the
hook options and body:

<hook-option> -> (needs-source? . <boolean>)
<hook-body> -> (import <library-reference>+) <definitions> <expressions>

   Hooks are not evaluated and executed by Sigil itself, since Sigil may
be installed in a different destination than that which the package in
question should be installed into.  Evaluating hooks in Sigil's own
context would hence pose the problem that the hook could rely neither
on libraries provided by its package nor its dependencies.  For this
reason, sigil spawns a small helper program.  With this way of
proceeding, an installation hook may `import' libraries that are part
of the package or its dependencies.

   However, due to the way the communication of Sigil with helper
programs running the hook is implemented currently, a hook must not use
the standard output or input ports, as these are used for communication
(the standard error port can be used, however).  This restriction will
be hopefully lifted in a later version of Sigil.

4.2.5.1 Hook options
....................

The only supported option supported currently is `needs-source?', which
specifies whether the hook requires the unpackaged source of the
package for its operation.  The `needs-source?' option defaults to `#f'.

4.2.5.2 The hook agent
......................

As mentioned before, the single-argument procedure that is the result of
the last expression of the hook body is executed by the package manager.
Its single argument is an "hook agent" procedure, which can be invoked
to trigger actions of the package manager on behalf of the hook.  The
first argument to this procedure is the name of the desired action,
further arguments depend on the action specified.  Currently, the
following actions are available:

 -- Action: install-file category dest-filename src-filename
     Install file SRC-FILENAME, which is a string referring to an
     existing file.

 -- Action: package-name
     Returns the name of the package being installed.

 -- Action: unpacked-source
     Returns the path of the contents of the extracted package as a
     string, or `#f', if the `needs-source?' option was not specified or
     `#f'.

Index
*****