Re: ASDF Builder (Common Lisp) & "package-inferred-system" Packages

[Katherine Cox-Buday <cox.katherine.e@gmail.com>]

Andy Patterson <ajpatter@uwaterloo.ca> writes:

> Hey Katherine,

Hey Andy! I'm so glad you're chiming in. Thanks for providing this build
system!

> On Sat, 12 Jan 2019 14:24:38 -0600 Katherine Cox-Buday
> <cox.katherine.e@gmail.com> wrote:
>
>> I've done some more digging, and I have a better idea of why this is
>> failing.
>> 
>> Running `(asdf:operate 'asdf:compile-bundle-op :foo)` on a
>> `package-inferred-system` produces several `.fasl` files -- one for
>> each sub-package instead of the usual `foo--system.fasl` file. The
>> `cleanup-files` phase then deletes[1] any `.fasl` files which do not
>> have a `--system.fasl` suffix.
>> 
>
> This phase has always been a bit of a hack. There are already some
> other packages where asdf doesn't produce an output file which matches
> the expected name, so we had to change that. I'd be in favour of
> removing the phase wholesale until we can create one which works
> better. Ultimately we should be querying asdf for the output files
> that it produced, and allowing the phase to be switched off with an
> argument.

ASDF also has the ability to search trees, so we may choose to abandon
the concept of a single fasl file altogether and instead create
`/lib/foo/*.fasl` folders. We could then let the fasl files ASDF
produces lay about the system as generated :) I don't yet have a
fully-formed opinion about this.

>> I tried working around this by manually concatenating all the
>> `.fasl` files together using `uiop:combine-fasls` which I believe
>> works. I also had to add a new build phase to take place after the
>> `create-asd-file` phase to do a string replace to change
>> dependencies back to the `foo/bar` style instead of `foo-bar`.
>> 
>
> We've done something similar (the opposite) with the slynk package so
> it's not unheard of. If there's a better way to handle it though, we
> should give it a go.
>
>> I still believe the build system should handle
>> `package-inferred-system` style CL packages properly so that
>> maintainers don't have to do this kind of tweaking for every package
>> (the number of which I believe will steadily increase).
>> 
>
> Definitely agree here. We currently don't have overly many packages
> relying on the build system so now would be a great time to fix it as
> much as possible.

I haven't yet gotten `:ningle` (my test package-inferred-system) to
work. It complains that it can't find `ningle/main`, and I don't know
why as that appears to be in the UIOP bundled fasl.

As we are approaching Guix 1.0, this would also be a good time to shore
this up so that CL users trying Guix for the first time have a pleasant
experience.

>> I'm trying to confirm that this whole things works, but I believe I
>> may be hitting another bug with CL being able to find dependencies.
>> It seems as though even if the dependenices are listed in the
>> generated `.asd` file, and my Guix SBCL can find that `.asd` and
>> attempt to load its system, SBCL cannot find the dependencies.
>> 
>> I'm trying to determine why this might be, but it's currently
>> blocking me from proposing a patch which adds 44 CL packages.
>> 
>
> Hmm, I'd need to see more information here to be able to offer
> anything useful. I think I've ran into similar problems in the past
> but I don't remember what the solutions were.

It was an issue on my end. In my
${HOME}/.config/common-lisp/source-registry.conf.d/asdf.conf file, I
had `(:tree "~/.guix-profile/share/common-lisp/sbcl-source/")` defined
instead of `(:tree "~/.guix-profile/lib/sbcl")`. This allowed ASDF to
find the source definitions, but not the `.asd` files including the
pointers to the packages' dependencies. Changing this completely solved
the dependency issue, although I think we ought to find a different way
to point to dependencies as ASDF may break us at some point.

>> My current theory is that we perform a `setf` on the
>> `*source-registry-parameter*` variable, but the ASDF manual
>> specifically, and strongly, says[2] that this is meant to be a
>> read-only variable. Maybe this is not working as intended?
>> 
>
> It's possible - this was the best way that I found at the time to
> associate a system with a file directly. AFAIK there's no way to ask
> asdf to load a system from a file directly. Any suggestions here are
> welcome.

I will continue to do some digging to try and form some kind of cogent
argument for an approach. To start, I need to get ningle to even work
when packaged!

-- 
Katherine