Maybe a way to get a few more developpers to work on Guix ?

Maybe a way to get a few more developpers to work on Guix ?

[Nicolas Graves via Development of GNU Guix and the GNU System distribution.]

https://www.bpifrance.fr/nos-appels-a-projets-concours/appel-a-projets-communs-numeriques-pour-lintelligence-artificielle-generative

Here's a call for proposal in French which could match a Guix project
with a focus on code generation through LLMs. This could itself help
Guix generate (and fix) package definitions.

-- 
Best regards,
Nicolas Graves

Re: Maybe a way to get a few more developpers to work on Guix ?

[indieterminacy]

On 22-06-2023 15:13, Nicolas Graves via "Development of GNU Guix and the 
GNU System distribution." wrote:
> https://www.bpifrance.fr/nos-appels-a-projets-concours/appel-a-projets-communs-numeriques-pour-lintelligence-artificielle-generative
> 
> Here's a call for proposal in French which could match a Guix project
> with a focus on code generation through LLMs. This could itself help
> Guix generate (and fix) package definitions.

Well if you are wanting to claw at that rabbit-hole:

This is a small EU AI fund for civil society not-for-profit 
organisations (<=30kEUR):
https://europeanaifund.org/newspublications/our-ecosystem-grants-programme-is-now-open/

If only Guix had a not for profit based in Europe that could provide 
such functionality....
I heard about some hot new foundation however - perhaps somebody could 
reach out to them to see if they are interested?

Applications in for the end of July.


-- 
Jonathan McHugh
indieterminacy@libre.brussels

Re: Maybe a way to get a few more developpers to work on Guix ?

[pinoaffe]

Nicolas Graves via "Development of GNU Guix and the GNU System distribution." <guix-devel@gnu.org> writes:
> ... code generation through LLMs. This could itself help Guix generate
> (and fix) package definitions.

I really don't think that that'd be a good match for Guix.  Writing
package definitions is easy, can be assisted with a snippet (such as
provided in /path/to/guix/etc/snippets/yas/scheme-mode/guix-package) and
writing a package definition takes hardly more work than verifying that
an LLM-provided package definition is reasonable.

In other words, I don't think a LLM could make it easier/faster to write
package definitions.

Kind regards,
pinoaffe

Re: Maybe a way to get a few more developpers to work on Guix ?

[Fannys]

pinoaffe <pinoaffe@gmail.com> writes:

> Nicolas Graves via "Development of GNU Guix and the GNU System distribution." <guix-devel@gnu.org> writes:
>> ... code generation through LLMs. This could itself help Guix generate
>> (and fix) package definitions.
>
> I really don't think that that'd be a good match for Guix.  Writing
> package definitions is easy, can be assisted with a snippet (such as
> provided in /path/to/guix/etc/snippets/yas/scheme-mode/guix-package) and
> writing a package definition takes hardly more work than verifying that
> an LLM-provided package definition is reasonable.
>
> In other words, I don't think a LLM could make it easier/faster to write
> package definitions.
>
> Kind regards,
> pinoaffe

Yeah agreed.

It just changes what we need from developers to people that verify that
the code makes sense from the LLM model. So doesn't really seem to solve
much for us.

Regards,
MSavoritias

Re: Maybe a way to get a few more developpers to work on Guix ?

[MSavoritias]

pinoaffe <pinoaffe@gmail.com> writes:

> Nicolas Graves via "Development of GNU Guix and the GNU System distribution." <guix-devel@gnu.org> writes:
>> ... code generation through LLMs. This could itself help Guix generate
>> (and fix) package definitions.
>
> I really don't think that that'd be a good match for Guix.  Writing
> package definitions is easy, can be assisted with a snippet (such as
> provided in /path/to/guix/etc/snippets/yas/scheme-mode/guix-package) and
> writing a package definition takes hardly more work than verifying that
> an LLM-provided package definition is reasonable.
>
> In other words, I don't think a LLM could make it easier/faster to write
> package definitions.
>
> Kind regards,
> pinoaffe

Yeah Agreed.

It seems to just shift the problem from "We need developers" to "We need
people that verify the output of the LLM." So we are basically stuck
with the same problem.

Regards,
MSavoritias

Re: Maybe a way to get a few more developpers to work on Guix ?

[Nicolas Graves via Development of GNU Guix and the GNU System distribution.]

On 2023-06-23 09:02, pinoaffe wrote:

> Nicolas Graves via "Development of GNU Guix and the GNU System distribution." <guix-devel@gnu.org> writes:
>
> In other words, I don't think a LLM could make it easier/faster to write
> package definitions.

I mostly agree, what might be useful however would be the capacity to
propose and re-test a patch series that fails on a very common / easy to
fix error (a forgotten input, a type of error which usually means the
test has to be disabled (network access... )) or that could
improve a patch series (fixing linting warnings, improving commit or
package description). In other words, helping the review process for
guix commits which I think is one of the points where it can be improved
substantially.

Maybe a workflow like "get the patch series to evaluate on
qa.guix.gnu.org" --> "if there are linting warnings or errors, process
the logs and make a useful default answer for the patch sender / or a
new patch series proposition" might help. 

I'm just suggesting because I've seen the opportunity ;)

-- 
Best regards,
Nicolas Graves

Re: Maybe a way to get a few more developpers to work on Guix ?

[Csepp]

Nicolas Graves via "Development of GNU Guix and the GNU System distribution." <guix-devel@gnu.org> writes:

> https://www.bpifrance.fr/nos-appels-a-projets-concours/appel-a-projets-communs-numeriques-pour-lintelligence-artificielle-generative
>
> Here's a call for proposal in French which could match a Guix project
> with a focus on code generation through LLMs. This could itself help
> Guix generate (and fix) package definitions.

Mandatory reading for anyone interested in this:
https://limited.systems/articles/climate-cost-of-ai-revolution/

IMHO LLMs for Guix are so damn not worth the effort.  It will not fix
any of the actual issues with Guix, like the huge performance gap
between it and traditional package managers.
But then again, wasteful software is the hot new trend thanks to AI and
crptocurrencies, so what do I know. :)

Re: Maybe a way to get a few more developpers to work on Guix ?

[Nicolas Graves via Development of GNU Guix and the GNU System distribution.]

On 2023-06-24 13:08, Csepp wrote:

> Nicolas Graves via "Development of GNU Guix and the GNU System distribution." <guix-devel@gnu.org> writes:
>
>> https://www.bpifrance.fr/nos-appels-a-projets-concours/appel-a-projets-communs-numeriques-pour-lintelligence-artificielle-generative
>>
>> Here's a call for proposal in French which could match a Guix project
>> with a focus on code generation through LLMs. This could itself help
>> Guix generate (and fix) package definitions.
>
> Mandatory reading for anyone interested in this:
> https://limited.systems/articles/climate-cost-of-ai-revolution/

I fully agree on that as well. I don't think training a full-fetched LLM
is nowhere worth it, but I've heard that just fine-tuning an open model
with quantized weights might get the job done training and energy costs
down to a few hundred bucks.

> IMHO LLMs for Guix are so damn not worth the effort.  It will not fix
> any of the actual issues with Guix, like the huge performance gap
> between it and traditional package managers.

I've also opened another discussion on the subject on guix-devel
recently. Do you have any benchmark material to back this up?

-- 
Best regards,
Nicolas Graves

Re: Maybe a way to get a few more developpers to work on Guix ?

[Vagrant Cascadian]

[-- Attachment #1: Type: text/plain, Size: 2394 bytes --]

On 2023-06-24, Nicolas Graves via "Development of GNU Guix and the GNU System distribution." wrote:
> On 2023-06-24 13:08, Csepp wrote:
>> Nicolas Graves via "Development of GNU Guix and the GNU System distribution." <guix-devel@gnu.org> writes:
>> IMHO LLMs for Guix are so damn not worth the effort.  It will not fix
>> any of the actual issues with Guix, like the huge performance gap
>> between it and traditional package managers.
>
> I've also opened another discussion on the subject on guix-devel
> recently. Do you have any benchmark material to back this up?

Well, I just ran "apt update" on Debian, and it took approximately 7
seconds, which was mostly spent downloading moderately sized files from
Debian mirrors (~1MB).

A corresponding "guix pull" took 299 seconds, downloading at least 8MB
(from a quick eyeball calculation as guix does not summarize the results
for me), and compiling all the various guix-*.drv that make up guix
pull. The vast majority of the time was spent compiling
derivations. This was also using a local copy of guix.git, so having to
update guix.git over the network would take even longer... (and it did
even spend a fair amount of time copying from the local guix.git on a
fast NVMe device)

Obviously, guix pull is doing a lot more, but it is ... doing a lot
more!

"apt install hello" (~2.3 seconds) and "guix install hello" (~1.5
seconds) were actually in a similar ballpark, which honestly surprised
me. Guix is much faster with "guix remove hello" ... although arguably
"guix remove hello && guix gc --delete $(guix build hello)" would be a
more similar operation, and although I did not time it, it was
reasonably fast, too.

So, presuming substitutes are available, the main slowness with guix
seems to be guix pull?

I did not test "guix system reconfigure" ... but in a way, that may be a
more real comparison than "guix install" ... although we are starting to
get into fundemental differences between apt and guix here.

The other big factor in my mind is that even with a single generation of
guix with nothing for guix gc to do, there may be multiple copies of
some libraries at various versions left around in the store, which is
probably not terribly space efficient... there are usually reasons for
this (e.g. bootstrapping from a tiny seed, yay!) but nevertheless, those
reasons have some cost (and benefits!).


live well,
  vagrant

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 227 bytes --]

Re: Maybe a way to get a few more developpers to work on Guix ?

[Csepp]

Vagrant Cascadian <vagrant@debian.org> writes:

> [[PGP Signed Part:Undecided]]
> On 2023-06-24, Nicolas Graves via "Development of GNU Guix and the GNU System distribution." wrote:
>> On 2023-06-24 13:08, Csepp wrote:
>>> Nicolas Graves via "Development of GNU Guix and the GNU System distribution." <guix-devel@gnu.org> writes:
>>> IMHO LLMs for Guix are so damn not worth the effort.  It will not fix
>>> any of the actual issues with Guix, like the huge performance gap
>>> between it and traditional package managers.
>>
>> I've also opened another discussion on the subject on guix-devel
>> recently. Do you have any benchmark material to back this up?
>
> Well, I just ran "apt update" on Debian, and it took approximately 7
> seconds, which was mostly spent downloading moderately sized files from
> Debian mirrors (~1MB).
>
> A corresponding "guix pull" took 299 seconds, downloading at least 8MB
> (from a quick eyeball calculation as guix does not summarize the results
> for me), and compiling all the various guix-*.drv that make up guix
> pull. The vast majority of the time was spent compiling
> derivations. This was also using a local copy of guix.git, so having to
> update guix.git over the network would take even longer... (and it did
> even spend a fair amount of time copying from the local guix.git on a
> fast NVMe device)
>
> Obviously, guix pull is doing a lot more, but it is ... doing a lot
> more!
>
> "apt install hello" (~2.3 seconds) and "guix install hello" (~1.5
> seconds) were actually in a similar ballpark, which honestly surprised
> me. Guix is much faster with "guix remove hello" ... although arguably
> "guix remove hello && guix gc --delete $(guix build hello)" would be a
> more similar operation, and although I did not time it, it was
> reasonably fast, too.
>
> So, presuming substitutes are available, the main slowness with guix
> seems to be guix pull?

NVMe (or even an SSD) helps a lot.  And I suspect your system also has a
good amount of RAM for IO caching and at least 4 modern 64 bit cores.
Try running guix pull on a 32 bit machine with 1 GB of RAM and an HDD
with LUKS for storage (and a lot of swap), you'll see a waaaay wider
performance gap.
Even simple operations like guix edit take much longer than they should.
Yes, Guix does more, but a lot of what it does could be sped up.

Slow guix pull

[Simon Tournier]

Hi,

On Sat, 24 Jun 2023 at 09:33, Vagrant Cascadian <vagrant@debian.org> wrote:

> So, presuming substitutes are available, the main slowness with guix
> seems to be guix pull?

I agree (and for the record, I would add in addition “guix search” too,
another story :-)).


Well, from my understanding about “guix pull”, the first bottleneck is
about “Computing Guix derivation for ...” from build-aux/build-self.scm.

Maybe I misread something and for sure I am probably missing some
details.  On my desktop machine, that part takes 84 seconds, roughly.
From my understanding, it’s because of this procedure,

--8<---------------cut here---------------start------------->8---
(define (proxy input output)
  "Dump the contents of INPUT to OUTPUT until EOF is reached on INPUT.
Display a spinner when nothing happens."
  (define spin
    (circular-list "-" "\\" "|" "/" "-" "\\" "|" "/"))

  (setvbuf input 'block 16384)
  (let loop ((spin spin))
    (match (select (list input) '() '() 1)
      ((() () ())
       (when (isatty? (current-error-port))
         (display (string-append "\b" (car spin))
                  (current-error-port))
         (force-output (current-error-port)))
       (loop (cdr spin)))
      (((_) () ())
       ;; Read from INPUT as much as can be read without blocking.
       (let ((bv (get-bytevector-some input)))
         (unless (eof-object? bv)
           (put-bytevector output bv)
           (loop spin)))))))
--8<---------------cut here---------------end--------------->8---

that is called by,

        ;; Wait for a connection on SOCK and proxy build output so it can be
        ;; processed according to the settings currently in effect (build
        ;; traces, verbosity level, and so on).
        (match (accept sock)
          ((port . _)
           (close-port sock)
           (delete-file node)
           (proxy port (current-build-output-port))))

And that dump is currently on strong annoyance, IMHO.  Because most of
the other derivations can be substituted but this part cannot, again
from my poor understanding.

Note that this part can be very painful.  For example, on my laptop from
2016, it takes ~180 seconds.  Well, 3 minutes for “dumping content”…
that’s a piece of content! :-)

Well, I do not know exactly which kind of content is living at ’port’
and then if ’(current-build-output-port)’ needs all this content.
Therefore, I do not know if something could be trimmed… but I guess
something could be improved here.


Cheers,
simon

Re: Maybe a way to get a few more developpers to work on Guix ?

[Csepp]

Nicolas Graves <ngraves@ngraves.fr> writes:

> On 2023-06-24 13:08, Csepp wrote:
>
>> Nicolas Graves via "Development of GNU Guix and the GNU System distribution." <guix-devel@gnu.org> writes:
>>
>>> https://www.bpifrance.fr/nos-appels-a-projets-concours/appel-a-projets-communs-numeriques-pour-lintelligence-artificielle-generative
>>>
>>> Here's a call for proposal in French which could match a Guix project
>>> with a focus on code generation through LLMs. This could itself help
>>> Guix generate (and fix) package definitions.
>>
>> Mandatory reading for anyone interested in this:
>> https://limited.systems/articles/climate-cost-of-ai-revolution/
>
> I fully agree on that as well. I don't think training a full-fetched LLM
> is nowhere worth it, but I've heard that just fine-tuning an open model
> with quantized weights might get the job done training and energy costs
> down to a few hundred bucks.

From the "Key points" section right at beginning of the linked article:
"Training of large AI models is not the problem"
"Large-scale use of large AI models would be unsustainable"

It's not the cost in money that I care about, it's the cost in
emissions, which is not included in the few hundred bucks.
(ie.: it's an externality)

However, using other machine learning models that can actually run
efficiently might make sense.
But really, what we need for package availability is more support code
in the form of better importers and updaters, better testing, better
coverage, more accessible workflows - the email workflow could certainly
be improved by *a lot* - better bug tracking, and so on.

There are also very well founded concerns about the quality of the code
that LLMs produce.