streamlining custom package development

streamlining custom package development

[Przemysław Kamiński]

[-- Attachment #1.1.1.1.1: Type: text/plain, Size: 2120 bytes --]

Hello,

I'm a Guix noob so please be patient :)

I currently want to use Guix to do some reproducible package management 
for my code.

So far my workflow is this:
1. I want to build some Haskell code
2. I need custom libs, so I write a simple package file (ghc-stuff.scm) 
like this one:

(define-public ghc-stuff
   (package
    (name "ghc-stuff
    (version "0.1.0")
    (source
     (local-file "../../stuff"
                 #:recursive? #t))
    (build-system haskell-build-system)
    (propagated-inputs
     (list <some-packages>))

3. I run `guix build -L <my-custom-guix-package-dir> -m manifest.scm 
(manifest only contains a reference to ghc-stuff.scm)
4. I get an error which looks like this:
View build log at 
'/var/log/guix/drvs/j2/wa3vw49l78m85z74x91sklqxwhx87y-ghc-hwodr-0.1.0.drv.bz2'

so I fire up

bunzip --stdout 
'/var/log/guix/drvs/j2/wa3vw49l78m85z74x91sklqxwhx87y-ghc-hwodr-0.1.0.drv.bz2'

and see that I am missing some Haskell packages.

5. I use 'guix search ghc-chart' only to find out that 1.9.3 version is 
there while I'm using 1.9.4. So I fire up 'ghc import hackage Chart' to 
show a package definition stub for 1.9.4, I add that to a 
'ghc-chart.scm' file in <my-custom-guix-package-dir>, import that into 
my ghc-stuff.scm file, add to (propagated-inputs).

6. Flush and repeat step 3.

This is tedious. Is there a way to automate this a bit more?

I tried:

cat <stuff>/stuff.cabal | guix import hackage -s -r

but it complains with

failed to download "/run/user/1000/guix-file.0hKo9A" from 
"https://hackage.haskell.org/package/stuff/stuff-0.1.0.0.tar.gz"

then throws a bunch of "Syntax error: unexpected end of input" and then 
spits out the manifest, but only for my <stuff> code so some packages 
are still missing and I need to add them manually.


Also, is there a way to use 'guix repl --listen=37146' in my setup? I 
guess it would be even more streamlined not to quit emacs to test the 
package build, however I'm such a noob that I even don't know how to 
build my package via Guix repl.

Best,
Przemek

[-- Attachment #1.1.1.1.2: OpenPGP public key --]
[-- Type: application/pgp-keys, Size: 1227 bytes --]

[-- Attachment #1.1.2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 236 bytes --]

[-- Attachment #1.2: publickey - cgenie@pm.me - 9cc42b0a.asc --]
[-- Type: application/pgp-keys, Size: 681 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 249 bytes --]

Re: streamlining custom package development

[Marek Paśnikowski]

> This is tedious. Is there a way to automate this a bit more?

Only slightly.  Do remember, that Guix is purely functional.  Your pain is with the chain of dependencies.

I recently went through a similar ordeal while packaging an update to Nix.  I suggest to:
1. look up the dependency list on the program's or library's documentation;
2. add the dependencies to the input lists;
3. attempt to build the target package and write the package definitions for the missing dependecies;
4. keep iterating the above until completion.

It should help you to think of this iteration as a Test Driven Development process.  The guix build  is your test suite, which you want to get green.

I also recommend to use the -v3 option - it will allow you to see the log immediately.

Best Regards,
Marek Pasnikowski

Re: streamlining custom package development

[Simon Tournier]

Hi,

On Fri, 03 Mar 2023 at 10:44, Przemysław Kamiński <cgenie@pm.me> wrote:

> This is tedious. Is there a way to automate this a bit more?

Nothing I am aware of.  Because Guix relies on some LTS, there is no
easy way, to my knowledge, to achieve what you would like.  Somehow you
have to package by yourself and manually your package “ghc-stuff”. :-)


> cat <stuff>/stuff.cabal | guix import hackage -s -r
>
> but it complains with
>
> failed to download "/run/user/1000/guix-file.0hKo9A" from 
> "https://hackage.haskell.org/package/stuff/stuff-0.1.0.0.tar.gz"

That’s expected.  Well, IIUC, you are interested only in the
dependencies and not the package itself.  Maybe it misses an option to
“guix import hackage” for excluding the direct package of the Cabal
file.


> then throws a bunch of "Syntax error: unexpected end of input" and then 
> spits out the manifest, but only for my <stuff> code so some packages 
> are still missing and I need to add them manually.

It is hard to tell more without more specific details. :-)

Maybe could you share some more details?



Cheers,
simon

Re: streamlining custom package development

[Ludovic Courtès]

Hi,

Przemysław Kamiński <cgenie@pm.me> skribis:

> 3. I run `guix build -L <my-custom-guix-package-dir> -m manifest.scm
> (manifest only contains a reference to ghc-stuff.scm)
> 4. I get an error which looks like this:
> View build log at
> '/var/log/guix/drvs/j2/wa3vw49l78m85z74x91sklqxwhx87y-ghc-hwodr-0.1.0.drv.bz2'
>
> so I fire up
>
> bunzip --stdout
> '/var/log/guix/drvs/j2/wa3vw49l78m85z74x91sklqxwhx87y-ghc-hwodr-0.1.0.drv.bz2'
>
> and see that I am missing some Haskell packages.

Note that the build log should be just above the “View build log” line
if you scroll up a little bit.  :-)

Otherwise, like Simon said, I’d recommend getting package definitions by
running ‘guix import hackage -r’, assuming said packages are on Hackage.
That should simplify things.

HTH,
Ludo’.

Re: streamlining custom package development

[Przemysław Kamiński]

[-- Attachment #1.1.1.1.1: Type: text/plain, Size: 3363 bytes --]

As for an example. Clone this repository

https://github.com/sras/servant-examples

Then cd into it and run

cat servant-examples.cabal | guix import hackage -s -r

The '-s' option reads from stdin, the '-r' option imports everything 
recursively.

It throws this output:

Starting download of /tmp/guix-file.TykbjO
 From 
https://hackage.haskell.org/package/servant-examples/servant-examples-0.1.0.0.tar.gz...
download failed 
"https://hackage.haskell.org/package/servant-examples/servant-examples-0.1.0.0.tar.gz" 
404 "Not Found"

Starting download of /tmp/guix-file.TykbjO
 From 
https://web.archive.org/web/20230307175126/https://hackage.haskell.org/package/servant-examples/servant-examples-0.1.0.0.tar.gz...
download failed 
"https://web.archive.org/web/20230307175126/https://hackage.haskell.org/package/servant-examples/servant-examples-0.1.0.0.tar.gz" 
404 "NOT FOUND"
Trying to use Disarchive to assemble /tmp/guix-file.TykbjO...
could not find its Disarchive specification
failed to download "/tmp/guix-file.TykbjO" from 
"https://hackage.haskell.org/package/servant-examples/servant-examples-0.1.0.0.tar.gz"
Syntax error: unexpected end of input
Syntax error: unexpected end of input
(define-public ghc-servant-examples
   (package
     (name "ghc-servant-examples")
     (version "0.1.0.0")
     (source (origin
               (method url-fetch)
               (uri (hackage-uri "servant-examples" version))
               (sha256
                (base32
                 "failed to download tar archive"))))
     (build-system haskell-build-system)
     (properties '((upstream-name . "servant-examples")))
     (inputs (list ghc-servant
                   ghc-servant-server
                   ghc-servant-swagger
                   ghc-swagger2
                   ghc-wai
                   ghc-aeson
                   ghc-warp))
     (home-page "https://github.com/githubuser/servant-examples#readme")
     (synopsis "")
     (description "")
     (license license:bsd-3)))


My guess the "Syntax error" is from the missing "ghc-servant-swagger" 
package.

Also, the source should be changed to (local-file "./" #:recursive? #t) 
but I guess the stdin option can't guess that. I guess one should add an 
'-f' option to read the .cabal file.

Anyways, it seems the "-r" option doesn't work here. I was expecting 
"ghc-servant-swagger" to be defined as well.

P.

On 7.03.2023 17:02, Ludovic Courtès wrote:
> Hi,
> 
> Przemysław Kamiński <cgenie@pm.me> skribis:
> 
>> 3. I run `guix build -L <my-custom-guix-package-dir> -m manifest.scm
>> (manifest only contains a reference to ghc-stuff.scm)
>> 4. I get an error which looks like this:
>> View build log at
>> '/var/log/guix/drvs/j2/wa3vw49l78m85z74x91sklqxwhx87y-ghc-hwodr-0.1.0.drv.bz2'
>>
>> so I fire up
>>
>> bunzip --stdout
>> '/var/log/guix/drvs/j2/wa3vw49l78m85z74x91sklqxwhx87y-ghc-hwodr-0.1.0.drv.bz2'
>>
>> and see that I am missing some Haskell packages.
> 
> Note that the build log should be just above the “View build log” line
> if you scroll up a little bit.  :-)
> 
> Otherwise, like Simon said, I’d recommend getting package definitions by
> running ‘guix import hackage -r’, assuming said packages are on Hackage.
> That should simplify things.
> 
> HTH,
> Ludo’.

[-- Attachment #1.1.1.1.2: OpenPGP public key --]
[-- Type: application/pgp-keys, Size: 1227 bytes --]

[-- Attachment #1.1.2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 236 bytes --]

[-- Attachment #1.2: publickey - cgenie@pm.me - 9cc42b0a.asc --]
[-- Type: application/pgp-keys, Size: 681 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 249 bytes --]

Re: streamlining custom package development

[Simon Tournier]

Hi,

On Tue, 07 Mar 2023 at 16:57, Przemysław Kamiński <cgenie@pm.me> wrote:

> Starting download of /tmp/guix-file.TykbjO
>  From 
> https://hackage.haskell.org/package/servant-examples/servant-examples-0.1.0.0.tar.gz...
> download failed 
> "https://hackage.haskell.org/package/servant-examples/servant-examples-0.1.0.0.tar.gz" 
> 404 "Not Found"

From <https://hackage.haskell.org/package/servant-examples>, I read,

0.4.2, 0.4.3, 0.4.3.1, 0.4.4, 0.4.4.2, 0.4.4.3, 0.4.4.4, 0.4.4.5, 0.4.4.6, 0.4.4.7

and 0.1.0.0 is not listed.  For instance, just tweak the Cabal file,

--8<---------------cut here---------------start------------->8---
$ git diff
diff --git a/servant-examples.cabal b/servant-examples.cabal
index f31e2a4..8861cf2 100644
--- a/servant-examples.cabal
+++ b/servant-examples.cabal
@@ -1,5 +1,5 @@
 name:                servant-examples
-version:             0.1.0.0
+version:             0.4.2
 -- synopsis:
 -- description:
 homepage:            https://github.com/githubuser/servant-examples#readme
--8<---------------cut here---------------end--------------->8---

then Guix finds it in the Hackage infrastructure.

--8<---------------cut here---------------start------------->8---
$ cat servant-examples.cabal | guix import hackage -s -r 

Starting download of /tmp/guix-file.yrblu1
From https://hackage.haskell.org/package/servant-examples/servant-examples-0.4.2.tar.gz...
 …es-0.4.2.tar.gz                              1.7MiB/s 00:00 | 7KiB transferred
Syntax error: unexpected end of input
Syntax error: unexpected end of input
Syntax error: unexpected end of input
Syntax error: unexpected end of input
(define-public ghc-servant-examples
  (package
    (name "ghc-servant-examples")
    (version "0.4.2")
    (source (origin
              (method url-fetch)
              (uri (hackage-uri "servant-examples" version))
              (sha256
               (base32
                "1nmgzkn0nv3p0bzr5ny0yx0g45fzf5qx7s868jp1nq7wbfd3yc0i"))))
    (build-system haskell-build-system)
    (inputs (list ghc-servant
                  ghc-servant-server
                  ghc-servant-swagger
                  ghc-swagger2
                  ghc-wai
                  ghc-aeson
                  ghc-warp))
    (home-page "https://github.com/githubuser/servant-examples#readme")
    (synopsis "")
    (description "")
    (license license:bsd-3)))
--8<---------------cut here---------------end--------------->8---


> My guess the "Syntax error" is from the missing "ghc-servant-swagger" 
> package.

Hum, I guess is that it comes from the option ’-s’.  For instance,

    guix import hackage servant-examples -r

does not display it.  However, I do not find what is the code that
display “Syntax error”.  Maybe a bug here, indeed.



Cheers,
simon