Re: Help packaging R Quarto Cli

[Wojtek Kosior via <help-guix@gnu.org>]

[-- Attachment #1: Type: text/plain, Size: 19389 bytes --]

> >> I found only one crate that use this method `git-fetch’ in the list :
> >> <https://git.savannah.gnu.org/cgit/guix.git/tree/gnu/packages/crates-io.scm#n1576>  
> >
> > There are plenty in other package definition files, e.g. in
> > gnu/packages/python-xyz.scm. It seems Rust is just that special case
> > where git checkout is rarely needed.  
> 
> Yes,  i’m not sure this case (source only without crate) is well managed by the actual cargo-build-system ?
> Anyway, discussions about “Rust Packaging” on IRC mention the building of a new system, something cargo-less.

What do you mean by "source only without crate"? From my understanding,
even when we use `(method uri-fetch)` with `(crate-uri)`, what gets
downloaded is still just the source code and not a prebuilt crate. The
crates repo serves both the crates and their corresponding source
tarballs, right?

Even though there is a difference from the `(method git-fetch)` case —
the output is a tarball instead of a directory — it's still just source
code and the untaring of it is — I believe — done automatically.

Or it is something else you meant?


> > I wanted to instead suggest not creating a separate Guix package
> > nor derivation for test_util. That’s what I meant when I wrote “it
> > seems awkward to be required to provide the test_util package as a
> > separate crate”. Sorry if I accidently mislead you with this. 
> 
> Yes you’re right, this is probably awkard to maintain.
> 
> Hum, i will retry. What’s motivating me to try this packaging way is
> the duration of compilation… This fail only occur at the test phase,
> so after 2 hours of compilation…
> 
> When i use the “–keep-failed” option, and run a new build after fail
> from the /tmp/guix-deno-xxx/ folder, everything finish well …
> 
> So this is more a problem at frontier between this nested package and
> “cargo-build-system” that don’t run some nested crate build.

Did you *just* run a new build inside `/tmp/guix-deno-xxx/`? Or did you
as well run a `cargo test` or similar command after it? Because I
understand it is `cargo test` invoked by cargo-build-system that is
failing[1].

> I will revert my code to manage the deno compilation without test phase.
> My objective is quarto, and i don’t know much about the next obstacle :)
> I see later for this nested crate.

Yup, it's a sane approach.

> >  
> >> How do you reference/call/reuse a package that already exist in your /gnu/store in a .scm file ?  
> >
> > You don’t :)
> >
> > That’s the point of a functional package manager - there’s never need
> > to *explicitly* reference files in the store. They are always referenced
> > indirectly through packages, derivations, plain-file’s, gexps, etc. And
> > they get created or reused automatically.
> >  
> 
> So you directly say into the scm, passing this unique path `/gnu/store/xxxxx-deno-util-1-0-1' to input function ??

No no no.

Let's recall what you wanted to do in the first place

> Another way is to first compile deno-test-util.scm, that install the
> corresponding crate into /gnu/store, and after that i give this local
> path to the inputs of my main rust-deno.scm . But i don’t know how to
> give this path to my cargo build system input . How do you
> reference/call/reuse a package that already exist in your /gnu/store
> in a .scm file ?

I'd say we already reached the decision that we don't want to make
rust-deno-test-util a distinct guix package. But for the sake of
learning, let's imagine that we do and that we want to reference its
crate manually in another package recipe.

Recall that you had the crate file at
`/gnu/store/ma04jfp0f33kf38cdn66qai60nhqxx7d-rust-deno-test-util-0.1.0/share/cargo/registry/test_util-0.1.0.crate`.
In the rust-deno recipe you were importing the record describing that
crate's package. It was being imported under the name
`rust-deno-test-util-0.1.0`.

So the `(arguments)` field of the recipe could contain sth like

    #:phases
    #~(modify-phases %standard-phases
        (add-after 'build 'do-sth-with-test-util-crate
          (lambda _
            (do-something-with
             #$(file-append rust-deno-test-util-0.1.0
                            "/share/cargo/registry/test_util-0.1.0.crate")))))

For simplicity I omitted other extra phases that you might put there.
Notice that the phases are now computed using `#~` which is syntactic
sugar for `(gexp)`. Also note that the crate file path gets introduced
with `#$(file-append ...)` where `#$` is syntactic sugar for `ungexp`.
`(file-append)` itself as well as `#~` and `#$` become available when
the `(guix gexp)` module is imported.

This is a relatively low-level, gexp-based approach to using other
packages during the build (and only the build). It probably looks like
black magic now. To better understand it, please consult your copy of
the "Guix black magic" handbook (aka the "G-Expressions" chapter of the
GNU Guix manual[2]).

If rust-deno-test-util was instead meant to be one of the `(inputs)`
(i.e. if it was one of the runtime deps), we could instead use the
following common idiom.

    #:phases
    `(modify-phases %standard-phases
       (add-after 'build 'do-sth-with-test-util-crate
         (lambda* (#:key inputs #:allow-other-keys)
           (let ((test-util (assoc-ref inputs "rust-deno-test-util")))
             (do-something-with
              (string-append test-util
                             "/share/cargo/registry/test_util-0.1.0.crate"))))))

Here I assumed that "rust-deno-test-util" is the name of the
rust-deno-test-util Guix package and that this name is being
automatically used as its key in the inputs association list. It is
possible to explicitly assign different, non-default string keys to the
packages listed among `(inputs)`. It is, however, not highly relevant
now — let's ignore this possibility.

I already showed 2 ways of accessing a file produced by some
prerequisite package/derivation. In both cases Guix shall automatically
recognize that rust-deno-test-util is a prerequisite and build it if it
is not yet present in `/gnu/store`. None of this approaches should be
needed, however. That's because in the case of cargo-build-system the standard
way of providing crates is by listing the required crate packages among
`#:cargo-inputs` or `#:cargo-development-inputs`. Just as you were
doing. cargo-build-system then by itself takes care of collecting the
crates inside the listed packages and instructing the `cargo` command
to find those crates.

I don't know what was still causing the error *after* you placed
rust-deno-test-util among `#:cargo-development-inputs`. I guess the
reason was specific to Deno build system and not to Guix nor to
cargo-build-system.

> Thanks for your support and your kind word, that help me to continue :)

Glad I could help even with my small Guix experience ^^

> Hi,
> 
> I’m happy to say, Deno is packaged (except the test, see the last conversion on this thread) compile and run on my machine :D
> 
> ┌────
> │ /-> /gnu/store/xvjymz07g2dy112cz4x6pz7v4q8p7c6a-rust-deno-1.25.2/bin/deno --version
> │ deno 1.25.2 (release, x86_64-unknown-linux-gnu)
> │ v8 10.6.194.5
> │ typescript 4.7.4
> │ 
> │ /-> /gnu/store/xvjymz07g2dy112cz4x6pz7v4q8p7c6a-rust-deno-1.25.2/bin/deno run hello-world.js
> │ Hello John
> │ Hello Sarah
> │ Hello Kai
> │ 
> └────

Great! 👏

> Next steps :
> • Packaging Quarto (that need Deno).
> • Merge and Cleaning mess with dependency.

Just a question, out of curiosity - are there any serious freedom
issues? I saw you are using some custom Guix channels in addition to
your own one and that made me wonder

Wojtek


[1] https://git.savannah.gnu.org/cgit/guix.git/tree/guix/build/cargo-build-system.scm?id=408a4ed071c9c52de207d799a698781d49fa727d#n209
[2] https://guix.gnu.org/manual/en/html_node/G_002dExpressions.html

-- (sig_start)
website: https://koszko.org/koszko.html
PGP: https://koszko.org/key.gpg
fingerprint: E972 7060 E3C5 637C 8A4F  4B42 4BC5 221C 5A79 FD1A

Meet Kraków saints!           #16: saint Jan z Dukli
Poznaj świętych krakowskich!  #16: święty Jan z Dukli
https://pl.wikipedia.org/wiki/Jan_z_Dukli
-- (sig_end)


On Thu, 15 Dec 2022 09:32:48 +0100
Sébastien Rey-Coyrehourcq <sebastien.rey-coyrehourcq@univ-rouen.fr> wrote:

> Hi,
> 
> I’m happy to say, Deno is packaged (except the test, see the last conversion on this thread) compile and run on my machine :D
> 
> ┌────
> │ /-> /gnu/store/xvjymz07g2dy112cz4x6pz7v4q8p7c6a-rust-deno-1.25.2/bin/deno --version
> │ deno 1.25.2 (release, x86_64-unknown-linux-gnu)
> │ v8 10.6.194.5
> │ typescript 4.7.4
> │ 
> │ /-> /gnu/store/xvjymz07g2dy112cz4x6pz7v4q8p7c6a-rust-deno-1.25.2/bin/deno run hello-world.js
> │ Hello John
> │ Hello Sarah
> │ Hello Kai
> │ 
> └────
> 
> Next steps :
> • Packaging Quarto (that need Deno).
> • Merge and Cleaning mess with dependency.
> 
> If you want to try :
> <https://git.sr.ht/~reyman/build-deno-guix>
> 
> Best regards,
> SR
> 
> “Sebastien Rey-Coyrehourcq” <sebastien.rey-coyrehourcq@univ-rouen.fr> writes:
> 
> >
> > Le Lundi, Octobre 24, 2022 13:43 CEST, Sébastien Rey-Coyrehourcq <sebastien.rey-coyrehourcq@univ-rouen.fr> a écrit:
> >  
> >> Hi,
> >>
> >> I’m trying to package Quarto Cli ( <https://github.com/quarto-dev/quarto-cli> ), used in combination with Pandoc to publish -reproducible- scientific document : website, blog, etc.
> >>
> >> I first think this is a classic gnu build : ./configure && make && make install BUT, there is a problem because the ./configure script bootstrap “Deno” during the run of configure.sh.
> >>
> >> Because this download and compilation of Deno occur during ./configure.sh running, guix cannot patch the  #!/bin/bash path, so ./configure failed. Deno seems also not packaged into guix.
> >>
> >> Do you have an idea to resolve this ? Perhaps we could try all together to do this.
> >>
> >> I’m starting with this quarto-cli.scm :
> >>
> >> ┌────
> >> │ (use-modules
> >> │ (guix packages)
> >> │ (guix download)
> >> │ (guix build-system gnu)
> >> │ (guix licenses)
> >> │ )
> >> │
> >> │ (define-public quarto-cli
> >> │   (package
> >> │     (name “Quarto-CLI”)
> >> │     (version “1.1.251”)
> >> │     (source (origin
> >> │               (method url-fetch)
> >> │               (uri (string-append “<https://github.com/quarto-dev/quarto-cli/archive/refs/tags/v"version".tar.gz>”))
> >> │               (sha256
> >> │                (base32
> >> │                 “1ycwrjndrrrciymnm3l0lhcd375fddkvjibvc0n084irg6z1lxn6”))))
> >> │     (build-system gnu-build-system)
> >> │     (synopsis “Quarto-cli”)
> >> │     (description
> >> │      “Quarto-cli description”)
> >> │     (home-page “<https://github.com/quarto-dev/quarto-cli>”)
> >> │     (license gpl3+)))
> >> │ quarto-cli
> >> │
> >> └────
> >>
> >> To compile and fail :
> >> guix build -f quarto-cli.scm
> >>
> >> Best,
> >> Sebastien RC.  
> >
> > Deno contain lot of packages dependencies actually,
> > here i comment all packages not packaged in rust after a simple run of guix import …
> >
> > #+BEGIN_SRC scheme
> >
> > (use-modules
> > (guix packages)
> > (guix build-system cargo)
> > (guix download)
> > (guix licenses)
> > (gnu packages rust)
> > (gnu packages crates-io)
> > )
> >
> > (define-public rust-deno-1
> >   (package
> >     (name “rust-deno”)
> >     (version “1.26.2”)
> >     (source (origin
> >               (method url-fetch)
> >               (uri (crate-uri “deno” version))
> >               (file-name (string-append name “-” version “.tar.gz”))
> >               (sha256
> >                (base32
> >                 “1yzvdkj8sq475kfbkms1lfysjddkfwcyqhp1ggalfbk4hqhbiz29”))))
> >     (build-system cargo-build-system)
> >     (arguments
> >      `(#:cargo-inputs ((“rust-atty” ,rust-atty-0.2)
> >                        (“rust-base64” ,rust-base64-0.13)
> > ;                       (“rust-cache-control” ,rust-cache-control-0.2)
> >                         (“rust-chrono” ,rust-chrono-0.4)
> > ;                       (“rust-clap” ,rust-clap-3)
> > ;                       (“rust-clap-complete” ,rust-clap-complete-3)
> > ;                       (“rust-clap-complete-fig” ,rust-clap-complete-fig-3)
> >                         (“rust-data-url” ,rust-data-url-0.1)
> > ;                       (“rust-deno-ast” ,rust-deno-ast-0.19)
> > ;                       (“rust-deno-broadcast-channel” ,rust-deno-broadcast-channel-0.67)
> > ;                       (“rust-deno-cache” ,rust-deno-cache-0.5)
> > ;                       (“rust-deno-console” ,rust-deno-console-0.73)
> > ;                       (“rust-deno-core” ,rust-deno-core-0.155)
> > ;                       (“rust-deno-core” ,rust-deno-core-0.155)
> > ;                       (“rust-deno-crypto” ,rust-deno-crypto-0.87)
> > ;                       (“rust-deno-doc” ,rust-deno-doc-0.46)
> > ;                       (“rust-deno-emit” ,rust-deno-emit-0.9)
> > ;                       (“rust-deno-fetch” ,rust-deno-fetch-0.96)
> > ;                       (“rust-deno-graph” ,rust-deno-graph-0.34)
> > ;                       (“rust-deno-lint” ,rust-deno-lint-0.33)
> > ;                       (“rust-deno-net” ,rust-deno-net-0.65)
> > ;                       (“rust-deno-node” ,rust-deno-node-0.10)
> > ;                       (“rust-deno-runtime” ,rust-deno-runtime-0.81)
> > ;                       (“rust-deno-task-shell” ,rust-deno-task-shell-0.5)
> > ;                       (“rust-deno-url” ,rust-deno-url-0.73)
> > ;                       (“rust-deno-web” ,rust-deno-web-0.104)
> > ;                       (“rust-deno-webgpu” ,rust-deno-webgpu-0.74)
> > ;                       (“rust-deno-websocket” ,rust-deno-websocket-0.78)
> > ;                       (“rust-deno-webstorage” ,rust-deno-webstorage-0.68)
> >                        (“rust-dissimilar” ,rust-dissimilar-1)
> > ;                       (“rust-dprint-plugin-json” ,rust-dprint-plugin-json-0.15)
> > ;                       (“rust-dprint-plugin-markdown” ,rust-dprint-plugin-markdown-0.14)
> > ;                       (“rust-dprint-plugin-typescript” ,rust-dprint-plugin-typescript-0.74)
> >                        (“rust-encoding-rs” ,rust-encoding-rs-0.8)
> >                        (“rust-env-logger” ,rust-env-logger-0.9)
> > ;                       (“rust-eszip” ,rust-eszip-0.28)
> > ;                       (“rust-fancy-regex” ,rust-fancy-regex-0.10)
> >                        (“rust-flate2” ,rust-flate2-1)
> >                        (“rust-fwdansi” ,rust-fwdansi-1)
> > ;                       (“rust-glibc-version” ,rust-glibc-version-0.1)
> >                        (“rust-http” ,rust-http-0.2)
> > ;                       (“rust-import-map” ,rust-import-map-0.12)
> >                        (“rust-indexmap” ,rust-indexmap-1)
> > ;                       (“rust-indicatif” ,rust-indicatif-0.17)
> > ;                       (“rust-jsonc-parser” ,rust-jsonc-parser-0.21)
> > ;                       (“rust-junction” ,rust-junction-0.2)
> >                        (“rust-libc” ,rust-libc-0.2)
> >                        (“rust-log” ,rust-log-0.4)
> > ;                       (“rust-mitata” ,rust-mitata-0.0.7)
> > ;                       (“rust-monch” ,rust-monch-0.2)
> > ;                       (“rust-napi-sym” ,rust-napi-sym-0.3)
> >                        (“rust-notify” ,rust-notify-5)
> >                        (“rust-once-cell” ,rust-once-cell-1)
> >                        (“rust-os-pipe” ,rust-os-pipe-1)
> >                        (“rust-percent-encoding” ,rust-percent-encoding-2)
> >                        (“rust-pin-project” ,rust-pin-project-1)
> >                        (“rust-rand” ,rust-rand-0.8)
> >                        (“rust-regex” ,rust-regex-1)
> >                        (“rust-regex” ,rust-regex-1)
> >                        (“rust-ring” ,rust-ring-0.16)
> > ;                       (“rust-rustyline” ,rust-rustyline-10)
> > ;                       (“rust-rustyline-derive” ,rust-rustyline-derive-0.7)
> >                        (“rust-semver” ,rust-semver-1)
> >                        (“rust-serde” ,rust-serde-1)
> >                        (“rust-serde” ,rust-serde-1)
> >                        (“rust-serde-json” ,rust-serde-json-1)
> >                        (“rust-serde-repr” ,rust-serde-repr-0.1)
> >                        (“rust-shell-escape” ,rust-shell-escape-0.1)
> >                        (“rust-tar” ,rust-tar-0.4)
> >                        (“rust-tempfile” ,rust-tempfile-3)
> >                        (“rust-text-size” ,rust-text-size-1)
> > ;                       (“rust-text-lines” ,rust-text-lines-0.6)
> >                        (“rust-tokio” ,rust-tokio-1)
> > ;                       (“rust-tokio-util” ,rust-tokio-util-0.7)
> > ;                       (“rust-tower-lsp” ,rust-tower-lsp-0.17)
> >                        (“rust-twox-hash” ,rust-twox-hash-1)
> >                        (“rust-typed-arena” ,rust-typed-arena-2)
> > ;                       (“rust-uuid” ,rust-uuid-1)
> >                        (“rust-walkdir” ,rust-walkdir-2)
> >                        (“rust-winapi” ,rust-winapi-0.3)
> >                        (“rust-winapi” ,rust-winapi-0.3)
> >                        (“rust-winres” ,rust-winres-0.1)
> > ;                      (“rust-zstd” ,rust-zstd-0.11)
> >                        )
> >        #:cargo-development-inputs (
> >                                    ;(“rust-deno-bench-util” ,rust-deno-bench-util-0.67)
> >                                    (“rust-dotenv” ,rust-dotenv-0.15)
> >                                    ;(“rust-flaky-test” ,rust-flaky-test-0.1)
> >                                    ;(“rust-nix” ,rust-nix-0.24)
> >                                    (“rust-once-cell” ,rust-once-cell-1)
> >                                    (“rust-os-pipe” ,rust-os-pipe-1)
> >                                    (“rust-pretty-assertions” ,rust-pretty-assertions-1)
> >                                    ;(“rust-trust-dns-client” ,rust-trust-dns-client-0.22)
> >                                    ;(“rust-trust-dns-server” ,rust-trust-dns-server-0.22))))
> >     (home-page “<https://github.com/denoland/deno>”)
> >     (synopsis “Provides the deno executable”)
> >     (description “This package provides the deno executable”)
> >     (license expat)))
> >
> > rust-deno-1
> >
> > #+END_SRC
> >
> >
> >
> >
> >
> >
> >
> >  



[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 228 bytes --]