> Yes, you’re right i correct this thing, but this is weird in all case, here i clone all deno repository (at tag v1.25.2), and this is renamed as `deno-test-util-checkout-0-1-0' because i’m only interested by this folder > > Ah, I see why you wanted a name like this. Well, since the sources are the same as those of rust-deno, perhaps it would make sense to use the same origin as in rust-deno recipe? But don't worry about this now. Your build of rust-deno-test-util evidently worked so cleaning up origin definitions can wait until everything else works :) > I found only one crate that use this method `git-fetch' in the list : > There are plenty in other package definition files, e.g. in gnu/packages/python-xyz.scm. It seems Rust is just that special case where git checkout is rarely needed. > So i suppose this is possible to reuse as a dependency in another build… > > This is an intuition but i think i have a problem of ordering in my way to do things : > > When i read the `cargo-build-system' sources here ( ) i see that the build input wait for an /gnu/store link that contain a `.crate' or the equivalent as `tar.gz' > > Or in my case, the /gnu/store link called by cargo contain only sources, because the `deno-util' need to be built before creating .crate into the gnu/store. Perhaps i could cheating the system by providing myself a `gnu/store/xxxx.tar.gz' that contain this sources. > > Inside my `rust-deno-test-util-1-0-0.scm` file i could `(invoke tar ...)' command to generate a tar.gz at the root of the this folder. Do you think that could work ? > > Hum, not sure … the real problem is the fact that guix don’t build the derivation completely? (creating and storing a .crate for deno-test-util-0-1-0 into /gnu/store) when i run build of main rust-deno.scm … only the source are present, and not the package resulting from the build of this derivation. > > Another way is to first compile deno-test-util.scm, that install the corresponding crate into /gnu/store, and after that i give this local path to the inputs of my main rust-deno.scm . But i don’t know how to give this path to my cargo build system input . I'm getting a bit lost in trying to figure out what you think right now. The `/gnu/store/ma04jfp0f33kf38cdn66qai60nhqxx7d-rust-deno-test-util-0.1.0/` was a proper build of your `rust-deno-test-util` package and it contains a proper crate under `share/cargo/registry/`. Code in `cargo-build-system.scm` looks for crates in subdirectory `share/cargo/registry/` of every rust input package — all seems correct under the assumption that we want test_util to be supplied to deno build as a separate Guix package. It's perhaps possible to achieve success with this assumption (I don't know) but I wanted to instead suggest not creating a separate Guix package nor derivation for test_util. That's what I meant when I wrote "it seems awkward to be required to provide the test_util package as a separate crate". Sorry if I accidently mislead you with this. Earlier you wrote > This nested crate package is not detected/compiled by actual > `cargo-build-system’, so i decide to package it myself , as > `rust-deno-test-util-0.1.0’ What seems to be the most straightforward and promising way is to (somehow?) fix the build system to detect the nested package. I know, it's easy to say — and you surely made extensive attempts before falling back to creating an extra definition for rust-deno-test-util. Hopefully, inspection of the build container will help shed some light on this. Also, the git-fetch issues will become nonexistent if you manage to do it this way :) Since your rust-deno recipe uses `(method url-fetch)` with `(crate-uri)`, I'm starting to wonder if the tarball that's being used in this case contains exactly the same files as the git checkout. I earlier assumed that it does when I suggested reusing the origin in your rust-deno-test-util recipe. But perhaps it doesn't? I could imagine deno devs omitting e.g. `test_util/` from the source tarball they upload to the crates repo. > How do you reference/call/reuse a package that already exist in your /gnu/store in a .scm file ? You don't :) That's the point of a functional package manager - there's never need to *explicitly* reference files in the store. They are always referenced indirectly through packages, derivations, plain-file's, gexps, etc. And they get created or reused automatically. Wojtek -- (sig_start) website: https://koszko.org/koszko.html PGP: https://koszko.org/key.gpg fingerprint: E972 7060 E3C5 637C 8A4F 4B42 4BC5 221C 5A79 FD1A Meet Kraków saints! #25: blessed Maksymilian Binkiewicz Poznaj świętych krakowskich! #25: błogosławiony Maksymilian Binkiewicz https://pl.wikipedia.org/wiki/Maksymilian_Binkiewicz -- (sig_end) On Wed, 14 Dec 2022 17:16:45 +0100 Sébastien Rey-Coyrehourcq wrote: > Wojtek Kosior writes: > > > > Hello again > > > > I am impressed with your determination :) > > > > thx :D > > > I see most packages in Guix that use `(method git-fetch)` also use > > `(file-name (git-file-name name version))` in `(origin)`. > > > > Note that this only affects the name under which the git checkout is > > saved. Whether the source checkout directory under `/gnu/store` is > > named `8479xfpn9hp2b3kc9d3596kpncan9d8w-rust-deno-test-util-0.1.0.tar.gz/` > > or something else should not be relevant to the build. > > > > Yes, you’re right i correct this thing, but this is weird in all case, here i clone all deno repository (at tag v1.25.2), and this is renamed as `deno-test-util-checkout-0-1-0' because i’m only interested by this folder > > > > I found only one crate that use this method `git-fetch' in the list : > > > So i suppose this is possible to reuse as a dependency in another build… > > This is an intuition but i think i have a problem of ordering in my way to do things : > > When i read the `cargo-build-system' sources here ( ) i see that the build input wait for an /gnu/store link that contain a `.crate' or the equivalent as `tar.gz' > > Or in my case, the /gnu/store link called by cargo contain only sources, because the `deno-util' need to be built before creating .crate into the gnu/store. Perhaps i could cheating the system by providing myself a `gnu/store/xxxx.tar.gz' that contain this sources. > > Inside my `rust-deno-test-util-1-0-0.scm` file i could `(invoke tar ...)' command to generate a tar.gz at the root of the this folder. Do you think that could work ? > > Hum, not sure … the real problem is the fact that guix don’t build the derivation completely? (creating and storing a .crate for deno-test-util-0-1-0 into /gnu/store) when i run build of main rust-deno.scm … only the source are present, and not the package resulting from the build of this derivation. > > Another way is to first compile deno-test-util.scm, that install the corresponding crate into /gnu/store, and after that i give this local path to the inputs of my main rust-deno.scm . But i don’t know how to give this path to my cargo build system input . How do you reference/call/reuse a package that already exist in your /gnu/store in a .scm file ? > > > Although I doubt I will be able to help with no knowledge of Rust, I’ll > > at least suggest that you attach the full rust-deno build logs here. > > This will perhaps at least enable others to help (the time needed to > > build it all does scare away!). > > > > Btw, it seems awkward to be required to provide the test_util package > > as a separate crate. I’d rather imagine Deno devs somehow using the > > test_util sources already in the deno tree. > > > > Yeah, but now it’s packaged and the package compile, i try one more time like that :D > > > Have you tried fiddling around inside the build container? If not, you > > may want to look at this manual node[1]. Perhaps after running some of > > the build commands manually you’ll get better ideas? > > Thanks i will look on that > > > > At the end, in the worst case, you could resort to disabling tests for > > the package. Nobody likes this “solution” but sometimes it is the only > > way to go ¯\_(ツ)_/¯ > > Ahah right, i probably do that if i want to finish in 2022 :D > > Best, > Sébastien Rey-Coyrehourcq > > > > > Good luck, > > Wojtek > > > > [1] > > > > – (sig_start) > > website: > > PGP: > > fingerprint: E972 7060 E3C5 637C 8A4F 4B42 4BC5 221C 5A79 FD1A > > > > Meet Kraków saints! #24: blessed Ludwik Pius Bartosik > > Poznaj świętych krakowskich! #24: błogosławiony Ludwik Pius Bartosik > > > > – (sig_end) > > > > > > On Wed, 14 Dec 2022 11:30:16 +0100 > > Sébastien Rey-Coyrehourcq wrote: > > > >> Hi, > >> > >> After two weeks of suffering and pain with this complex rust packaging, and > >> thanks to people on the libera #guix chat, my “rust packaging adventure” is > >> near ending… at least with deno, i see quarto after that… > >> > >> Everything compile by parts, and i need a final help to merge things. > >> > >> I only have a problem at `testing-phase’, with a local rust package defined on Deno `/test_util’ folder. This package is not published at all on crates.io and it’s needed at test-phase… > >> > >> This nested crate package is not detected/compiled by actual `cargo-build-system’, so i decide to package it myself , as `rust-deno-test-util-0.1.0’ > >> > >> I tested and this crate compile well, with two derivation : > >> > >> • `/gnu/store/ma04jfp0f33kf38cdn66qai60nhqxx7d-rust-deno-test-util-0.1.0/’ > >> • `/gnu/store/zp5flzykz7y5n35kqrlryqkynvrvcw3z-rust-deno-test-util-0.1.0.drv’ > >> > >> `/gnu/store/ma04jfp0f33kf38cdn66qai60nhqxx7d-rust-deno-test-util-0.1.0/share/cargo/registry/’ contain the test_util-0.1.0.crate > >> > >> So … i added this *.scm* to my *deno/deno-dep* list of package in the *rust-channel* needed by deno to compile : > >> > >> > >> *My problem is simple?* i added this crate as a dependency of my main `rust-deno.scm’ : > >> • `#:use-module (deno-dep rust-deno-test-util-0-1-0)’ > >> • `#:cargo-development-inputs `(((“rust-deno-test-util”, rust-deno-test-util-0.1.0) … )’ > >> > >> BUT BUT BUT … rust-deno-test-util is not found and not added to `/guix-vendor’ > >> during deno crate retrieval, so there is a problem somewhere during packaging > >> retrieval from local `/gnu/store’. I found in the log that > >> > >> `/gnu/store/8479xfpn9hp2b3kc9d3596kpncan9d8w-rust-deno-test-util-0.1.0.tar.gz/’ > >> contain the sources and not a `tar.gz’ like others crates. I suppose this is > >> part of the problem, but i don’t know how to solve that. > >> > >> See by yourself using : > >> > >> guix time-machine -C channels.scm – build -f rust-deno.scm > >> > >> that return : > >> > >> error: no matching package named `deno_test_util` found > >> > >> **Info to reproduce** > >> > >> • All these package needed by Deno are defined into my-rust channel here : > >> • The building and channel scm needed to build deno are here : > >> • /Final Warning :/ Deno build (outside rust build) take at least 2 hours on my machine, rusty-v8 is huge and take itself ~30 to 45 minutes. > >> > >> Any *final* help appreciated ! > >> > >> Best , > >> Sébastien Rey-Coyrehourcq > >> > >> > >> Sébastien Rey-Coyrehourcq writes: > >> > >> > > >> > Hi, > >> > > >> > Thanks a lot, that helps me to make one more step :) > >> > > >> > I set : > >> > > >> > ┌──── > >> > │ (arguments > >> > │ `(#:phases > >> > │ (modify-phases %standard-phases > >> > │ (add-before ’configure ’set-source > >> > │ (lambda _ > >> > │ (setenv “V8_FROM_SOURCE” “1”) > >> > │ (setenv “RUST_BACKTRACE” “1”) > >> > │ (setenv “CLANG_BASE_PATH” (getenv “CMAKE_PREFIX_PATH”)) > >> > │ #t))) > >> > └──── > >> > > >> > I also try C_INCLUDE_PATH > >> > > >> > and native input : > >> > > >> > ┌──── > >> > │ (native-inputs (list ninja gn clang-toolchain ccache clang)) > >> > └──── > >> > > >> > I have now another problem during compilation of v8, i don’t understand why “clang” path is not well recognized by the buildscript, test is defined here : > >> > > >> > > >> > > >> > The backtrace : > >> > > >> > error: failed to run custom build command for `v8 v0.49.0` > >> > > >> > Caused by: > >> > process didn’t exit successfully: `/tmp/guix-build-rust-deno-1.25.2.drv-0/deno-1.25.2/target/release/build/v8-bbb68ec56db1d802/build-script-build` (exit status: 101) > >> > — stdout > >> > cargo:rerun-if-changed=.gn > >> > cargo:rerun-if-changed=BUILD.gn > >> > cargo:rerun-if-changed=src/binding.cc > >> > cargo:rerun-if-env-changed=CCACHE > >> > cargo:rerun-if-env-changed=CLANG_BASE_PATH > >> > cargo:rerun-if-env-changed=DENO_TRYBUILD > >> > cargo:rerun-if-env-changed=DOCS_RS > >> > cargo:rerun-if-env-changed=GN > >> > cargo:rerun-if-env-changed=GN_ARGS > >> > cargo:rerun-if-env-changed=HOST > >> > cargo:rerun-if-env-changed=NINJA > >> > cargo:rerun-if-env-changed=OUT_DIR > >> > cargo:rerun-if-env-changed=RUSTY_V8_ARCHIVE > >> > cargo:rerun-if-env-changed=RUSTY_V8_MIRROR > >> > cargo:rerun-if-env-changed=SCCACHE > >> > cargo:rerun-if-env-changed=V8_FORCE_DEBUG > >> > cargo:rerun-if-env-changed=V8_FROM_SOURCE > >> > cargo:rustc-link-lib=static=rusty_v8 > >> > using Chromiums clang > >> > clang_base_path /tmp/guix-build-rust-deno-1.25.2.drv-0/deno-1.25.2/target/release/clang > >> > > >> > — stderr > >> > thread ’main’ panicked at ’called `Result::unwrap()` on an `Err` value: Os { code: 2, kind: NotFound, message: “No such file or directory” }’, /tmp/guix-build-rust-deno-1.25.2.drv-0/deno-1.25.2/guix-vendor/rust-v8-0.49.0.tar.gz/build.rs:537:6 > >> > stack backtrace: > >> > 0: rust_begin_unwind > >> > 1: core::panicking::panic_fmt > >> > 2: core::result::unwrap_failed > >> > 3: core::result::Result::unwrap > >> > 4: build_script_build::clang_download > >> > 5: build_script_build::build_v8 > >> > 6: build_script_build::main > >> > 7: core::ops::function::FnOnce::call_once > >> > note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace. > >> > > >> > Best, > >> > > >> > > >> > Wojtek Kosior writes: > >> > > >> >>> Hi, > >> >>> > >> >>> You’re both right, seems there is a flag to skip binary downloading and > >> >>> compile the V8 lib. > >> >>> > >> >>> […] > >> >> > >> >> Good to see you found it :) > >> >> > >> >>> So, my packaging friend :), what’s the best way to push an “export > >> >>> V8_FROM_SOURCE=1” or something like that into my rust-deno.scm ? > >> >>> > >> >>> Best, > >> >>> > >> >>> SR > >> >> > >> >> When I first read your question, I did not know the exact function. But > >> >> I knew where to look. So I thought I’d better share my way of learning > >> >> rather than just the solution ;) > >> >> > >> >> I started with > >> >> > >> >> grep -R ’export’ ~/.config/guix/current/share/guile/site/3.0/gnu/packages/ | less > >> >> > >> >> That showed a lot of results. I noticed a line like this > >> >> > >> >>> /home/urz/.config/guix/current/share/guile/site/3.0/gnu/packages/engineering.scm: (setenv “HOME” (getcwd)) ;; cmake needs this to export modules > >> >> > >> >> Thought it might be the thing I was looking for, so I did > >> >> > >> >> less /home/urz/.config/guix/current/share/guile/site/3.0/gnu/packages/engineering.scm > >> >> > >> >> and navigated to this line. This seems to be it. The (setenv) function. > >> >> Can be used as from a modified packaging phase function (as you can see > >> >> in engineering.scm). > >> >> > >> >> Hope I helped. Good luck once again! > >> >> > >> >> W. > >> >> > >> >> P.S. Make sure you know some ’less’ shortcuts if you’re going to do > >> >> things this way. It cad speed things up ^^ > >> >> > >> >> – (sig_start) > >> >> website: > >> >> PGP: > >> >> fingerprint: E972 7060 E3C5 637C 8A4F 4B42 4BC5 221C 5A79 FD1A > >> >> > >> >> Meet Kraków saints! #12: saint Jadwiga Andegaweńska > >> >> Poznaj świętych krakowskich! #12: święta Jadwiga Andegaweńska > >> >> > >> >> – (sig_end) > >> >> > >> >> > >> >> On Wed, 16 Nov 2022 21:38:47 +0100 > >> >> Sebastien Rey-Coyrehourcq wrote: > >> >> > >> >>> Hi, > >> >>> > >> >>> You’re both right, seems there is a flag to skip binary downloading and > >> >>> compile the V8 lib. > >> >>> > >> >>> From the githubpage () : “V8 is > >> >>> very large and takes a long time to compile. Many users will prefer to > >> >>> use a prebuilt version of V8. We publish static libs for every version > >> >>> of rusty v8 on Github . > >> >>> > >> >>> Binaries builds are turned on by default: |cargo build| will initiate a > >> >>> download from github to get the static lib. To disable this build using > >> >>> the |V8_FROM_SOURCE| environmental variable. > >> >>> > >> >>> When making changes to rusty_v8 itself, it should be tested by build > >> >>> from source. The CI always builds from source” > >> >>> > >> >>> So, my packaging friend :), what’s the best way to push an “export > >> >>> V8_FROM_SOURCE=1” or something like that into my rust-deno.scm ? > >> >>> > >> >>> Best, > >> >>> > >> >>> SR > >> >>> > >> >>> Le 15/11/2022 à 08:58, Efraim Flashner a écrit : > >> >>> > On Mon, Nov 14, 2022 at 11:30:47PM +0100, Sébastien Rey-Coyrehourcq wrote: > >> >>> >> Hi, > >> >>> >> > >> >>> >> After some day of packaging rust crate, i progress and deno start to > >> >>> >> compile … but after 1min i have this error when cargo start compiling > >> >>> >> *rust-v8-0.49* . Any rust + guix help appreciated. > >> >>> >> > >> >>> >> I push the channel to reproduce the problem here : > >> >>> >> > >> >>> >> The rust scm repo : git.sr.ht:~reyman/rust-channel > >> >>> >> Channel info to put into *channels.scm* : > >> >>> >> The *rust-deno.scm* file to build : > >> >>> >> The command : guix time-machine -C channels.scm – build -f rust-deno.scm > >> >>> >> > >> >>> >> And the rust error : > >> >>> >> > >> >>> >> — > >> >>> >> > >> >>> >> error: failed to run custom build command for `v8 v0.49.0` > >> >>> >> > >> >>> >> Caused by: > >> >>> >> process didn’t exit successfully: `/tmp/guix-build-rust-deno-1.25.2.drv-0/deno-1.25.2/target/release/build/v8-bbb68ec56db1d802/build-script-build` (exit status: 101) > >> >>> >> — stdout > >> >>> >> cargo:rerun-if-changed=.gn > >> >>> >> cargo:rerun-if-changed=BUILD.gn > >> >>> >> cargo:rerun-if-changed=src/binding.cc > >> >>> >> cargo:rerun-if-env-changed=CCACHE > >> >>> >> cargo:rerun-if-env-changed=CLANG_BASE_PATH > >> >>> >> cargo:rerun-if-env-changed=DENO_TRYBUILD > >> >>> >> cargo:rerun-if-env-changed=DOCS_RS > >> >>> >> cargo:rerun-if-env-changed=GN > >> >>> >> cargo:rerun-if-env-changed=GN_ARGS > >> >>> >> cargo:rerun-if-env-changed=HOST > >> >>> >> cargo:rerun-if-env-changed=NINJA > >> >>> >> cargo:rerun-if-env-changed=OUT_DIR > >> >>> >> cargo:rerun-if-env-changed=RUSTY_V8_ARCHIVE > >> >>> >> cargo:rerun-if-env-changed=RUSTY_V8_MIRROR > >> >>> >> cargo:rerun-if-env-changed=SCCACHE > >> >>> >> cargo:rerun-if-env-changed=V8_FORCE_DEBUG > >> >>> >> cargo:rerun-if-env-changed=V8_FROM_SOURCE > >> >>> >> cargo:rustc-link-lib=static=rusty_v8 > >> >>> >> download lockfile: “/tmp/guix-build-rust-deno-1.25.2.drv-0/deno-1.25.2/target/release/build/lib_download.fslock” > >> >>> >> static lib URL: > >> >>> >> cargo:rustc-link-search=/tmp/guix-build-rust-deno-1.25.2.drv-0/deno-1.25.2/target/release/gn_out/obj > >> >>> >> Downloading > >> >>> >> Python downloader failed, trying with curl. > >> >>> > Looks like you need to patch rust-v8-0.49 to not try to download > >> >>> > librusty_v8_release… but instead you’ll have to build it from source > >> >>> > and let it know where to find it. > >> >>> > > >> >>> >> — stderr > >> >>> >> thread ’main’ panicked at ’called `Result::unwrap()` on an `Err` value: > >> >>> >> Os { code: 2, kind: NotFound, message: “No such file or directory” }’, > >> >>> >> /tmp/guix-build-rust-deno-1.25.2.drv-0/deno-1.25.2/guix-vendor/rust-v8-0.49.0.tar.gz/build.rs:405:10 > >> >>> >> note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace > >> >>> >> warning: build failed, waiting for other jobs to finish… > >> >>> >> error: build failed > >> >>> >> error: in phase ’build’: uncaught exception: > >> >>> >> %exception #<&invoke-error program: “cargo” arguments: (“build” “–release”) exit-status: 101 term-signal: #f stop-signal: #f> > >> >>> >> phase `build’ failed after 105.5 seconds > >> >>> >> command “cargo” “build” “–release” failed with status 101 > >> >>> >> builder for `/gnu/store/g4m5c558l1q4g1kggzg2v9vkw352nnaj-rust-deno-1.25.2.drv’ failed with exit code 1 > >> >>> >> la compilation de /gnu/store/g4m5c558l1q4g1kggzg2v9vkw352nnaj-rust-deno-1.25.2.drv a échoué > >> >>> >> Vous trouverez le journal de compilation dans « /var/log/guix/drvs/g4/m5c558l1q4g1kggzg2v9vkw352nnaj-rust-deno-1.25.2.drv.gz ». > >> >>> >> guix build: erreur : build of `/gnu/store/g4m5c558l1q4g1kggzg2v9vkw352nnaj-rust-deno-1.25.2.drv’ failed > >> >>> > > >> >> > >> >> > >> > > > > >