Wojtek Kosior writes: >> Yes, you’re right i correct this thing, but this is weird in all case, here i > clone all deno repository (at tag v1.25.2), and this is renamed as > `deno-test-util-checkout-0-1-0’ because i’m only interested by this folder >> >> > > Ah, I see why you wanted a name like this. Well, since the sources are > the same as those of rust-deno, perhaps it would make sense to use the > same origin as in rust-deno recipe? > > But don’t worry about this now. Your build of rust-deno-test-util > evidently worked so cleaning up origin definitions can wait until > everything else works :) > >> I found only one crate that use this method `git-fetch’ in the list : >> > > There are plenty in other package definition files, e.g. in > gnu/packages/python-xyz.scm. It seems Rust is just that special case > where git checkout is rarely needed. Yes, i’m not sure this case (source only without crate) is well managed by the actual cargo-build-system ? Anyway, discussions about “Rust Packaging” on IRC mention the building of a new system, something cargo-less. > >> So i suppose this is possible to reuse as a dependency in another build… >> >> This is an intuition but i think i have a problem of ordering in my way to do things : >> >> When i read the `cargo-build-system’ sources here ( > > ) i see that the build input wait for an /gnu/store link that contain a `.crate’ > or the equivalent as `tar.gz’ >> >> Or in my case, the /gnu/store link called by cargo contain only sources, > because the `deno-util’ need to be built before creating .crate into the > gnu/store. Perhaps i could cheating the system by providing myself a > `gnu/store/xxxx.tar.gz’ that contain this sources. >> >> Inside my `rust-deno-test-util-1-0-0.scm` file i could `(invoke tar …)’ > command to generate a tar.gz at the root of the this folder. Do you think that > could work ? >> >> Hum, not sure … the real problem is the fact that guix don’t build the > derivation completely? (creating and storing a .crate for deno-test-util-0-1-0 > into /gnu/store) when i run build of main rust-deno.scm … only the source are > present, and not the package resulting from the build of this derivation. >> >> Another way is to first compile deno-test-util.scm, that install the > corresponding crate into /gnu/store, and after that i give this local path to > the inputs of my main rust-deno.scm . But i don’t know how to give this path to > my cargo build system input . > > I’m getting a bit lost in trying to figure out what you think right > now. The > `/gnu/store/ma04jfp0f33kf38cdn66qai60nhqxx7d-rust-deno-test-util-0.1.0/` > was a proper build of your `rust-deno-test-util` package and it > contains a proper crate under `share/cargo/registry/`. Code in > `cargo-build-system.scm` looks for crates in subdirectory > `share/cargo/registry/` of every rust input package — all seems correct > under the assumption that we want test_util to be supplied to deno > build as a separate Guix package. > > It’s perhaps possible to achieve success with this assumption (I don’t > know) but I wanted to instead suggest not creating a separate Guix > package nor derivation for test_util. That’s what I meant when I wrote > “it seems awkward to be required to provide the test_util package as a > separate crate”. Sorry if I accidently mislead you with this. > Yes you’re right, this is probably awkard to maintain. Hum, i will retry. What’s motivating me to try this packaging way is the duration of compilation… This fail only occur at the test phase, so after 2 hours of compilation… When i use the “–keep-failed” option, and run a new build after fail from the /tmp/guix-deno-xxx/ folder, everything finish well … So this is more a problem at frontier between this nested package and “cargo-build-system” that don’t run some nested crate build. > Earlier you wrote > >> This nested crate package is not detected/compiled by actual >> `cargo-build-system’, so i decide to package it myself , as >> `rust-deno-test-util-0.1.0’ > > What seems to be the most straightforward and promising way is to > (somehow?) fix the build system to detect the nested package. I know, > it’s easy to say — and you surely made extensive attempts before > falling back to creating an extra definition for rust-deno-test-util. > Hopefully, inspection of the build container will help shed some light > on this. > > Also, the git-fetch issues will become nonexistent if you manage to do > it this way :) > I will revert my code to manage the deno compilation without test phase. My objective is quarto, and i don’t know much about the next obstacle :) I see later for this nested crate. > Since your rust-deno recipe uses `(method url-fetch)` with > `(crate-uri)`, I’m starting to wonder if the tarball that’s being used > in this case contains exactly the same files as the git checkout. I > earlier assumed that it does when I suggested reusing the origin in > your rust-deno-test-util recipe. But perhaps it doesn’t? I could > imagine deno devs omitting e.g. `test_util/` from the source tarball > they upload to the crates repo.: As say, the `deno_root_package/test_util/' folder is well downloaded even with `method url-fetch' > >> How do you reference/call/reuse a package that already exist in your /gnu/store in a .scm file ? > > You don’t :) > > That’s the point of a functional package manager - there’s never need > to *explicitly* reference files in the store. They are always referenced > indirectly through packages, derivations, plain-file’s, gexps, etc. And > they get created or reused automatically. > So you directly say into the scm, passing this unique path `/gnu/store/xxxxx-deno-util-1-0-1' to input function ?? Thanks for your support and your kind word, that help me to continue :) Best > Wojtek > > – (sig_start) > website: > PGP: > fingerprint: E972 7060 E3C5 637C 8A4F 4B42 4BC5 221C 5A79 FD1A > > Meet Kraków saints! #25: blessed Maksymilian Binkiewicz > Poznaj świętych krakowskich! #25: błogosławiony Maksymilian Binkiewicz > > – (sig_end) > > > On Wed, 14 Dec 2022 17:16:45 +0100 > Sébastien Rey-Coyrehourcq wrote: > >> Wojtek Kosior writes: >> >> >> > Hello again >> > >> > I am impressed with your determination :) >> > >> >> thx :D >> >> > I see most packages in Guix that use `(method git-fetch)` also use >> > `(file-name (git-file-name name version))` in `(origin)`. >> > >> > Note that this only affects the name under which the git checkout is >> > saved. Whether the source checkout directory under `/gnu/store` is >> > named `8479xfpn9hp2b3kc9d3596kpncan9d8w-rust-deno-test-util-0.1.0.tar.gz/` >> > or something else should not be relevant to the build. >> > >> >> Yes, you’re right i correct this thing, but this is weird in all case, here i > clone all deno repository (at tag v1.25.2), and this is renamed as > `deno-test-util-checkout-0-1-0’ because i’m only interested by this folder >> >> >> >> I found only one crate that use this method `git-fetch’ in the list : >> >> >> So i suppose this is possible to reuse as a dependency in another build… >> >> This is an intuition but i think i have a problem of ordering in my way to do things : >> >> When i read the `cargo-build-system’ sources here ( > > ) i see that the build input wait for an /gnu/store link that contain a `.crate’ > or the equivalent as `tar.gz’ >> >> Or in my case, the /gnu/store link called by cargo contain only sources, > because the `deno-util’ need to be built before creating .crate into the > gnu/store. Perhaps i could cheating the system by providing myself a > `gnu/store/xxxx.tar.gz’ that contain this sources. >> >> Inside my `rust-deno-test-util-1-0-0.scm` file i could `(invoke tar …)’ > command to generate a tar.gz at the root of the this folder. Do you think that > could work ? >> >> Hum, not sure … the real problem is the fact that guix don’t build the > derivation completely? (creating and storing a .crate for deno-test-util-0-1-0 > into /gnu/store) when i run build of main rust-deno.scm … only the source are > present, and not the package resulting from the build of this derivation. >> >> Another way is to first compile deno-test-util.scm, that install the > corresponding crate into /gnu/store, and after that i give this local path to > the inputs of my main rust-deno.scm . But i don’t know how to give this path to > my cargo build system input . How do you reference/call/reuse a package that > already exist in your /gnu/store in a .scm file ? >> >> > Although I doubt I will be able to help with no knowledge of Rust, I’ll >> > at least suggest that you attach the full rust-deno build logs here. >> > This will perhaps at least enable others to help (the time needed to >> > build it all does scare away!). >> > >> > Btw, it seems awkward to be required to provide the test_util package >> > as a separate crate. I’d rather imagine Deno devs somehow using the >> > test_util sources already in the deno tree. >> > >> >> Yeah, but now it’s packaged and the package compile, i try one more time like that :D >> >> > Have you tried fiddling around inside the build container? If not, you >> > may want to look at this manual node[1]. Perhaps after running some of >> > the build commands manually you’ll get better ideas? >> >> Thanks i will look on that >> > >> > At the end, in the worst case, you could resort to disabling tests for >> > the package. Nobody likes this “solution” but sometimes it is the only >> > way to go ¯\_(ツ)_/¯ >> >> Ahah right, i probably do that if i want to finish in 2022 :D >> >> Best, >> Sébastien Rey-Coyrehourcq >> >> > >> > Good luck, >> > Wojtek >> > >> > [1] >> > >> > – (sig_start) >> > website: >> > PGP: >> > fingerprint: E972 7060 E3C5 637C 8A4F 4B42 4BC5 221C 5A79 FD1A >> > >> > Meet Kraków saints! #24: blessed Ludwik Pius Bartosik >> > Poznaj świętych krakowskich! #24: błogosławiony Ludwik Pius Bartosik >> > >> > – (sig_end) >> > >> > >> > On Wed, 14 Dec 2022 11:30:16 +0100 >> > Sébastien Rey-Coyrehourcq wrote: >> > >> >> Hi, >> >> >> >> After two weeks of suffering and pain with this complex rust packaging, and >> >> thanks to people on the libera #guix chat, my “rust packaging adventure” is >> >> near ending… at least with deno, i see quarto after that… >> >> >> >> Everything compile by parts, and i need a final help to merge things. >> >> >> >> I only have a problem at `testing-phase’, with a local rust package defined > on Deno `/test_util’ folder. This package is not published at all on crates.io > and it’s needed at test-phase… >> >> >> >> This nested crate package is not detected/compiled by actual `cargo-build-system’, so i decide to package it myself , as `rust-deno-test-util-0.1.0’ >> >> >> >> I tested and this crate compile well, with two derivation : >> >> >> >> • `/gnu/store/ma04jfp0f33kf38cdn66qai60nhqxx7d-rust-deno-test-util-0.1.0/’ >> >> • `/gnu/store/zp5flzykz7y5n35kqrlryqkynvrvcw3z-rust-deno-test-util-0.1.0.drv’ >> >> >> >> `/gnu/store/ma04jfp0f33kf38cdn66qai60nhqxx7d-rust-deno-test-util-0.1.0/share/cargo/registry/’ contain the test_util-0.1.0.crate >> >> >> >> So … i added this *.scm* to my *deno/deno-dep* list of package in the *rust-channel* needed by deno to compile : >> >> >> >> >> >> *My problem is simple?* i added this crate as a dependency of my main `rust-deno.scm’ : >> >> • `#:use-module (deno-dep rust-deno-test-util-0-1-0)’ >> >> • `#:cargo-development-inputs `(((“rust-deno-test-util”, rust-deno-test-util-0.1.0) … )’ >> >> >> >> BUT BUT BUT … rust-deno-test-util is not found and not added to `/guix-vendor’ >> >> during deno crate retrieval, so there is a problem somewhere during packaging >> >> retrieval from local `/gnu/store’. I found in the log that >> >> >> >> `/gnu/store/8479xfpn9hp2b3kc9d3596kpncan9d8w-rust-deno-test-util-0.1.0.tar.gz/’ >> >> contain the sources and not a `tar.gz’ like others crates. I suppose this is >> >> part of the problem, but i don’t know how to solve that. >> >> >> >> See by yourself using : >> >> >> >> guix time-machine -C channels.scm – build -f rust-deno.scm >> >> >> >> that return : >> >> >> >> error: no matching package named `deno_test_util` found >> >> >> >> **Info to reproduce** >> >> >> >> • All these package needed by Deno are defined into my-rust channel here : >> >> • The building and channel scm needed to build deno are here : >> >> • /Final Warning :/ Deno build (outside rust build) take at least 2 hours on my machine, rusty-v8 is huge and take itself ~30 to 45 minutes. >> >> >> >> Any *final* help appreciated ! >> >> >> >> Best , >> >> Sébastien Rey-Coyrehourcq >> >> >> >> >> >> Sébastien Rey-Coyrehourcq writes: >> >> >> >> > >> >> > Hi, >> >> > >> >> > Thanks a lot, that helps me to make one more step :) >> >> > >> >> > I set : >> >> > >> >> > ┌──── >> >> > │ (arguments >> >> > │ `(#:phases >> >> > │ (modify-phases %standard-phases >> >> > │ (add-before ’configure ’set-source >> >> > │ (lambda _ >> >> > │ (setenv “V8_FROM_SOURCE” “1”) >> >> > │ (setenv “RUST_BACKTRACE” “1”) >> >> > │ (setenv “CLANG_BASE_PATH” (getenv “CMAKE_PREFIX_PATH”)) >> >> > │ #t))) >> >> > └──── >> >> > >> >> > I also try C_INCLUDE_PATH >> >> > >> >> > and native input : >> >> > >> >> > ┌──── >> >> > │ (native-inputs (list ninja gn clang-toolchain ccache clang)) >> >> > └──── >> >> > >> >> > I have now another problem during compilation of v8, i don’t understand > why “clang” path is not well recognized by the buildscript, test is defined here > : >> >> > >> >> > >> >> > >> >> > The backtrace : >> >> > >> >> > error: failed to run custom build command for `v8 v0.49.0` >> >> > >> >> > Caused by: >> >> > process didn’t exit successfully: > `/tmp/guix-build-rust-deno-1.25.2.drv-0/deno-1.25.2/target/release/build/v8-bbb68ec56db1d802/build-script-build` > (exit status: 101) >> >> > — stdout >> >> > cargo:rerun-if-changed=.gn >> >> > cargo:rerun-if-changed=BUILD.gn >> >> > cargo:rerun-if-changed=src/binding.cc >> >> > cargo:rerun-if-env-changed=CCACHE >> >> > cargo:rerun-if-env-changed=CLANG_BASE_PATH >> >> > cargo:rerun-if-env-changed=DENO_TRYBUILD >> >> > cargo:rerun-if-env-changed=DOCS_RS >> >> > cargo:rerun-if-env-changed=GN >> >> > cargo:rerun-if-env-changed=GN_ARGS >> >> > cargo:rerun-if-env-changed=HOST >> >> > cargo:rerun-if-env-changed=NINJA >> >> > cargo:rerun-if-env-changed=OUT_DIR >> >> > cargo:rerun-if-env-changed=RUSTY_V8_ARCHIVE >> >> > cargo:rerun-if-env-changed=RUSTY_V8_MIRROR >> >> > cargo:rerun-if-env-changed=SCCACHE >> >> > cargo:rerun-if-env-changed=V8_FORCE_DEBUG >> >> > cargo:rerun-if-env-changed=V8_FROM_SOURCE >> >> > cargo:rustc-link-lib=static=rusty_v8 >> >> > using Chromiums clang >> >> > clang_base_path /tmp/guix-build-rust-deno-1.25.2.drv-0/deno-1.25.2/target/release/clang >> >> > >> >> > — stderr >> >> > thread ’main’ panicked at ’called `Result::unwrap()` on an `Err` value: > Os { code: 2, kind: NotFound, message: “No such file or directory” }’, > /tmp/guix-build-rust-deno-1.25.2.drv-0/deno-1.25.2/guix-vendor/rust-v8-0.49.0.tar.gz/build.rs:537:6 >> >> > stack backtrace: >> >> > 0: rust_begin_unwind >> >> > 1: core::panicking::panic_fmt >> >> > 2: core::result::unwrap_failed >> >> > 3: core::result::Result::unwrap >> >> > 4: build_script_build::clang_download >> >> > 5: build_script_build::build_v8 >> >> > 6: build_script_build::main >> >> > 7: core::ops::function::FnOnce::call_once >> >> > note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace. >> >> > >> >> > Best, >> >> > >> >> > >> >> > Wojtek Kosior writes: >> >> > >> >> >>> Hi, >> >> >>> >> >> >>> You’re both right, seems there is a flag to skip binary downloading and >> >> >>> compile the V8 lib. >> >> >>> >> >> >>> […] >> >> >> >> >> >> Good to see you found it :) >> >> >> >> >> >>> So, my packaging friend :), what’s the best way to push an “export >> >> >>> V8_FROM_SOURCE=1” or something like that into my rust-deno.scm ? >> >> >>> >> >> >>> Best, >> >> >>> >> >> >>> SR >> >> >> >> >> >> When I first read your question, I did not know the exact function. But >> >> >> I knew where to look. So I thought I’d better share my way of learning >> >> >> rather than just the solution ;) >> >> >> >> >> >> I started with >> >> >> >> >> >> grep -R ’export’ ~/.config/guix/current/share/guile/site/3.0/gnu/packages/ | less >> >> >> >> >> >> That showed a lot of results. I noticed a line like this >> >> >> >> >> >>> > /home/urz/.config/guix/current/share/guile/site/3.0/gnu/packages/engineering.scm: > (setenv “HOME” (getcwd)) ;; cmake needs this to export modules >> >> >> >> >> >> Thought it might be the thing I was looking for, so I did >> >> >> >> >> >> less /home/urz/.config/guix/current/share/guile/site/3.0/gnu/packages/engineering.scm >> >> >> >> >> >> and navigated to this line. This seems to be it. The (setenv) function. >> >> >> Can be used as from a modified packaging phase function (as you can see >> >> >> in engineering.scm). >> >> >> >> >> >> Hope I helped. Good luck once again! >> >> >> >> >> >> W. >> >> >> >> >> >> P.S. Make sure you know some ’less’ shortcuts if you’re going to do >> >> >> things this way. It cad speed things up ^^ >> >> >> >> >> >> – (sig_start) >> >> >> website: >> >> >> PGP: >> >> >> fingerprint: E972 7060 E3C5 637C 8A4F 4B42 4BC5 221C 5A79 FD1A >> >> >> >> >> >> Meet Kraków saints! #12: saint Jadwiga Andegaweńska >> >> >> Poznaj świętych krakowskich! #12: święta Jadwiga Andegaweńska >> >> >> >> >> >> – (sig_end) >> >> >> >> >> >> >> >> >> On Wed, 16 Nov 2022 21:38:47 +0100 >> >> >> Sebastien Rey-Coyrehourcq wrote: >> >> >> >> >> >>> Hi, >> >> >>> >> >> >>> You’re both right, seems there is a flag to skip binary downloading and >> >> >>> compile the V8 lib. >> >> >>> >> >> >>> From the githubpage () : “V8 is >> >> >>> very large and takes a long time to compile. Many users will prefer to >> >> >>> use a prebuilt version of V8. We publish static libs for every version >> >> >>> of rusty v8 on Github . >> >> >>> >> >> >>> Binaries builds are turned on by default: |cargo build| will initiate a >> >> >>> download from github to get the static lib. To disable this build using >> >> >>> the |V8_FROM_SOURCE| environmental variable. >> >> >>> >> >> >>> When making changes to rusty_v8 itself, it should be tested by build >> >> >>> from source. The CI always builds from source” >> >> >>> >> >> >>> So, my packaging friend :), what’s the best way to push an “export >> >> >>> V8_FROM_SOURCE=1” or something like that into my rust-deno.scm ? >> >> >>> >> >> >>> Best, >> >> >>> >> >> >>> SR >> >> >>> >> >> >>> Le 15/11/2022 à 08:58, Efraim Flashner a écrit : >> >> >>> > On Mon, Nov 14, 2022 at 11:30:47PM +0100, Sébastien Rey-Coyrehourcq wrote: >> >> >>> >> Hi, >> >> >>> >> >> >> >>> >> After some day of packaging rust crate, i progress and deno start to >> >> >>> >> compile … but after 1min i have this error when cargo start compiling >> >> >>> >> *rust-v8-0.49* . Any rust + guix help appreciated. >> >> >>> >> >> >> >>> >> I push the channel to reproduce the problem here : >> >> >>> >> >> >> >>> >> The rust scm repo : git.sr.ht:~reyman/rust-channel >> >> >>> >> Channel info to put into *channels.scm* : >> >> >>> >> The *rust-deno.scm* file to build : >> >> >>> >> The command : guix time-machine -C channels.scm – build -f rust-deno.scm >> >> >>> >> >> >> >>> >> And the rust error : >> >> >>> >> >> >> >>> >> — >> >> >>> >> >> >> >>> >> error: failed to run custom build command for `v8 v0.49.0` >> >> >>> >> >> >> >>> >> Caused by: >> >> >>> >> process didn’t exit successfully: > `/tmp/guix-build-rust-deno-1.25.2.drv-0/deno-1.25.2/target/release/build/v8-bbb68ec56db1d802/build-script-build` > (exit status: 101) >> >> >>> >> — stdout >> >> >>> >> cargo:rerun-if-changed=.gn >> >> >>> >> cargo:rerun-if-changed=BUILD.gn >> >> >>> >> cargo:rerun-if-changed=src/binding.cc >> >> >>> >> cargo:rerun-if-env-changed=CCACHE >> >> >>> >> cargo:rerun-if-env-changed=CLANG_BASE_PATH >> >> >>> >> cargo:rerun-if-env-changed=DENO_TRYBUILD >> >> >>> >> cargo:rerun-if-env-changed=DOCS_RS >> >> >>> >> cargo:rerun-if-env-changed=GN >> >> >>> >> cargo:rerun-if-env-changed=GN_ARGS >> >> >>> >> cargo:rerun-if-env-changed=HOST >> >> >>> >> cargo:rerun-if-env-changed=NINJA >> >> >>> >> cargo:rerun-if-env-changed=OUT_DIR >> >> >>> >> cargo:rerun-if-env-changed=RUSTY_V8_ARCHIVE >> >> >>> >> cargo:rerun-if-env-changed=RUSTY_V8_MIRROR >> >> >>> >> cargo:rerun-if-env-changed=SCCACHE >> >> >>> >> cargo:rerun-if-env-changed=V8_FORCE_DEBUG >> >> >>> >> cargo:rerun-if-env-changed=V8_FROM_SOURCE >> >> >>> >> cargo:rustc-link-lib=static=rusty_v8 >> >> >>> >> download lockfile: “/tmp/guix-build-rust-deno-1.25.2.drv-0/deno-1.25.2/target/release/build/lib_download.fslock” >> >> >>> >> static lib URL: >> >> >>> >> cargo:rustc-link-search=/tmp/guix-build-rust-deno-1.25.2.drv-0/deno-1.25.2/target/release/gn_out/obj >> >> >>> >> Downloading >> >> >>> >> Python downloader failed, trying with curl. >> >> >>> > Looks like you need to patch rust-v8-0.49 to not try to download >> >> >>> > librusty_v8_release… but instead you’ll have to build it from source >> >> >>> > and let it know where to find it. >> >> >>> > >> >> >>> >> — stderr >> >> >>> >> thread ’main’ panicked at ’called `Result::unwrap()` on an `Err` value: >> >> >>> >> Os { code: 2, kind: NotFound, message: “No such file or directory” }’, >> >> >>> >> /tmp/guix-build-rust-deno-1.25.2.drv-0/deno-1.25.2/guix-vendor/rust-v8-0.49.0.tar.gz/build.rs:405:10 >> >> >>> >> note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace >> >> >>> >> warning: build failed, waiting for other jobs to finish… >> >> >>> >> error: build failed >> >> >>> >> error: in phase ’build’: uncaught exception: >> >> >>> >> %exception #<&invoke-error program: “cargo” arguments: (“build” “–release”) exit-status: 101 term-signal: #f stop-signal: #f> >> >> >>> >> phase `build’ failed after 105.5 seconds >> >> >>> >> command “cargo” “build” “–release” failed with status 101 >> >> >>> >> builder for `/gnu/store/g4m5c558l1q4g1kggzg2v9vkw352nnaj-rust-deno-1.25.2.drv’ failed with exit code 1 >> >> >>> >> la compilation de /gnu/store/g4m5c558l1q4g1kggzg2v9vkw352nnaj-rust-deno-1.25.2.drv a échoué >> >> >>> >> Vous trouverez le journal de compilation dans « /var/log/guix/drvs/g4/m5c558l1q4g1kggzg2v9vkw352nnaj-rust-deno-1.25.2.drv.gz ». >> >> >>> >> guix build: erreur : build of `/gnu/store/g4m5c558l1q4g1kggzg2v9vkw352nnaj-rust-deno-1.25.2.drv’ failed >> >> >>> > >> >> >> >> >> >> >> >> > >> > >> > > >