From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp1.migadu.com ([2001:41d0:303:e224::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms13.migadu.com with LMTPS id aCcTCJ9maGfOFwEA62LTzQ:P1 (envelope-from ) for ; Sun, 22 Dec 2024 19:21:03 +0000 Received: from aspmx1.migadu.com ([2001:41d0:303:e224::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp1.migadu.com with LMTPS id aCcTCJ9maGfOFwEA62LTzQ (envelope-from ) for ; Sun, 22 Dec 2024 20:21:03 +0100 X-Envelope-To: larch@yhetil.org Authentication-Results: aspmx1.migadu.com; dkim=pass header.d=efficios.com header.s=smtpout1 header.b=hQilpmsP; spf=pass (aspmx1.migadu.com: domain of "guix-devel-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-devel-bounces+larch=yhetil.org@gnu.org"; dmarc=pass (policy=none) header.from=efficios.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1734895263; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post:dkim-signature; bh=L9r9N3f2ehKOqMp9NAPDDvJRW/LnxN/NDPjqwqeaOU4=; b=A6JAhtgZ+tSdTHjVQnWeIk0O9e3IC7XSKvFgxeiOHa6LIh1yc7PRya1mFbbt/sbMwe7lQl UUv1t4lC0mID3wn/paCT5d2QfXGYY5Tb2MeVlzhALugq4jNTsPnOC86QQdkyYFil8vNrD4 bjx0uqHLO78cZrodpXll1duuqxjQv4swDEF/Wk5CmkJwnfNzQXqz2u5rEPX/kQpEZGArcJ NxJgvC7onciV4ECEOs8orgwqEXLfchU4/U1K3OjgSwEsSNCWtZcFcwxwojWwK+dGvuEirJ 4t0jCvtvzoDSngsXkxtPwi6swmVE/eiWhZXvXIAGV1YSp1s+2f+dWDx1e7O8kw== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=pass header.d=efficios.com header.s=smtpout1 header.b=hQilpmsP; spf=pass (aspmx1.migadu.com: domain of "guix-devel-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-devel-bounces+larch=yhetil.org@gnu.org"; dmarc=pass (policy=none) header.from=efficios.com ARC-Seal: i=1; s=key1; d=yhetil.org; t=1734895263; a=rsa-sha256; cv=none; b=YFvwPuMjPWbKReU470mz/C7Wg3eK9xGGWyTyXFhxr+DSnxV3T8LrcjeCQ4vDCwlNvhpI1T jL1hPaR6hM2/AMieQ8M+337TB8b0Z5JXpMXfP2P64bjHaAFsEsyTbHt2j7biivlheP4LbQ 7KVnb4oF8oM4VKf/MJVn4vqt24W30B/6dWVfVSkDUMH3fY1bKhxlO2nebRja3c9g5Srn2d eXIpPhLaABLB325SOdVfcQcvj0OB8NUsCAgyNK16I8/SoAi/jpF9kUbD10Ovu2wGIIDHUv NO3/tGlMCnVRaga276bfJhhO2MPieNT+4Kbqj78SdEvyZR6y3Hf4f7AvjDpFQg== Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id DB76E33DA5 for ; Sun, 22 Dec 2024 20:21:02 +0100 (CET) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tPRV9-0004DQ-Q2; Sun, 22 Dec 2024 14:20:31 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tOO9C-0004XY-RH for guix-devel@gnu.org; Thu, 19 Dec 2024 16:33:30 -0500 Received: from smtpout.efficios.com ([2607:5300:400:ed00::31e5]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tOO9A-00027U-4r for guix-devel@gnu.org; Thu, 19 Dec 2024 16:33:30 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=efficios.com; s=smtpout1; t=1734644006; bh=Jr1hpeTVyKPTp1XZJCW2u0mV/3ioqNAhoZd4KlHj96o=; h=From:To:Cc:Subject:In-Reply-To:References:Date:From; b=hQilpmsPsm7JnxqVmTWodcFsyvS7/WNK/aBYf6gNdDEwGlspV+l7kABlwpWsD4DIf rmdKUeRt1FB+3W+pz7uLMwWFlJAW6/pHXZeHWYPi5TXJxZ55TsUEiStH69BmKjYVxl nepMkq/B4mIaPlQ8GEgp1g9AFdsLYrH8gT6+qxvGROuvsg2mdfuvzJZWyomv5AcSfj XtT6LaiZXfoT68G9isg5xsJvwCIbIxM108YO39/YttSJBtdGsQ3AFHPNbxTSeIn6FB NBEQJ5tfDLpE4eKIuDWF1rOU1O4gsNRPkgc9+6Heih3jkpMgj3zpzyZUo5EpFksaGM LqT1zjazLYOzg== Received: from localhost (157-208-8-209.mc.derytele.com [157.208.8.209]) by smtpout.efficios.com (Postfix) with ESMTPSA id 4YDkLZ1rRXz10TQ; Thu, 19 Dec 2024 16:33:26 -0500 (EST) From: Olivier Dion To: Divya Ranjan , guix-devel@gnu.org Cc: pjotr.public12@thebird.nl Subject: Re: On a Guile-based Build-Tool complimenting Guix In-Reply-To: <87wmfvo2je.fsf@subvertising.org> Organization: EfficiOS References: <87wmfvo2je.fsf@subvertising.org> Date: Thu, 19 Dec 2024 16:33:25 -0500 Message-ID: <87h66zicii.fsf@laura> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Received-SPF: pass client-ip=2607:5300:400:ed00::31e5; envelope-from=odion@efficios.com; helo=smtpout.efficios.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-Mailman-Approved-At: Sun, 22 Dec 2024 14:20:29 -0500 X-BeenThere: guix-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+larch=yhetil.org@gnu.org Sender: guix-devel-bounces+larch=yhetil.org@gnu.org X-Migadu-Flow: FLOW_IN X-Migadu-Country: US X-Migadu-Scanner: mx12.migadu.com X-Migadu-Spam-Score: -2.12 X-Spam-Score: -2.12 X-Migadu-Queue-Id: DB76E33DA5 X-TUID: 5Oxzda/4wEg3 Hi, For what it is worth, I have been using my own build system purely in Guile for a few years now. I use it in my main project libpatch (https://git.sr.ht/~old/libpatch/tree). It is compatible with the gnu-build-system (configure && make && make check && make install), but it does not have to if you want to diverge from this flow. However, it is order of magnitude faster than the autotools stuff. Configuration for example is a single Guile instance that may do some fork, but not that much. Building is also fast, althought it would be optimal if fibers were available (I use par-map but the bottleneck is due to waitpid). I have plan to add guile-parallel to it instead since it is also purely Guile based. My project is a mix of C, C++ and Guile. Thus I've only added support for these, but I it is possible to extend it since it is base on GOOPS. It is tested on Ubuntu, Debian, Fedora, Arch Linux and Guix. Here are some snippets of what it looks like: project/libpatch/build/libraries.scm (C libraries): --8<---------------cut here---------------start------------->8--- (define-module (project libpatch build libraries) #:use-module ((config) #:prefix conf:) #:use-module (ice-9 match) #:use-module (project build c) #:use-module (project utils list) #:use-module (srfi srfi-26) #:export (libpatch libpatch-ftrace)) (define libpatch-common (c-binary (inputs '("src/lib/libpatch-common/io.c" "src/lib/libpatch-common/read.c" "src/lib/libpatch-common/sleep.c" "src/lib/libpatch-common/write.c")) (cppflags (list (string-append "-I" conf:srcdir "/src/lib/libpatch-common"))) (library? #t) (shared? #f) (output "src/lib/libpatch-common.a"))) (define libpatch (c-binary (inputs (cons libpatch-common (cond-list '("src/lib/libpatch/core/address.c" "src/lib/libpatch/core/attr.c" "src/lib/libpatch/core/canon.c" "src/lib/libpatch/core/commit.c" "src/lib/libpatch/core/configure.c" "src/lib/libpatch/core/const.c" "src/lib/libpatch/core/coverage.c" "src/lib/libpatch/core/disable.c" "src/lib/libpatch/core/disasm.c" "src/lib/libpatch/core/drop.c" "src/lib/libpatch/core/dump.c" "src/lib/libpatch/core/dwarf.c" "src/lib/libpatch/core/dynamic.c" "src/lib/libpatch/core/enable.c" "src/lib/libpatch/core/fini.c" "src/lib/libpatch/core/gc.c" "src/lib/libpatch/core/global.c" "src/lib/libpatch/core/init.c" "src/lib/libpatch/core/integrity.c" "src/lib/libpatch/core/last-error.c" "src/lib/libpatch/core/make.c" "src/lib/libpatch/core/olx.c" "src/lib/libpatch/core/panic.c" "src/lib/libpatch/core/patch.c" "src/lib/libpatch/core/resolve.c" "src/lib/libpatch/core/states.c" "src/lib/libpatch/core/thread.c" "src/lib/libpatch/core/tls.c" "src/lib/libpatch/core/trampoline.c" "src/lib/libpatch/core/wxe.c" "src/lib/libpatch/core/page-allocator.c") ((string=3D? conf:arch "x86_64") "src/lib/libpatch/x86_64/const.c" "src/lib/libpatch/x86_64/disasm.c" "src/lib/libpatch/x86_64/dwarf.c" "src/lib/libpatch/x86_64/dynamic.c" "src/lib/libpatch/x86_64/generic-handlers.S" "src/lib/libpatch/x86_64/init.c" "src/lib/libpatch/x86_64/patch.c" "src/lib/libpatch/x86_64/ret-handlers.S")))) (external-dependencies (list conf:capstone conf:libdw conf:libolx conf:liburcu-bp)) (cppflags (list (string-append "-I" conf:srcdir "/src/lib/libpatch") (string-append "-I" conf:srcdir "/src/lib/libpatch-common") (string-append "-I" conf:srcdir "/src/lib/libpatch/" conf:arc= h "/include") "-DLIBPATCH_CORE")) (cflags '("-fvisibility=3Dhidden")) (libs '("-pthread")) (shared? #t) (library? #t) (output "src/lib/libpatch.so") (install-location conf:libdir))) (define libpatch-ftrace (c-binary (enabled? conf:build-libpatch-ftrace) (inputs (list libpatch "src/lib/libpatch-ftrace/libpatch-ftrace.c")) (external-dependencies (list conf:lttng-ust)) (cppflags (list (string-append "-I" conf:srcdir "/src/lib/libpatch-ftrace"))) (cflags (list "-fvisibility=3Dhidden")) (shared? #t) (library? #t) (output "src/lib/libpatch-ftrace.so") (install-location conf:libdir))) --8<---------------cut here---------------end--------------->8--- project/libpatch/build/bindings.scm (Guile bindings to Libpatch): --8<---------------cut here---------------start------------->8--- (define-module (project libpatch build bindings) #:use-module ((config) #:prefix conf:) #:use-module (project build) #:use-module (project build guile) #:use-module (project libpatch build libraries) #:export (bindings)) (define guile-go-dir (string-append conf:libdir "/guile/3.0/site-ccache/libpatch")) (define guile-sources '("src/guile/libpatch/ffi.scm" "src/guile/libpatch/patch.scm")) (define guile-modules (map (lambda (source) (guile-module (inputs (list source libpatch)) (load-paths (list (string-append (builddir) "/src/guile") (string-append (srcdir) "/src/guile"))) (install-location guile-go-dir))) guile-sources)) (define bindings guile-modules) --8<---------------cut here---------------end--------------->8--- configure (compatible with configure.ac): --8<---------------cut here---------------start------------->8--- #!/bin/sh #-*-Scheme-*- GUILE=3D"$(command -v guile || command -v guile3.0)" if [ -z "$GUILE" ]; then echo "Missing guile executable." exit 1 fi DIR=3D$(dirname "$0") THIS_PATH=3D$(realpath "$DIR") exec $GUILE -L "$THIS_PATH" --no-auto-compile -e main -s "$0" "$@" !# (use-modules (ice-9 exceptions) (ice-9 match) (project configure) (project configure config-file) (project configure guess) (project configure option) (project configure package) (project configure template-file) (project configure toolchain) (project configure variable) (project file-system search) (project progress) (project utils colors) (srfi srfi-1) (srfi srfi-26) (system base target)) (define (make-simple-action name value) (lambda* (this-opt _ #:key variables #:allow-other-keys) (let ((var (assoc-ref variables name))) (if var (set-variable-value! var value) (raise-exception (make-exception (make-programming-error) (make-exception-with-irritants name) (make-exception-with-message "Unknown variable"))))))) (define dyninst-allowed? #t) (define liteinst-allowed? #t) (define lttng-allowed? #t) (define* (has-dyninst? #:key packages #:allow-other-keys) (and dyninst-allowed? (package-exists? (assoc-ref packages "dyninst")))) (define* (has-liteinst? #:key packages #:allow-other-keys) (and liteinst-allowed? (package-exists? (assoc-ref packages "liteinst")))) (define* (has-lttng-ust? #:key packages #:allow-other-keys) (and lttng-allowed? (package-exists? (assoc-ref packages "lttng-ust")))) (define (support-flags? . flags) (lambda* (#:key variables #:allow-other-keys) (apply toolchain-support-flags? (variable-value (assoc-ref variables "CC")) flags))) (define libpatch-configuration (configuration (unique-file "dev-env") (arguments (cdr (program-arguments))) (packages (list (package (name "capstone")) (package (name "libdw") (minimum-version "0.158")) (package (name "libolx")) (package (name "liburcu-bp")) (package (name "lttng-ust") (required? #f)) (package (name "dyninst") (resolver (lambda (this) (let* ((lib (find-library "libdyninstAPI.so")) (header (find-header "BPatch.h")) (exists? (and lib header #t))) (set! (package-exists? this) exists?) (when exists? (set! (package-libs-only-l this) "-ldyninstAPI") (set! (package-libs-only-L this) (string-append "-L" (dirname lib))) (set! (package-cflags-only-I this) (string-append "-I" (dirname header))))))) (required? #f)) (package (name "liteinst") (resolver (lambda (this) (let* ((lib (find-library "libliteinst.so")) (header (find-header "liteinst.hpp")) (exists? (and lib header #t))) (set! (package-exists? this) exists?) (when exists? (set! (package-libs-only-l this) "-lliteinst") (set! (package-libs-only-L this) (string-append "-L" (dirname lib))) (set! (package-cflags-only-I this) (string-append "-I" (dirname header))))))) (required? #f)))) (variables (append (list (variable (name "LIBPATCH_VERSION") (value "1.0.0")) (variable (name "GUILE_BIN") (value (delay (or (find-binary/progress "guile") (find-binary/progress "guile3.0") (push-error! (make-exception (make-external-error) (make-exception-with-irritants "guile") (make-exception-with-message "missing required binary"))))))) (variable (name "GUILD_BIN") (value (delay (or (find-binary/progress "guild") (find-binary/progress "guild3.0") (push-error! (make-exception (make-external-error) (make-exception-with-irritants "guild") (make-exception-with-message "missing required binary"))))))) (variable (name "CC") (value (delay (guess-c-toolchain)))) (variable (name "CXX") (value (delay (guess-c++-toolchain #:required? #f)))) (variable (name "arch") (value (delay (target-cpu)))) (variable (name "build-benchmarks") (value #t)) (variable (name "tsan-native-benchmarks") (value #f)) (variable (name "build-libpatch-ftrace") (value has-lttng-ust?)) (variable (name "build-libpatch-coverage") (value #t)) (variable (name "build-libpatch-integrity") (value #t)) (variable (name "build-manpages") (value (delay (and (find-binary/progress "emacs") #t)))) (variable (name "has-man") (value (delay (and (find-binary/progress "man") #t)))) (variable (name "manpages-source-highlight") (value (delay (and (find-binary/progress "source-highlight") #t)))) (variable (name "has-dyninst") (value has-dyninst?)) (variable (name "has-gdb-inproctrace") (value (delay (and (find-binary/progress "gdb") (find-library "libinproctrace.so"))))) (variable (name "has-liteinst") (value has-liteinst?)) (variable (name "has-csmith") (value (delay (and (find-binary/progress "csmith") #t)))) (variable (name "csmith-include-directory") (value (delay (and=3D> (find-header "csmith/csmith.h") dirname)))) (variable (name "environ") (value (delay (environ))))) %standard-directory-variables)) (options (append (list (option (switch "disable-ftrace-build") (synopsis "do no build libpatch-ftrace.so") (action (make-simple-action "build-libpatch-ftrace" #f))) (option (switch "disable-patch-coverage-build") (synopsis "do not build libpatch-coverage.so") (action (make-simple-action "build-libpatch-coverage" #f))) (option (switch "disable-patch-integrity-build") (synopsis "do not build libpatch-integrity.so") (action (make-simple-action "build-libpatch-integrity" #f))) (option (switch "enable-tsan-on-native-benchmarks") (synopsis "enable TSAN for native benchmarks") (action (make-simple-action "tsan-native-benchmarks" #t))) (option (switch "without-manpages") (synopsis "disable manpages generation") (action (make-simple-action "build-manpages" #f))) (option (switch "without-lttng") (synopsis "disable any usage of LTTng") (action (lambda _ (set! lttng-allowed? #f)))) (option (switch "without-dyninst") (synopsis "disable any usage of Dyninst for benchmarking") (action (lambda _ (set! dyninst-allowed? #f)))) (option (switch "without-liteinst") (synopsis "disable any usage of LiteInst for benchmarking") (action (lambda _ (set! liteinst-allowed? #f)))) (option (switch "without-benchmarks") (synopsis "do not compile benchmarks") (action (make-simple-action "build-benchmarks" #f)))) %standard-directory-options)) (templates (list (template-file (input "aux/Makefile.in") (output "Makefile")) (template-file (input "aux/project-repl.in") (output "project-repl") (mode #o744)) (template-file (input "src/lib/pkgconfig/libpatch.pc.in")) (template-file (input "src/bin/patch-ftrace.in") (mode #o744)) (template-file (input "src/bin/patch-coverage.in") (mode #o744)) (template-file (input "src/bin/patch-integrity.in") (mode #o744)) (template-file (input "scripts/debug-test.in") (mode #o744)))) (config-files (list (c-config-file (path ".config.h") (namespace "CONFIG_")) (scm-config-file (path "config.scm") (namespace "config")))) (status-file "config.status"))) (define (main args) (with-exception-handler (lambda (exn) (if (eq? 'quit (exception-kind exn)) (raise-exception exn) (begin (print-exception (current-error-port) #f (exception-kind exn) (exception-args exn)) (primitive-exit EXIT_FAILURE)))) (lambda () (call-with-values (lambda () (configure libpatch-configuration)) (lambda (variables _ errors) (with-nest-progress "features summary" (for-each (match-lambda ((feature-name . feature-description) (let ((name+feature (assoc feature-name variables))) (if name+feature (progress "~a ~a" (if (cdr name+feature) (colorize-string "OK" %enabled-color) (colorize-string "NO" %disabled-color= )) feature-description) (raise-exception (make-exception (make-exception-with-origin "Features summary") (make-programming-error) (make-exception-with-irritants feature-name) (make-exception-with-message "Unknown feature"))))))) '(("build-benchmarks" . "compile benchmarks") ("build-libpatch-ftrace" . "compile libpatch-ftace.so") ("build-libpatch-coverage" . "compile libpatch-coverage.so= ") ("build-libpatch-integrity" . "compile libpatch-integrity.= so") ("build-manpages" . "generate man pages")))) (exit (if (null? errors) EXIT_SUCCESS EXIT_FAILURE))))))) --8<---------------cut here---------------end--------------->8--- On Thu, 19 Dec 2024, Divya Ranjan wrote: > Hello Guix, > > The other day, after being frustrated of build systems (auto-tools, meson= , maven etc.), I wondered why doesn=E2=80=99t Guix which has such powerful = tools within it (`guix build`, `guix pack` etc.) also not have a purely Gui= le-based build tool? After all, our goal is to make deployment, and buildin= g both more declarative and away from the all-too-familiar =E2=80=9Cdepende= ncy hell=E2=80=9D. > > I am not exactly sure how I want to go with this, but I want to extend (i= f needed) the capabilities of Guix, to allow the developer of a package to = use it also to build the package effectively replacing existing build tools= . Since we already have build-system, instead of executing make (or whateve= r other tool) commands from it, we could modify it to itself have all those= things that a Makefile would have. > > The developer would use Guile to declare their build config, I am again n= ot sure what this might exactly look like, but can=E2=80=99t we have it suc= h that we provide the developer with some tools to _declare_ a custom and p= ackage-specific build-system in Guile (just like our familiar gnu-build-sys= tem), but this is purely in Guile and executes whatever commands, path decl= arations and other interactions (such as calling gcc) directly from Guile a= nd not by just calling `make`. I haven=E2=80=99t thought through this clear= ly, but even if this doesn=E2=80=99t work, the main idea I=E2=80=99d like t= o propose is to fully replace existing build-tools by making a new Guile bu= ild tool to work alongside Guix. > > Ideally, once the developer has a build config ready, one can just wrap i= t up with a package definition in Guile, just like the ones we create to pa= ckage something upstream. This package definition can then be used in `guix= build -f package.scm` to result in a fully transactional building process = that focuses not on getting out of dependency hell, but just declaring your= config right. And all of this without having to learn yet another build to= ol that might disappear, and of course, without leaving the comfortable wor= ld of Lisp (Scheme). > > I was indicated by others that such an idea has previously been conceievd= among Guix developers themselves, namely as a GSoC proposal[0]. I couldn= =E2=80=99t find if that has progressed towards anything, nor could find any= thing in the mailing list. I did see Pjotr volunteering to mentor for it, I= =E2=80=99ve CC-ed them to see if they=E2=80=99re still interested in such a= project. Meanwhile, I=E2=80=99d like input from other Guix core developers= on what they think of this, and if they could provide me with some leads o= n where to go with this. > > > [0]: https://libreplanet.org/wiki/Group:Guix/GSoC-2024 > > Regards, > --=20 > Divya Ranjan, > Philosophy, Mathematics, Libre Software. > --=20 Olivier Dion EfficiOS Inc. https://www.efficios.com