From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp1 ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms11 with LMTPS id YMfeF9gh/F5PTgAA0tVLHw (envelope-from ) for ; Wed, 01 Jul 2020 05:40:40 +0000 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp1 with LMTPS id EIyME9gh/F4BLQAAbx9fmQ (envelope-from ) for ; Wed, 01 Jul 2020 05:40:40 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 321319403C9 for ; Wed, 1 Jul 2020 05:40:39 +0000 (UTC) Received: from localhost ([::1]:56838 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jqVU4-0003CE-Vz for larch@yhetil.org; Wed, 01 Jul 2020 01:40:37 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:32884) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jqVTs-0003C5-AA for help-guix@gnu.org; Wed, 01 Jul 2020 01:40:24 -0400 Received: from mail-pg1-x530.google.com ([2607:f8b0:4864:20::530]:41550) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jqVTo-0003eu-BB for help-guix@gnu.org; Wed, 01 Jul 2020 01:40:23 -0400 Received: by mail-pg1-x530.google.com with SMTP id g67so10241399pgc.8 for ; Tue, 30 Jun 2020 22:40:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=asu-edu.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:references:date:in-reply-to:message-id :user-agent:mime-version; bh=4OdZ4gkmYd5zWf1RZh1GLEZh1PMFIAKXEIdRLPuUMxA=; b=Sruays0LNkPdF+E6+JrW2vUI4IaIcZAc34lEnLfJ/Ohg8aYtfGgdLxZAySw3cjLiVI F9IqpAR9K2LjIOyx/3LN4+qUJc2nrPaZnV9lha5QUAjRc0XXm6rxOagbcx+jxheN+ssL wrLCmp9XqBg/zVjVrFH3qJHNEsM6YgWsm8IMWzVy1+p4O28hRCKCgfQs9Zc/msdr871H v/hb7vH0nR3KgdE6K7eh0QQGsaF9MCcw8dN511hrqLaIyIiULrb6C0wXo97HR1OPW9js UXWKH6R807hYqFegMABi87fKPSt6Ti49Y3ElY8v/Fj5w2dhGBzNWETByZj+SlOadO4A1 9vuw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:references:date:in-reply-to :message-id:user-agent:mime-version; bh=4OdZ4gkmYd5zWf1RZh1GLEZh1PMFIAKXEIdRLPuUMxA=; b=RIyV6O+YOvf9yNQsymC11Xwss4pQh1Ovexd8ks0p94dO+LvFzkf8DMfnFqbC2x8+BO vCKDr5ru20S0f4c82AGZMopbQBu6WPnzvu6bxxV39mpc6Mk60j5/Aw5u0s3d8OX7vm+V iawxWNgHTRFN+PxirY5LNUafWBup+fs2HM1Rp9qfurHXLcTOuvS3sg4WP0+T5ZoPJweL /gC6h1qmlFeyA/MpsbxhcIC456KJRDgPUnMZwslEVaYCN33+s+Jt0+QGSI4mJ0V7k/DU C4S9ZAhK8OYgayfIZc53KziROFBW72n7L69f0ZF2Uz+BIgx3cChmZUfO8ZU5IKlcQ72Z H++w== X-Gm-Message-State: AOAM530fxjH8E58pCSga0RDo8J6cEGlXIRzFE30GA39JsVjkB2OMclI4 QkMx/pSTPbgCPuF3QU8gUILbRskDLTQ= X-Google-Smtp-Source: ABdhPJxlUKDdp9riVVucowRmTuI2mIoqEn/HDGbxGH8DEtLfM7yQiMhc6Tc+mIviJ82YyuCo+C+Qkw== X-Received: by 2002:a63:417:: with SMTP id 23mr17321936pge.44.1593582016409; Tue, 30 Jun 2020 22:40:16 -0700 (PDT) Received: from ecenter ([2600:1700:83b0:8bd0::59c]) by smtp.gmail.com with ESMTPSA id w15sm3920566pjb.44.2020.06.30.22.40.15 (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 30 Jun 2020 22:40:15 -0700 (PDT) From: John Soo To: Mathieu Othacehe Subject: Re: BPF in linux-libre References: <44046537-51AD-4BD7-8442-41738C4D8C98@asu.edu> <87lfkqknla.fsf@gnu.org> <87h7vdd6ms.fsf@asu.edu> <87wo46uhdr.fsf@gnu.org> <87zh8wph7h.fsf@asu.edu> <87wo3uxfrq.fsf@gnu.org> Date: Tue, 30 Jun 2020 22:40:11 -0700 In-Reply-To: <87wo3uxfrq.fsf@gnu.org> (Mathieu Othacehe's message of "Fri, 26 Jun 2020 12:50:17 +0200") Message-ID: <87h7ur3i9w.fsf@asu.edu> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.3 (gnu/linux) MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" Received-SPF: pass client-ip=2607:f8b0:4864:20::530; envelope-from=jsoo1@asu.edu; helo=mail-pg1-x530.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: help-guix@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Help Guix Errors-To: help-guix-bounces+larch=yhetil.org@gnu.org Sender: "Help-Guix" X-Scanner: scn0 Authentication-Results: aspmx1.migadu.com; dkim=pass header.d=asu-edu.20150623.gappssmtp.com header.s=20150623 header.b=Sruays0L; dmarc=fail reason="SPF not aligned (relaxed), DKIM not aligned (relaxed)" header.from=asu.edu (policy=none); spf=pass (aspmx1.migadu.com: domain of help-guix-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=help-guix-bounces@gnu.org X-Spam-Score: -1.11 X-TUID: MXJY9v7QwV7j --=-=-= Content-Type: text/plain Hi Mathieu and Guix, I forgot to mention that I do have an updated patchset where libbpf is found. I've attached my work. This is I think closer to the desired effect but now there is a compile error I do not understand. I hope you can get further than I did. - John --=-=-= Content-Type: text/x-patch; charset=utf-8 Content-Disposition: attachment; filename=0001-gnu-Add-libbpf.patch Content-Transfer-Encoding: quoted-printable >From 6560dc2a5eedb3040bdd5fba8d8b6950b7a2b6d1 Mon Sep 17 00:00:00 2001 From: John Soo Date: Sat, 13 Jun 2020 14:53:50 -0700 Subject: [PATCH 1/5] gnu: Add libbpf. * gnu/packages/linux.scm (libbpf): New variable. --- gnu/packages/linux.scm | 46 ++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 46 insertions(+) diff --git a/gnu/packages/linux.scm b/gnu/packages/linux.scm index cadfd186a1..c0ac8f24bc 100644 --- a/gnu/packages/linux.scm +++ b/gnu/packages/linux.scm @@ -45,6 +45,7 @@ ;;; Copyright =C2=A9 2020 Jan (janneke) Nieuwenhuizen ;;; Copyright =C2=A9 2020 Brice Waegeneire ;;; Copyright =C2=A9 2020 Morgan Smith +;;; Copyright =C2=A9 2020 John Soo ;;; ;;; This file is part of GNU Guix. ;;; @@ -7139,3 +7140,48 @@ cache data store that is used by network file system= s such as @code{AFS} and @code{NFS} to cache data locally on disk. The content of the cache is persistent over reboots.") (license license:gpl2+))) + +(define-public libbpf + (let* ((commit "6a1615c263b679c17ecb292fa897f159e826dc10")) + (package + (name "libbpf") + (version "0.0.8") + (source + (origin + (method git-fetch) + (uri (git-reference + (url "https://github.com/libbpf/libbpf") + (commit commit))) + (sha256 + (base32 + "02vbpg9v5sjcw7ihximy63cjmz82q5izkp91i44m1qp6qj5qn4sr")))) + (build-system gnu-build-system) + (inputs + `(("libelf" ,libelf) + ("pkg-config" ,pkg-config) + ("zlib" ,zlib))) + (arguments + `(#:tests? #f ; No tests + #:make-flags + (list + (string-append "PREFIX=3D''") + (string-append "DESTDIR=3D" (assoc-ref %outputs "out")) + (string-append + "CC=3D" (assoc-ref %build-inputs "gcc") "/bin/gcc")) + #:phases + (modify-phases %standard-phases + (delete 'configure) + (add-before 'build 'pre-build + (lambda* (#:key inputs #:allow-other-keys) + (substitute* "scripts/check-reallocarray.sh" + (("/bin/rm" rm) + (string-append (assoc-ref inputs "coreutils") rm))) + (chdir "src") + #t))))) + (home-page "https://github.com/libbpf/libbpf") + (synopsis "BPF CO-RE (Compile Once =E2=80=93 Run Everywhere)") + (description + "Libbpf supports building BPF CO-RE-enabled applications, which, in +contrast to BCC, do not require Clang/LLVM runtime being deployed to target +servers and does not rely on kernel-devel headers being available.") + (license `(,license:lgpl2.1 ,license:bsd-2))))) --=20 2.26.2 --=-=-= Content-Type: text/x-patch Content-Disposition: attachment; filename=0002-gnu-Add-bpf-extra-linux-options.patch >From 124267a629c30d0acb5e5c931772cc8d0f3c43ac Mon Sep 17 00:00:00 2001 From: John Soo Date: Sat, 13 Jun 2020 22:52:55 -0700 Subject: [PATCH 2/5] gnu: Add %bpf-extra-linux-options. * gnu/packages/linux (%bpf-extra-linux-options): New variable. --- gnu/packages/linux.scm | 16 ++++++++++++++++ 1 file changed, 16 insertions(+) diff --git a/gnu/packages/linux.scm b/gnu/packages/linux.scm index c0ac8f24bc..4daa371c6b 100644 --- a/gnu/packages/linux.scm +++ b/gnu/packages/linux.scm @@ -619,6 +619,22 @@ for ARCH and optionally VARIANT, or #f if there is no such configuration." ("CONFIG_CIFS" . m) ("CONFIG_9P_FS" . m))) +;; See https://github.com/iovisor/bcc/blob/master/INSTALL.md#kernel-configuration +(define %bpf-extra-linux-options + `(("CONFIG_BPF" . #t) + ("CONFIG_BPF_SYSCALL" . #t) + ;; optional, for tc filters + ("CONFIG_NET_CLS_BPF" . m) + ;; optional, for tc actions + ("CONFIG_NET_ACT_BPF" . m) + ("CONFIG_BPF_JIT" . #t) + ;; for Linux kernel versions 4.1 through 4.6 + ;; ("CONFIG_HAVE_BPF_JIT" . y) + ;; for Linux kernel versions 4.7 and later + ("CONFIG_HAVE_EBPF_JIT" . #t) + ;; optional, for kprobes + ("CONFIG_BPF_EVENTS" . #t))) + (define (config->string options) (string-join (map (match-lambda ((option . 'm) -- 2.26.2 --=-=-= Content-Type: text/x-patch Content-Disposition: attachment; filename=0003-gnu-Add-linux-libre-with-bpf.patch >From 88c88d787a399069f0554c8635f1f3496a9fd1e2 Mon Sep 17 00:00:00 2001 From: John Soo Date: Sat, 13 Jun 2020 22:54:24 -0700 Subject: [PATCH 3/5] gnu: Add linux-libre-with-bpf. * gnu/packages/linux.scm (linux-libre-with-bpf): New variable. --- gnu/packages/linux.scm | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/gnu/packages/linux.scm b/gnu/packages/linux.scm index 4daa371c6b..f3eec978ec 100644 --- a/gnu/packages/linux.scm +++ b/gnu/packages/linux.scm @@ -7157,6 +7157,16 @@ cache data store that is used by network file systems such as @code{AFS} and persistent over reboots.") (license license:gpl2+))) +(define-public linux-libre-with-bpf + (make-linux-libre* linux-libre-5.4-version + linux-libre-5.4-source + '("x86_64-linux" "i686-linux" "armhf-linux" "aarch64-linux" "riscv64-linux") + #:extra-version "bpf" + #:configuration-file kernel-config + #:extra-options + (append %bpf-extra-linux-options + %default-extra-linux-options))) + (define-public libbpf (let* ((commit "6a1615c263b679c17ecb292fa897f159e826dc10")) (package -- 2.26.2 --=-=-= Content-Type: text/x-patch Content-Disposition: attachment; filename=0004-gnu-Add-bcc.patch >From 0358dccc866a392512ddb15f9fee0e24e6bc837c Mon Sep 17 00:00:00 2001 From: John Soo Date: Sat, 13 Jun 2020 23:16:11 -0700 Subject: [PATCH 4/5] gnu: Add bcc. * gnu/packages/linux.scm (bcc): New variable. --- gnu/packages/linux.scm | 65 ++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 65 insertions(+) diff --git a/gnu/packages/linux.scm b/gnu/packages/linux.scm index f3eec978ec..0b8b08883a 100644 --- a/gnu/packages/linux.scm +++ b/gnu/packages/linux.scm @@ -97,6 +97,8 @@ #:use-module (gnu packages haskell-xyz) #:use-module (gnu packages libunwind) #:use-module (gnu packages libusb) + #:use-module (gnu packages llvm) + #:use-module (gnu packages lua) #:use-module (gnu packages man) #:use-module (gnu packages maths) #:use-module (gnu packages multiprecision) @@ -131,6 +133,7 @@ #:use-module (gnu packages rsync) #:use-module (gnu packages selinux) #:use-module (gnu packages swig) + #:use-module (gnu packages version-control) #:use-module (guix build-system cmake) #:use-module (guix build-system gnu) #:use-module (guix build-system go) @@ -7211,3 +7214,65 @@ persistent over reboots.") contrast to BCC, do not require Clang/LLVM runtime being deployed to target servers and does not rely on kernel-devel headers being available.") (license `(,license:lgpl2.1 ,license:bsd-2))))) + +(define-public bcc + (let* ((ver "0.14.0") + (commit (string-append "v" ver)) + (revision "1")) + (package + (name "bcc") + (version (git-version ver revision commit)) + (source + (origin + (method git-fetch) + (uri (git-reference + (url "https://github.com/iovisor/bcc") + (commit commit))) + (file-name (git-file-name name version)) + (sha256 + (base32 + "08m21avzamr48qwshd4r5hlcckk1kvgrb1i6qw373b7la89jf5an")))) + (build-system cmake-build-system) + (inputs + `(;; TODO: package optional integrations + ;; ("arping" ,argping) + ;; ("netperf" ,netperf) + ;; ("iperf" ,iperf) or ("iperf3" ,iperf3) + ("bison" ,bison) + ("clang-toolchain" ,clang-toolchain) + ("flex" ,flex) + ;; FIXME: Timestamp some other way. + ("git" ,git) + ("libbpf" ,(package-source libbpf)) + ;; LibElf required but libelf does not contain + ;; archives, only object files. + ;; https://github.com/iovisor/bcc/issues/504 + ("elfutils" ,elfutils) + ("linux-libre-headers" ,linux-libre-headers) + ("luajit" ,luajit) + ("python-wrapper" ,python-wrapper))) + (arguments + `(;; Tests all require sudo and a "standard" file heirarchy + #:tests? #f + ;; LIBBPF_INCLUDE_DIR should work with the output of libbpf - + ;; i.e. ,libbpf instead of ,(package-source libbpf) in inputs + ;; but it seems there could be a bug + ;; #:configure-flags + ;; (list + ;; (string-append + ;; "-DLIBBPF_INCLUDE_DIR=" (assoc-ref %build-inputs "libbpf") "/include")) + #:phases + (modify-phases %standard-phases + (add-after 'unpack 'copy-libbpf + (lambda* (#:key inputs #:allow-other-keys) + (delete-file-recursively "src/cc/libbpf") + (copy-recursively (assoc-ref inputs "libbpf") "src/cc/libbpf")))))) + (home-page "https://github.com/iovisor/bcc") + (synopsis "Tools for BPF on Linux") + (description + "BCC is a toolkit for creating efficient kernel tracing and +manipulation programs, and includes several useful tools and examples. It +makes use of extended BPF (Berkeley Packet Filters), formally known as eBPF, a +new feature that was first added to Linux 3.15. Much of what BCC uses requires +Linux 4.1 and above.") + (license license:asl2.0)))) -- 2.26.2 --=-=-= Content-Type: text/x-patch Content-Disposition: attachment; filename=0005-WIP-gnu-Add-bpftrace.patch >From 3176597e20946e7b3564479f655ac51b65109b5f Mon Sep 17 00:00:00 2001 From: John Soo Date: Sat, 13 Jun 2020 23:16:56 -0700 Subject: [PATCH 5/5] [WIP] gnu: Add bpftrace. * gnu/packages/linux.scm (bpftrace): New variable. --- gnu/packages/linux.scm | 60 ++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 60 insertions(+) diff --git a/gnu/packages/linux.scm b/gnu/packages/linux.scm index 0b8b08883a..00c4c41c82 100644 --- a/gnu/packages/linux.scm +++ b/gnu/packages/linux.scm @@ -7276,3 +7276,63 @@ makes use of extended BPF (Berkeley Packet Filters), formally known as eBPF, a new feature that was first added to Linux 3.15. Much of what BCC uses requires Linux 4.1 and above.") (license license:asl2.0)))) + +(define-public bpftrace + (let* ((ver "0.10.0") + (commit (string-append "v" ver)) + (revision "1")) + (package + (name "bpftrace") + (version (git-version ver revision commit)) + (source + (origin + (method git-fetch) + (uri (git-reference + (url "https://github.com/iovisor/bpftrace") + (commit commit))) + (file-name (git-file-name name ver)) + (sha256 + (base32 + "023ardywbw5w8815j2ny9rrp2xlpxndqaa7v2njjm8109p7ilsdn")))) + (build-system cmake-build-system) + (inputs + `(("bcc" ,bcc) + ("binutils" ,binutils) + ("bison" ,bison) + ("clang-toolchain" ,clang-toolchain) + ("elfutils" ,elfutils) + ("flex" ,flex) + ;; FIXME: Tests require googletest but not from system + ;; ("googletest" ,googletest) + ("libbpf" ,libbpf) + ("libiberty" ,libiberty) + ("linux-libre-headers" ,linux-libre-headers) + ("zlib" ,zlib))) + (arguments + `(#:configure-flags + (let ((libbpf (assoc-ref %build-inputs "libbpf")) + (zlib (assoc-ref %build-inputs "zlib")) + (binutils (assoc-ref %build-inputs "binutils"))) + `(;; FIXME: Make tests not clone the googletest repository + "-DBUILD_TESTING=OFF" + "-DSTATIC_LINKING=ON" + ;; ,(string-append "-DLIBBFD_LIBRARY=" binutils) + ;; ,(string-append "-DLIBBFD_INCLUDE_DIRS=" binutils "/include") + ,(string-append "-DLIBZ_LIBRARIES=" zlib "/lib") + ;; ,(string-append "-DLIBBPF_LIBRARY=" libbpf) + ;; ,(string-append "-DLIBBPF_LIBRARIES=" libbpf "/lib") + ;; ,(string-append "-DLIBBPF_INCLUDE_DIRS=" libbpf "/include") + ;; "-DLIBBFD_DISASM_FOUR_ARGS_SIGNATURE=ON" + )))) + (home-page "https://github.com/iovisor/bpftrace") + (synopsis "High-level tracing language for Linux eBPF") + (description + "bpftrace is a high-level tracing language for Linux enhanced Berkeley +Packet Filter (eBPF) available in recent Linux kernels (4.x). bpftrace uses +LLVM as a backend to compile scripts to BPF-bytecode and makes use of BCC for +interacting with the Linux BPF system, as well as existing Linux tracing +capabilities: kernel dynamic tracing (kprobes), user-level dynamic +tracing (uprobes), and tracepoints. The bpftrace language is inspired by awk +and C, and predecessor tracers such as DTrace and SystemTap. bpftrace was +created by Alastair Robertson.") + (license license:asl2.0)))) -- 2.26.2 --=-=-=--