From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp2 ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms11 with LMTPS id YMzgGtboCV8OFwAA0tVLHw (envelope-from ) for ; Sat, 11 Jul 2020 16:29:10 +0000 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp2 with LMTPS id gLzeFtboCV+rSAAAB5/wlQ (envelope-from ) for ; Sat, 11 Jul 2020 16:29:10 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id DA8A9940B75 for ; Sat, 11 Jul 2020 16:29:09 +0000 (UTC) Received: from localhost ([::1]:52504 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1juINA-00023n-TJ for larch@yhetil.org; Sat, 11 Jul 2020 12:29:08 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:56046) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1juIN3-00023X-UP for guix-patches@gnu.org; Sat, 11 Jul 2020 12:29:01 -0400 Received: from debbugs.gnu.org ([209.51.188.43]:33300) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1juIN3-0002Pt-Kq for guix-patches@gnu.org; Sat, 11 Jul 2020 12:29:01 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1juIN3-0000CN-I8 for guix-patches@gnu.org; Sat, 11 Jul 2020 12:29:01 -0400 X-Loop: help-debbugs@gnu.org Subject: [bug#42227] BPF in linux-libre Resent-From: John Soo Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Sat, 11 Jul 2020 16:29:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 42227 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: To: Mathieu Othacehe Cc: 42227@debbugs.gnu.org Received: via spool by 42227-submit@debbugs.gnu.org id=B42227.1594484920733 (code B ref 42227); Sat, 11 Jul 2020 16:29:01 +0000 Received: (at 42227) by debbugs.gnu.org; 11 Jul 2020 16:28:40 +0000 Received: from localhost ([127.0.0.1]:44846 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1juIMe-0000Bg-6m for submit@debbugs.gnu.org; Sat, 11 Jul 2020 12:28:40 -0400 Received: from mail-pl1-f173.google.com ([209.85.214.173]:44921) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1juIMY-0000BR-Ph for 42227@debbugs.gnu.org; Sat, 11 Jul 2020 12:28:34 -0400 Received: by mail-pl1-f173.google.com with SMTP id w17so3480865ply.11 for <42227@debbugs.gnu.org>; Sat, 11 Jul 2020 09:28:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=asu-edu.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:references:date:in-reply-to:message-id :user-agent:mime-version; bh=/byqWv/ezmWal3vJAbecPt7lzlFcqKZM2IUzZXal6ds=; b=SIkro7OPV/UL2srhN4xZtgWAliOJnEDnhlkSphfH4uK9fPl27f4hr4uTDeP78CQ8wT e0y82VnsiVc5fTMqWtoaGpz2ry8ALRsRiSIkpzHDwd0+/mdtio5yNy6JeX7QiEVli7d2 Gi7DAYfHH+kglmYUDf6YgdPXkfXsPJT1YMYE+4NJnGjLr3ekzMPpOXM38rnfGtPiUmIf mXzIPIwCcPvWtX5QUkOBF/xFRWOE6hFTJwPXRTuL184aRn3MGPsaO0IF1GRe7DKjOoaI CSm5EnP5C/7r51fPxjCvdA6reHvryPMiNc/hJlPovOn5Hm1KP3oHVm/z9AuCROMFkFEd cUYg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:references:date:in-reply-to :message-id:user-agent:mime-version; bh=/byqWv/ezmWal3vJAbecPt7lzlFcqKZM2IUzZXal6ds=; b=FzMhvG2+665OSwxaUMU3SVoLfOqyK0WrWJ3jClSpAgs9+YJ9YwUF8wfhXoSCuibAlT 7Ck0uLivF8Fp4/WW2fjvyYNXIoZMbwqw8CLB7PEnbaAEK5TSvcDwigzWhef7bIn9aFLA uvTIB1hyrTugcmXai+pVboVfZ270kP5X8PYV/LobOiP/HL1krANyHR0QKptBuATGtSmp X3NJkFGqGKwsRNJiaO5u7e/6hN4CVdZk2nnSQ9xV6WvsZDDvZJILlLppau2aGkkTDeBn Nl8H0J9EfoPsWu6qI6SohVLoH3632nCNYSnSHMixGYBa4XQYcaWibopWbQzI4m0o/YGU zkYQ== X-Gm-Message-State: AOAM530cATBbLogJlz0wQ+gy/UiXplsDjdehgFiZwSu0HEYV7NdwkCGZ klYd5z6inwJLQFL/RbQ54jtgJz1g6xc= X-Google-Smtp-Source: ABdhPJz/9FYzkV9OFXPmLP1LSlwAX+pMMTPbpLbEvW6Ho/LuYsNQHGqd1dixiumjGjZE+QTKJqzqsQ== X-Received: by 2002:a17:902:9886:: with SMTP id s6mr40339096plp.112.1594484904444; Sat, 11 Jul 2020 09:28:24 -0700 (PDT) Received: from ecenter ([2600:1700:83b0:8bd0::6a0]) by smtp.gmail.com with ESMTPSA id w29sm9276305pfq.128.2020.07.11.09.28.22 (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Sat, 11 Jul 2020 09:28:23 -0700 (PDT) From: John Soo References: <44046537-51AD-4BD7-8442-41738C4D8C98@asu.edu> <87lfkqknla.fsf@gnu.org> <87h7vdd6ms.fsf@asu.edu> <87wo46uhdr.fsf@gnu.org> <87zh8wph7h.fsf@asu.edu> <87wo3uxfrq.fsf@gnu.org> <871rlsegf2.fsf@asu.edu> <87h7umjruz.fsf@gnu.org> <87zh8dlbeu.fsf@asu.edu> <87d0587p1o.fsf@gnu.org> Date: Sat, 11 Jul 2020 09:28:19 -0700 In-Reply-To: <87d0587p1o.fsf@gnu.org> (Mathieu Othacehe's message of "Mon, 06 Jul 2020 15:26:11 +0200") Message-ID: <87r1ti9fto.fsf@asu.edu> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.3 (gnu/linux) MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" X-Spam-Score: 0.0 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-Spam-Score: -1.0 (-) X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+larch=yhetil.org@gnu.org Sender: "Guix-patches" X-Scanner: scn0 Authentication-Results: aspmx1.migadu.com; dkim=fail (rsa verify failed) header.d=asu-edu.20150623.gappssmtp.com header.s=20150623 header.b=SIkro7OP; dmarc=fail reason="SPF not aligned (relaxed)" header.from=asu.edu (policy=none); spf=pass (aspmx1.migadu.com: domain of guix-patches-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=guix-patches-bounces@gnu.org X-Spam-Score: 0.09 X-TUID: a/NA/lMwm1Uf --=-=-= Content-Type: text/plain Hi Mathieu, > Hey John, > > I'll take more time to review patches 4 and 5. However, while trying > some of the examples packaged by BCC, I have the following error: > > mathieu@meru:~/guix-master$ /gnu/store/rv51f9n1w9i92m9qsg9k3ilsy3hyhjf3-bcc-0.15.0/share/bcc/tools/execsnoop > Traceback (most recent call last): > File "/gnu/store/rv51f9n1w9i92m9qsg9k3ilsy3hyhjf3-bcc-0.15.0/share/bcc/tools/execsnoop", line 21, in > from bcc import BPF > ModuleNotFoundError: No module named 'bcc' > > I think an additional wrapping is necessary. Could you please have a > look? I'm also removing help-guix, and opening a proper guix-patches > ticket. I wrapped the PYTHONPATH around the various provided python tools. I also found a spare path that required patching. I am not sure this fixes every tool but I did get a few to work now. Thanks for attaching guix-patches. I also added debugfs as a requirement for a bpf system. To use it %bpf-file-systems can be used in place of %base-file-systems in the operating system definition. Thanks, John --=-=-= Content-Type: text/x-patch Content-Disposition: attachment; filename=0001-gnu-Add-debugfs-file-system.patch >From cef37cce474bba3d023ad5426da52050469b6196 Mon Sep 17 00:00:00 2001 From: John Soo Date: Thu, 9 Jul 2020 02:43:14 -0700 Subject: [PATCH 1/4] gnu: Add %debugfs file-system. * gnu/system/file-systems.scm. --- gnu/system/file-systems.scm | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/gnu/system/file-systems.scm b/gnu/system/file-systems.scm index 0f94577760..57e0d64d01 100644 --- a/gnu/system/file-systems.scm +++ b/gnu/system/file-systems.scm @@ -428,6 +428,14 @@ TARGET in the other system." '("cpuset" "cpu" "cpuacct" "memory" "devices" "freezer" "blkio" "perf_event" "pids"))))) +(define %debugfs + (file-system + (type "debugfs") + (device "none") + (mount-point "/sys/kernel/debug") + (check? #f) + (create-mount-point? #t))) + (define %elogind-file-systems ;; We don't use systemd, but these file systems are needed for elogind, ;; which was extracted from systemd. -- 2.27.0 --=-=-= Content-Type: text/x-patch Content-Disposition: attachment; filename=0002-gnu-Add-bpf-file-systems.patch >From 44d3564b5552605f0ab4f1d06812cc804f046229 Mon Sep 17 00:00:00 2001 From: John Soo Date: Thu, 9 Jul 2020 02:43:48 -0700 Subject: [PATCH 2/4] gnu: Add %bpf-file-systems. * gnu/system/file-systems.scm (%bpf-file-systems): New variable. --- gnu/system/file-systems.scm | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/gnu/system/file-systems.scm b/gnu/system/file-systems.scm index 57e0d64d01..ee200e4055 100644 --- a/gnu/system/file-systems.scm +++ b/gnu/system/file-systems.scm @@ -74,6 +74,7 @@ %elogind-file-systems %base-file-systems + %bpf-file-systems %container-file-systems @@ -476,6 +477,9 @@ TARGET in the other system." %shared-memory-file-system %immutable-store)) +(define %bpf-file-systems + (cons %debugfs %base-file-systems)) + ;; File systems for Linux containers differ from %base-file-systems in that ;; they impose additional restrictions such as no-exec or need different ;; options to function properly. -- 2.27.0 --=-=-= Content-Type: text/x-patch Content-Disposition: attachment; filename=0003-gnu-Add-bcc.patch >From f023b62085bf8a1f00092163f827b9dd28b87519 Mon Sep 17 00:00:00 2001 From: John Soo Date: Sat, 13 Jun 2020 23:16:11 -0700 Subject: [PATCH 3/4] gnu: Add bcc. * gnu/packages/linux.scm (bcc): New variable. --- gnu/packages/linux.scm | 85 ++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 85 insertions(+) diff --git a/gnu/packages/linux.scm b/gnu/packages/linux.scm index 94d3b37845..b3922b0770 100644 --- a/gnu/packages/linux.scm +++ b/gnu/packages/linux.scm @@ -98,6 +98,8 @@ #:use-module (gnu packages haskell-xyz) #:use-module (gnu packages libunwind) #:use-module (gnu packages libusb) + #:use-module (gnu packages llvm) + #:use-module (gnu packages lua) #:use-module (gnu packages man) #:use-module (gnu packages maths) #:use-module (gnu packages multiprecision) @@ -132,6 +134,7 @@ #:use-module (gnu packages rsync) #:use-module (gnu packages selinux) #:use-module (gnu packages swig) + #:use-module (gnu packages version-control) #:use-module (guix build-system cmake) #:use-module (guix build-system gnu) #:use-module (guix build-system go) @@ -7229,3 +7232,85 @@ persistent over reboots.") contrast to BCC, do not require the Clang/LLVM runtime or linux kernel headers.") (license `(,license:lgpl2.1 ,license:bsd-2)))) + +(define-public bcc + (package + (name "bcc") + (version "0.15.0") + (source + (origin + (method git-fetch) + (uri (git-reference + (url "https://github.com/iovisor/bcc") + (commit (string-append "v" version)))) + (file-name (git-file-name name version)) + (sha256 + (base32 + "1d5j9zanffa1c7lpi5fcrdlx1n7hy86xl82fam2xqr0s41q4ipxw")))) + (build-system cmake-build-system) + (native-inputs + `(("bison" ,bison) + ("flex" ,flex))) + (inputs + `(;; TODO: package optional integrations + ;; ("arping" ,argping) + ;; ("netperf" ,netperf) + ;; ("iperf" ,iperf) or ("iperf3" ,iperf3) + ("clang-toolchain" ,clang-toolchain) + ("libbpf" ,(package-source libbpf)) + ;; LibElf required but libelf does not contain + ;; archives, only object files. + ;; https://github.com/iovisor/bcc/issues/504 + ("elfutils" ,elfutils) + ("linux-libre-headers" ,linux-libre-headers) + ("luajit" ,luajit) + ("python-wrapper" ,python-wrapper))) + (arguments + `(;; Tests all require sudo and a "standard" file heirarchy + #:tests? #f + #:configure-flags + (let ((revision ,version)) + `(,(string-append "-DREVISION=" revision))) + #:phases + (modify-phases %standard-phases + ;; FIXME: "-DCMAKE_USE_LIBBPF_PACKAGE=ON" + ;; Does not make bcc use libbpf from system + (add-after 'unpack 'copy-libbpf + (lambda* (#:key inputs #:allow-other-keys) + (delete-file-recursively "src/cc/libbpf") + (copy-recursively + (assoc-ref inputs "libbpf") "src/cc/libbpf"))) + (add-after 'copy-libbpf 'substitute-libbc + (lambda* (#:key outputs #:allow-other-keys) + (substitute* "src/python/bcc/libbcc.py" + (("(libbcc\\.so.*)\\b" _ libbcc) + (string-append + (assoc-ref outputs "out") "/lib/" libbcc))))) + (add-after 'install 'wrap-tools + (lambda* (#:key outputs #:allow-other-keys) + (use-modules (ice-9 textual-ports)) + (let* ((out (assoc-ref outputs "out")) + (lib (string-append out "/lib")) + (tools (string-append out "/share/bcc/tools")) + (python-executable? + (lambda (filename _) + (call-with-input-file filename + (lambda (port) + (string-contains (get-line port) "/bin/python")))))) + (for-each + (lambda (python-executable) + (format #t "wrapping: ~A~%" python-executable) + (wrap-program python-executable + `("PYTHONPATH" ":" prefix + (,(string-append lib "/python3.8/site-packages"))))) + (find-files tools python-executable?)) + #t)))))) + (home-page "https://github.com/iovisor/bcc") + (synopsis "Tools for BPF on Linux") + (description + "BCC is a toolkit for creating efficient kernel tracing and +manipulation programs, and includes several useful tools and examples. It +makes use of extended BPF (Berkeley Packet Filters), formally known as eBPF, a +new feature that was first added to Linux 3.15. Much of what BCC uses requires +Linux 4.1 and above.") + (license license:asl2.0))) -- 2.27.0 --=-=-= Content-Type: text/x-patch Content-Disposition: attachment; filename=0004-gnu-Add-bpftrace.patch >From f17d7242b3821b6a5600bfd5e9a0f75fa4054d60 Mon Sep 17 00:00:00 2001 From: John Soo Date: Sat, 13 Jun 2020 23:16:56 -0700 Subject: [PATCH 4/4] gnu: Add bpftrace. * gnu/packages/linux.scm (bpftrace): New variable. * gnu/packages/patches/bpftrace-disable-bfd-disasm.patch: Disable bfd disassembly for bpftrace. * gnu/local.mk (dist_patch_DATA): Add bpftrace-disable-bfd-disasm.patch. --- gnu/local.mk | 1 + gnu/packages/linux.scm | 48 +++++++++++++++++++ .../patches/bpftrace-disable-bfd-disasm.patch | 15 ++++++ 3 files changed, 64 insertions(+) create mode 100644 gnu/packages/patches/bpftrace-disable-bfd-disasm.patch diff --git a/gnu/local.mk b/gnu/local.mk index a277e63fa4..5c9d39663f 100644 --- a/gnu/local.mk +++ b/gnu/local.mk @@ -819,6 +819,7 @@ dist_patch_DATA = \ %D%/packages/patches/bitcoin-core-python-compat.patch \ %D%/packages/patches/blender-2.79-newer-ffmpeg.patch \ %D%/packages/patches/blender-2.79-python-3.7-fix.patch \ + %D%/packages/patches/bpftrace-disable-bfd-disasm.patch \ %D%/packages/patches/busybox-1.31.1-fix-build-with-glibc-2.31.patch \ %D%/packages/patches/byobu-writable-status.patch \ %D%/packages/patches/calibre-no-updates-dialog.patch \ diff --git a/gnu/packages/linux.scm b/gnu/packages/linux.scm index b3922b0770..8f19ca11d2 100644 --- a/gnu/packages/linux.scm +++ b/gnu/packages/linux.scm @@ -7314,3 +7314,51 @@ makes use of extended BPF (Berkeley Packet Filters), formally known as eBPF, a new feature that was first added to Linux 3.15. Much of what BCC uses requires Linux 4.1 and above.") (license license:asl2.0))) + +(define-public bpftrace + (package + (name "bpftrace") + (version "0.10.0") + (source + (origin + (method git-fetch) + (uri (git-reference + (url "https://github.com/iovisor/bpftrace") + (commit (string-append "v" version)))) + (file-name (git-file-name name version)) + (sha256 + (base32 + "023ardywbw5w8815j2ny9rrp2xlpxndqaa7v2njjm8109p7ilsdn")) + (patches (search-patches "bpftrace-disable-bfd-disasm.patch")))) + (build-system cmake-build-system) + (native-inputs + `(("bison" ,bison) + ("flex" ,flex))) + (inputs + `(("bcc" ,bcc) + ("clang-toolchain" ,clang-toolchain) + ("elfutils" ,elfutils) + ;; FIXME: Tests require googletest but clone repository + ;; ("googletest" ,googletest) + ("libbpf" ,libbpf) + ("linux-libre-headers" ,linux-libre-headers))) + (arguments + `(#:tests? #f ; FIXME: Enable when googletest from guix is used + #:configure-flags + '(;; FIXME: Make tests not clone the googletest repository + "-DBUILD_TESTING=OFF" + ;; FIXME: libbfd misses some link dependencies + ;; When fixed, remove patch + "-DHAVE_BFD_DISASM=OFF"))) + (home-page "https://github.com/iovisor/bpftrace") + (synopsis "High-level tracing language for Linux eBPF") + (description + "bpftrace is a high-level tracing language for Linux enhanced Berkeley +Packet Filter (eBPF) available in recent Linux kernels (4.x). bpftrace uses +LLVM as a backend to compile scripts to BPF-bytecode and makes use of BCC for +interacting with the Linux BPF system, as well as existing Linux tracing +capabilities: kernel dynamic tracing (kprobes), user-level dynamic +tracing (uprobes), and tracepoints. The bpftrace language is inspired by awk +and C, and predecessor tracers such as DTrace and SystemTap. bpftrace was +created by Alastair Robertson.") + (license license:asl2.0))) diff --git a/gnu/packages/patches/bpftrace-disable-bfd-disasm.patch b/gnu/packages/patches/bpftrace-disable-bfd-disasm.patch new file mode 100644 index 0000000000..8565d8d851 --- /dev/null +++ b/gnu/packages/patches/bpftrace-disable-bfd-disasm.patch @@ -0,0 +1,15 @@ +diff --git a/CMakeLists.txt b/CMakeLists.txt +index e89a6a9..a594786 100644 +--- a/CMakeLists.txt ++++ b/CMakeLists.txt +@@ -126,10 +126,6 @@ find_package(LibBpf) + find_package(LibBfd) + find_package(LibOpcodes) + +-if(${LIBBFD_FOUND} AND ${LIBOPCODES_FOUND}) +- set(HAVE_BFD_DISASM TRUE) +-endif() +- + include(CheckIncludeFile) + check_include_file("sys/sdt.h" HAVE_SYSTEMTAP_SYS_SDT_H) + -- 2.27.0 --=-=-=--