[bug#42227] BPF in linux-libre

[bug#42227] BPF in linux-libre

[Mathieu Othacehe]

Hey John,

> I think I tidied up the description to match the Guix situation. What do
> you think now?

Yes it's fine, thanks for the updated serie! I pushed the first patch
and patches two and three squashed together.

> I could see it being a useful default.  BPF seems like a nice technology
> but I am making these patches to experiment with it myself.  Because I
> haven't used it much I can't really speak on the pros of making it
> default.  Other than my gut feeling that seems like something that
> should be opted into rather than opting out of I have no strong feelings
> on including it by default.  The only other downside I see is that
> putting in the default might make the linux definitions less composable.
> The way it is now, one can assemble a (mostly) bpf-capable system from
> the pieces in gnu/packages/linux.scm.

Ok, thanks for explaining. I don't have much experience with BPF
either. For now we can work with a separate linux-libre, and will see
about merging it into the default, when we'll have more perspective.

I'll take more time to review patches 4 and 5. However, while trying
some of the examples packaged by BCC, I have the following error:

--8<---------------cut here---------------start------------->8---
mathieu@meru:~/guix-master$ /gnu/store/rv51f9n1w9i92m9qsg9k3ilsy3hyhjf3-bcc-0.15.0/share/bcc/tools/execsnoop
Traceback (most recent call last):
  File "/gnu/store/rv51f9n1w9i92m9qsg9k3ilsy3hyhjf3-bcc-0.15.0/share/bcc/tools/execsnoop", line 21, in <module>
    from bcc import BPF
ModuleNotFoundError: No module named 'bcc'
--8<---------------cut here---------------end--------------->8---

I think an additional wrapping is necessary. Could you please have a
look? I'm also removing help-guix, and opening a proper guix-patches
ticket.

Thanks,

Mathieu

[bug#42227] BPF in linux-libre

[John Soo]

[-- Attachment #1: Type: text/plain, Size: 1059 bytes --]

Hi Mathieu,

> Hey John,
>
> I'll take more time to review patches 4 and 5. However, while trying
> some of the examples packaged by BCC, I have the following error:
>
> mathieu@meru:~/guix-master$ /gnu/store/rv51f9n1w9i92m9qsg9k3ilsy3hyhjf3-bcc-0.15.0/share/bcc/tools/execsnoop
> Traceback (most recent call last):
>   File "/gnu/store/rv51f9n1w9i92m9qsg9k3ilsy3hyhjf3-bcc-0.15.0/share/bcc/tools/execsnoop", line 21, in <module>
>     from bcc import BPF
> ModuleNotFoundError: No module named 'bcc'
>
> I think an additional wrapping is necessary. Could you please have a
> look? I'm also removing help-guix, and opening a proper guix-patches
> ticket.

I wrapped the PYTHONPATH around the various provided python tools.
I also found a spare path that required patching.
I am not sure this fixes every tool but I did get a few to work now.

Thanks for attaching guix-patches.

I also added debugfs as a requirement for a bpf system.  To use it
%bpf-file-systems can be used in place of %base-file-systems in the
operating system definition.

Thanks,

John


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-gnu-Add-debugfs-file-system.patch --]
[-- Type: text/x-patch, Size: 978 bytes --]

From cef37cce474bba3d023ad5426da52050469b6196 Mon Sep 17 00:00:00 2001
From: John Soo <jsoo1@asu.edu>
Date: Thu, 9 Jul 2020 02:43:14 -0700
Subject: [PATCH 1/4] gnu: Add %debugfs file-system.

* gnu/system/file-systems.scm.
---
 gnu/system/file-systems.scm | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/gnu/system/file-systems.scm b/gnu/system/file-systems.scm
index 0f94577760..57e0d64d01 100644
--- a/gnu/system/file-systems.scm
+++ b/gnu/system/file-systems.scm
@@ -428,6 +428,14 @@ TARGET in the other system."
                '("cpuset" "cpu" "cpuacct" "memory" "devices" "freezer"
                  "blkio" "perf_event" "pids")))))
 
+(define %debugfs
+  (file-system
+    (type "debugfs")
+    (device "none")
+    (mount-point "/sys/kernel/debug")
+    (check? #f)
+    (create-mount-point? #t)))
+
 (define %elogind-file-systems
   ;; We don't use systemd, but these file systems are needed for elogind,
   ;; which was extracted from systemd.
-- 
2.27.0


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #3: 0002-gnu-Add-bpf-file-systems.patch --]
[-- Type: text/x-patch, Size: 1077 bytes --]

From 44d3564b5552605f0ab4f1d06812cc804f046229 Mon Sep 17 00:00:00 2001
From: John Soo <jsoo1@asu.edu>
Date: Thu, 9 Jul 2020 02:43:48 -0700
Subject: [PATCH 2/4] gnu: Add %bpf-file-systems.

* gnu/system/file-systems.scm (%bpf-file-systems): New variable.
---
 gnu/system/file-systems.scm | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/gnu/system/file-systems.scm b/gnu/system/file-systems.scm
index 57e0d64d01..ee200e4055 100644
--- a/gnu/system/file-systems.scm
+++ b/gnu/system/file-systems.scm
@@ -74,6 +74,7 @@
             %elogind-file-systems
 
             %base-file-systems
+            %bpf-file-systems
             %container-file-systems
 
             <file-system-mapping>
@@ -476,6 +477,9 @@ TARGET in the other system."
         %shared-memory-file-system
         %immutable-store))
 
+(define %bpf-file-systems
+  (cons %debugfs %base-file-systems))
+
 ;; File systems for Linux containers differ from %base-file-systems in that
 ;; they impose additional restrictions such as no-exec or need different
 ;; options to function properly.
-- 
2.27.0


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #4: 0003-gnu-Add-bcc.patch --]
[-- Type: text/x-patch, Size: 4783 bytes --]

From f023b62085bf8a1f00092163f827b9dd28b87519 Mon Sep 17 00:00:00 2001
From: John Soo <jsoo1@asu.edu>
Date: Sat, 13 Jun 2020 23:16:11 -0700
Subject: [PATCH 3/4] gnu: Add bcc.

* gnu/packages/linux.scm (bcc): New variable.
---
 gnu/packages/linux.scm | 85 ++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 85 insertions(+)

diff --git a/gnu/packages/linux.scm b/gnu/packages/linux.scm
index 94d3b37845..b3922b0770 100644
--- a/gnu/packages/linux.scm
+++ b/gnu/packages/linux.scm
@@ -98,6 +98,8 @@
   #:use-module (gnu packages haskell-xyz)
   #:use-module (gnu packages libunwind)
   #:use-module (gnu packages libusb)
+  #:use-module (gnu packages llvm)
+  #:use-module (gnu packages lua)
   #:use-module (gnu packages man)
   #:use-module (gnu packages maths)
   #:use-module (gnu packages multiprecision)
@@ -132,6 +134,7 @@
   #:use-module (gnu packages rsync)
   #:use-module (gnu packages selinux)
   #:use-module (gnu packages swig)
+  #:use-module (gnu packages version-control)
   #:use-module (guix build-system cmake)
   #:use-module (guix build-system gnu)
   #:use-module (guix build-system go)
@@ -7229,3 +7232,85 @@ persistent over reboots.")
 contrast to BCC, do not require the Clang/LLVM runtime or linux kernel
 headers.")
     (license `(,license:lgpl2.1 ,license:bsd-2))))
+
+(define-public bcc
+  (package
+    (name "bcc")
+    (version "0.15.0")
+    (source
+     (origin
+       (method git-fetch)
+       (uri (git-reference
+             (url "https://github.com/iovisor/bcc")
+             (commit (string-append "v" version))))
+       (file-name (git-file-name name version))
+       (sha256
+        (base32
+         "1d5j9zanffa1c7lpi5fcrdlx1n7hy86xl82fam2xqr0s41q4ipxw"))))
+    (build-system cmake-build-system)
+    (native-inputs
+     `(("bison" ,bison)
+       ("flex" ,flex)))
+    (inputs
+     `(;; TODO: package optional integrations
+       ;; ("arping" ,argping)
+       ;; ("netperf" ,netperf)
+       ;; ("iperf" ,iperf) or ("iperf3" ,iperf3)
+       ("clang-toolchain" ,clang-toolchain)
+       ("libbpf" ,(package-source libbpf))
+       ;; LibElf required but libelf does not contain
+       ;; archives, only object files.
+       ;; https://github.com/iovisor/bcc/issues/504
+       ("elfutils" ,elfutils)
+       ("linux-libre-headers" ,linux-libre-headers)
+       ("luajit" ,luajit)
+       ("python-wrapper" ,python-wrapper)))
+    (arguments
+     `(;; Tests all require sudo and a "standard" file heirarchy
+       #:tests? #f
+       #:configure-flags
+       (let ((revision ,version))
+         `(,(string-append "-DREVISION=" revision)))
+       #:phases
+       (modify-phases %standard-phases
+         ;; FIXME: "-DCMAKE_USE_LIBBPF_PACKAGE=ON"
+         ;; Does not make bcc use libbpf from system
+         (add-after 'unpack 'copy-libbpf
+           (lambda* (#:key inputs #:allow-other-keys)
+             (delete-file-recursively "src/cc/libbpf")
+             (copy-recursively
+              (assoc-ref inputs "libbpf") "src/cc/libbpf")))
+         (add-after 'copy-libbpf 'substitute-libbc
+           (lambda* (#:key outputs #:allow-other-keys)
+             (substitute* "src/python/bcc/libbcc.py"
+               (("(libbcc\\.so.*)\\b" _ libbcc)
+                (string-append
+                 (assoc-ref outputs "out") "/lib/" libbcc)))))
+         (add-after 'install 'wrap-tools
+           (lambda* (#:key outputs #:allow-other-keys)
+             (use-modules (ice-9 textual-ports))
+             (let* ((out (assoc-ref outputs "out"))
+                    (lib (string-append out "/lib"))
+                    (tools (string-append out "/share/bcc/tools"))
+                    (python-executable?
+                     (lambda (filename _)
+                       (call-with-input-file filename
+                         (lambda (port)
+                           (string-contains (get-line port) "/bin/python"))))))
+               (for-each
+                (lambda (python-executable)
+                  (format #t "wrapping: ~A~%" python-executable)
+                  (wrap-program python-executable
+                    `("PYTHONPATH" ":" prefix
+                      (,(string-append lib "/python3.8/site-packages")))))
+                (find-files tools python-executable?))
+               #t))))))
+    (home-page "https://github.com/iovisor/bcc")
+    (synopsis "Tools for BPF on Linux")
+    (description
+     "BCC is a toolkit for creating efficient kernel tracing and
+manipulation programs, and includes several useful tools and examples.  It
+makes use of extended BPF (Berkeley Packet Filters), formally known as eBPF, a
+new feature that was first added to Linux 3.15.  Much of what BCC uses requires
+Linux 4.1 and above.")
+    (license license:asl2.0)))
-- 
2.27.0


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #5: 0004-gnu-Add-bpftrace.patch --]
[-- Type: text/x-patch, Size: 4334 bytes --]

From f17d7242b3821b6a5600bfd5e9a0f75fa4054d60 Mon Sep 17 00:00:00 2001
From: John Soo <jsoo1@asu.edu>
Date: Sat, 13 Jun 2020 23:16:56 -0700
Subject: [PATCH 4/4] gnu: Add bpftrace.

* gnu/packages/linux.scm (bpftrace): New variable.
* gnu/packages/patches/bpftrace-disable-bfd-disasm.patch: Disable bfd
disassembly for bpftrace.
* gnu/local.mk (dist_patch_DATA): Add bpftrace-disable-bfd-disasm.patch.
---
 gnu/local.mk                                  |  1 +
 gnu/packages/linux.scm                        | 48 +++++++++++++++++++
 .../patches/bpftrace-disable-bfd-disasm.patch | 15 ++++++
 3 files changed, 64 insertions(+)
 create mode 100644 gnu/packages/patches/bpftrace-disable-bfd-disasm.patch

diff --git a/gnu/local.mk b/gnu/local.mk
index a277e63fa4..5c9d39663f 100644
--- a/gnu/local.mk
+++ b/gnu/local.mk
@@ -819,6 +819,7 @@ dist_patch_DATA =						\
   %D%/packages/patches/bitcoin-core-python-compat.patch		\
   %D%/packages/patches/blender-2.79-newer-ffmpeg.patch		\
   %D%/packages/patches/blender-2.79-python-3.7-fix.patch	\
+  %D%/packages/patches/bpftrace-disable-bfd-disasm.patch	\
   %D%/packages/patches/busybox-1.31.1-fix-build-with-glibc-2.31.patch \
   %D%/packages/patches/byobu-writable-status.patch		\
   %D%/packages/patches/calibre-no-updates-dialog.patch		\
diff --git a/gnu/packages/linux.scm b/gnu/packages/linux.scm
index b3922b0770..8f19ca11d2 100644
--- a/gnu/packages/linux.scm
+++ b/gnu/packages/linux.scm
@@ -7314,3 +7314,51 @@ makes use of extended BPF (Berkeley Packet Filters), formally known as eBPF, a
 new feature that was first added to Linux 3.15.  Much of what BCC uses requires
 Linux 4.1 and above.")
     (license license:asl2.0)))
+
+(define-public bpftrace
+  (package
+    (name "bpftrace")
+    (version "0.10.0")
+    (source
+     (origin
+       (method git-fetch)
+       (uri (git-reference
+             (url "https://github.com/iovisor/bpftrace")
+             (commit (string-append "v" version))))
+       (file-name (git-file-name name version))
+       (sha256
+        (base32
+         "023ardywbw5w8815j2ny9rrp2xlpxndqaa7v2njjm8109p7ilsdn"))
+       (patches (search-patches "bpftrace-disable-bfd-disasm.patch"))))
+    (build-system cmake-build-system)
+    (native-inputs
+     `(("bison" ,bison)
+       ("flex" ,flex)))
+    (inputs
+     `(("bcc" ,bcc)
+       ("clang-toolchain" ,clang-toolchain)
+       ("elfutils" ,elfutils)
+       ;; FIXME: Tests require googletest but clone repository
+       ;; ("googletest" ,googletest)
+       ("libbpf" ,libbpf)
+       ("linux-libre-headers" ,linux-libre-headers)))
+    (arguments
+     `(#:tests? #f ; FIXME: Enable when googletest from guix is used
+       #:configure-flags
+       '(;; FIXME: Make tests not clone the googletest repository
+         "-DBUILD_TESTING=OFF"
+         ;; FIXME: libbfd misses some link dependencies
+         ;; When fixed, remove patch
+         "-DHAVE_BFD_DISASM=OFF")))
+    (home-page "https://github.com/iovisor/bpftrace")
+    (synopsis "High-level tracing language for Linux eBPF")
+    (description
+     "bpftrace is a high-level tracing language for Linux enhanced Berkeley
+Packet Filter (eBPF) available in recent Linux kernels (4.x).  bpftrace uses
+LLVM as a backend to compile scripts to BPF-bytecode and makes use of BCC for
+interacting with the Linux BPF system, as well as existing Linux tracing
+capabilities: kernel dynamic tracing (kprobes), user-level dynamic
+tracing (uprobes), and tracepoints.  The bpftrace language is inspired by awk
+and C, and predecessor tracers such as DTrace and SystemTap.  bpftrace was
+created by Alastair Robertson.")
+    (license license:asl2.0)))
diff --git a/gnu/packages/patches/bpftrace-disable-bfd-disasm.patch b/gnu/packages/patches/bpftrace-disable-bfd-disasm.patch
new file mode 100644
index 0000000000..8565d8d851
--- /dev/null
+++ b/gnu/packages/patches/bpftrace-disable-bfd-disasm.patch
@@ -0,0 +1,15 @@
+diff --git a/CMakeLists.txt b/CMakeLists.txt
+index e89a6a9..a594786 100644
+--- a/CMakeLists.txt
++++ b/CMakeLists.txt
+@@ -126,10 +126,6 @@ find_package(LibBpf)
+ find_package(LibBfd)
+ find_package(LibOpcodes)
+ 
+-if(${LIBBFD_FOUND} AND ${LIBOPCODES_FOUND})
+-  set(HAVE_BFD_DISASM TRUE)
+-endif()
+-
+ include(CheckIncludeFile)
+ check_include_file("sys/sdt.h" HAVE_SYSTEMTAP_SYS_SDT_H)
+ 
-- 
2.27.0

[bug#42227] BPF in linux-libre

[Mathieu Othacehe]

Hey John,

Thanks for your patience!

> I wrapped the PYTHONPATH around the various provided python tools.
> I also found a spare path that required patching.
> I am not sure this fixes every tool but I did get a few to work now.

Yes, I tested some of them, looks fine :).

> I also added debugfs as a requirement for a bpf system.  To use it
> %bpf-file-systems can be used in place of %base-file-systems in the
> operating system definition.

Actually, I wonder if we could mount debugfs by default, by adding it to
%base-file-systems. Any objections?

In the meantime I pushed the bcc patch with a few edits. Regarding
bpftrace, I'd like to avoid the "-DHAVE_BFD_DISASM=OFF" patching, I
found this ticket which seems related:
https://github.com/iovisor/bpftrace/issues/1106, but didn't make any
significant progress yet.

Thanks,

Mathieu

[bug#42227] BPF in linux-libre

[John Soo]

Hi Mathieu!


> Thanks for your patience!

No problem. I'm quite busy too.

>> I also added debugfs as a requirement for a bpf system.  To use it
>> %bpf-file-systems can be used in place of %base-file-systems in the
>> operating system definition.
>
> Actually, I wonder if we could mount debugfs by default, by adding it to
> %base-file-systems. Any objections?

That seems ok. I did find a few questions about debugfs on old irc logs
and mailing lists.  My only concern again is that I would prefer to opt
in to such a thing.  debugfs is much simpler than the bpf kernel flags
though, so maybe it will be ok to remove in the future.

> In the meantime I pushed the bcc patch with a few edits. Regarding
> bpftrace, I'd like to avoid the "-DHAVE_BFD_DISASM=OFF" patching, I
> found this ticket which seems related:
> https://github.com/iovisor/bpftrace/issues/1106, but didn't make any
> significant progress yet.

I have tried every which way I can to make HAVE_BFD_DISASM work.  A kind
persn from the bpftrace irc directed me to this PR:
https://github.com/iovisor/bpftrace/pull/1095

But I cannot see anything guix does differently that would cause it to
fail. My only feeling is perhaps our configure flags for binutils might
be causing the issue.

As is, however, bpftrace does work even with out HAVE_BFD_DISASM and I
even used it to debug a few processes recently.

Thanks again!

- John

bug#42227: BPF in linux-libre

[Mathieu Othacehe]

Hey,

> That seems ok. I did find a few questions about debugfs on old irc logs
> and mailing lists.  My only concern again is that I would prefer to opt
> in to such a thing.  debugfs is much simpler than the bpf kernel flags
> though, so maybe it will be ok to remove in the future.

Yeah, but I saw that Ubuntu for instance is enabling it by default, so I
guess it could help to have the same behaviour in Guix System. Added it
with: 6bb07e91e1ab9367f636a3a5e9d52a9e0772aa89.

> But I cannot see anything guix does differently that would cause it to
> fail. My only feeling is perhaps our configure flags for binutils might
> be causing the issue.
>
> As is, however, bpftrace does work even with out HAVE_BFD_DISASM and I
> even used it to debug a few processes recently.

Gave it another try and I think if we could get "binutils" to produce a
dynamic version of libbfd.a, that would make the trick. Anyway, let's
proceed without BFD support for now. Pushed bpftrace as
c55acb073248392b1387017378f36a1d378fa7c4.

Closing the serie, thank you!

Mathieu