Packaging hwinfo

Packaging hwinfo

[phodina]

Dear All,

I'm attempting to package hwinfo which gathers HW information about the machine.
You'll find the package definition attached below together with the definition for libx86emu.

The issue I'm currently facing is that the libhd which is inside the repo is not detected by Makefile:

starting phase `build'
gcc -c -O2 -Wall -Wno-pointer-sign -pipe -g   -I/tmp/guix-build-hwinfo-21.75.drv-0/source/src/hd  -Lsrc hwinfo.c
make: *** No rule to make target '/tmp/guix-build-hwinfo-21.75.drv-0/source/src/libhd.a', needed by 'hwinfo'.  Stop.

The build continues but obviously fails when linking the final binary together. Does anybody have an idea how to tell make where to find the target?

Kind regards
Petr

---
(define-public libx86emu
(package
  (name "libx86emu")
  (version "3.1")
  (source (origin
            (method git-fetch)
            (uri
	      (git-reference
		(url "https://github.com/wfeldt/libx86emu")
		(commit version)))
	    (file-name (git-file-name name version))
            (sha256
             (base32
              "104xqc6nj9rpi7knl3dfqvasf087hlz2n5yndb1iycw35a6j509b"))))
  (build-system gnu-build-system)
  (native-inputs `(("git" ,git) ("perl" ,perl)))
  (arguments
    `(#:tests? #f
      #:make-flags (list (string-append "CC=" ,(cc-for-target)) (string-append "DESTDIR=" %output) (string-append "LIBDIR=/lib"))
      #:phases (modify-phases %standard-phases
		(delete 'configure)
		(add-after 'unpack 'fix-version-and-usr
                 (lambda* (#:key inputs #:allow-other-keys)
	          (delete-file "git2log")
		  (let* ((file (open-file "VERSION" "a")))
			(display ,version file)
			(close-port file))
                  (substitute* "Makefile"
                           (("/usr") "/"))
		   #t)))))
  (synopsis "x86 emulation library")
  (description "libx86emu is a small library to emulate x86 instructions. The focus here is not a complete emulation (go for qemu for this) but to cover enough for typical firmware blobs.
@enumerate
@item intercept any memory access or directly map real memory ranges
@item intercept any i/o access, map real i/o ports, or block any real i/o
@item intercept any interrupt
@item provides hook to run after each instruction
@item recognizes a special x86 instruction that can trigger logging
@item has integrated logging
@end enumerate")
  (home-page "https://github.com/wfeldt/libx86emu")
  (license license:bsd-1)))

(define-public hwinfo
(package
  (name "hwinfo")
  (version "21.75")
  (source (origin
            (method git-fetch)
            (uri
	      (git-reference
		(url "https://github.com/openSUSE/hwinfo")
		(commit version)))
	    (file-name (git-file-name name version))
            (sha256
             (base32
              "139bgzwi8iy1dz0g8mqpq9iig8klsmnb5c2sp0v7qgbgh7xxnqn3"))))
  (build-system gnu-build-system)
  (native-inputs `(("flex" ,flex) ("perl" ,perl) ("pkg-config" ,pkg-config)))
  (inputs `(("libx86emu" ,libx86emu) ("util-linux:lib" ,util-linux "lib")))
  (arguments
    `(#:tests? #f
      #:make-flags (list (string-append "DESTDIR=" %output) (string-append "CC=" ,(cc-for-target)) (string-append "HWINFO_VERSION=" ,version) (string-append "LIBDIR=" (assoc-ref %outputs "out") "/usr/lib"))
      #:phases (modify-phases %standard-phases
		(delete 'configure)
		(add-after 'unpack 'fix-sbin-and-flex
                 (lambda* (#:key inputs #:allow-other-keys)
	          (delete-file "git2log")
		  (let* ((file (open-file "VERSION" "a")))
			(display ,version file)
			(close-port file))
                  (substitute* "Makefile.common"
                           (("-I$(TOPDIR)/src/hd") (string-append (assoc-ref build-inputs "source") "/src/hd")))
                  (substitute* "Makefile"
                           (("/sbin") "/bin")
			   (("^TARGETS.*") "TARGETS = hwinfo hwinfo.pc\n")
			   (("/usr") "/"))
		  (substitute* "src/isdn/cdb/Makefile"
			   (("lex isdn_cdb.lex") "flex isdn_cdb.lex"))
		  (substitute* "hwinfo.pc.in"
			   (("prefix=/usr") (string-append "prefix=" %output)))
		   #t)))))
  (synopsis "Hardware information tool")
  (description "hwinfo/libhd are used to probe for the hardware present in the system. It can be used to generate a system overview log which can be later used for support.")
  (home-page "https://github.com/openSUSE/hwinfo")
  (license license:gpl2)))

Re: Packaging hwinfo

[Wiktor Żelazny]

[-- Attachment #1: Type: text/plain, Size: 1267 bytes --]

On Sat, Jul 24, 2021 at 03:56:33PM +0000, phodina wrote:

> The issue I'm currently facing is that the libhd which is inside the
> repo is not detected by Makefile:
>
> starting phase `build'
> gcc -c -O2 -Wall -Wno-pointer-sign -pipe -g   -I/tmp/guix-build-hwinfo-21.75.drv-0/source/src/hd  -Lsrc hwinfo.c
> make: *** No rule to make target '/tmp/guix-build-hwinfo-21.75.drv-0/source/src/libhd.a', needed by 'hwinfo'.  Stop.

Hi phodina,

Do you come across this issue also while compiling by hand, rather than
via the Guix daemon?

Here [1], you can see the same problem tackled by Gentoo community. It
was 2005 and related to another package, rather than directly hwinfo,
but maybe it would be worth taking a look at the current hwinfo Gentoo
recipe [2]. At some point, it says

   # Workaround from Arch, if using source tarballs from github

Here’s [3] how Arch folks do it. There’s this line

   CFLAGS+="-fPIC -I$srcdir/$pkgname/src/hd -w" \

Could it be related to libhd?

Good luck,

WŻ

[1]: https://bugs.gentoo.org/show_bug.cgi?id=60848#c3
[2]: https://gitweb.gentoo.org/repo/gentoo.git/tree/sys-apps/hwinfo/hwinfo-21.74.ebuild
[3]: https://github.com/archlinux/svntogit-community/blob/packages/hwinfo/trunk/PKGBUILD

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 963 bytes --]

Re: Packaging hwinfo

[phodina]

> Hi phodina,
>
> Do you come across this issue also while compiling by hand, rather than
>
> via the Guix daemon?
>
> Here [1], you can see the same problem tackled by Gentoo community. It
>
> was 2005 and related to another package, rather than directly hwinfo,
>
> but maybe it would be worth taking a look at the current hwinfo Gentoo
>
> recipe [2]. At some point, it says
>
> Workaround from Arch, if using source tarballs from github
> ==========================================================
>
> Here’s [3] how Arch folks do it. There’s this line
>
> CFLAGS+="-fPIC -I$srcdir/$pkgname/src/hd -w" \
>
> Could it be related to libhd?
>
> Good luck,
>
> WŻ
>
> [1]: https://bugs.gentoo.org/show_bug.cgi?id=60848#c3
>
> [2]: https://gitweb.gentoo.org/repo/gentoo.git/tree/sys-apps/hwinfo/hwinfo-21.74.ebuild
>
> [3]: https://github.com/archlinux/svntogit-community/blob/packages/hwinfo/trunk/PKGBUILD

Hello Wiktor,

Yes, this is due to the libhd. I probably didn't hightlight it in the description.

As you suggested I already had look at Arch (previous user).

I built it successfully on Arch using the PKGBUILD.
They pass the include flag with the dir -I . Without it points to $(TOPDIR)/src/libhd hence the substitution of as otherwise it expands into -I../../src/hd

I applied this flag but the build is still failing. Besides I also opened an issue on [1] though nobody probably even noticed :-(

 [1]: https://github.com/openSUSE/hwinfo/issues/103

Re: Packaging hwinfo

[Wiktor Żelazny]

[-- Attachment #1: Type: text/plain, Size: 4897 bytes --]

On Sat, Jul 24, 2021 at 06:20:18PM +0000, phodina wrote:

> > CFLAGS+="-fPIC -I$srcdir/$pkgname/src/hd -w" \
> >
> > Could it be related to libhd?
>
> Yes, this is due to the libhd. I probably didn't hightlight it in the
> description.

You described the problem very clearly, IMO. As “it”, I meant that line
from PKGBUILD.

> As you suggested I already had look at Arch (previous user).
>
> I built it successfully on Arch using the PKGBUILD.

Using pacman or abs, you mean? Can you build it from source by hand on
Guix?

> I applied this flag but the build is still failing.

Did you apply it while attempting to build hwinfo by hand?

As for your Guix package definition, I can see some substitution there.
I would try CFLAGS. Maybe you will find inspiration in the existing Guix
packages:

   $ grep FLAGS=-I gnu/packages/*.scm
   gnu/packages/astronomy.scm:                  (string-append "CPPFLAGS=-I" netpbm "/include/netpbm")
   gnu/packages/audio.scm:       #:make-flags (list (string-append "CFLAGS=-I"
   gnu/packages/audio.scm:       (list (string-append "-DCMAKE_CXX_FLAGS=-I"
   gnu/packages/cobol.scm:                               (string-append "JSON_C_CFLAGS=-I"
   gnu/packages/emulators.scm:         (list (string-append "CPPFLAGS=-I" (assoc-ref %build-inputs "sdl2")
   gnu/packages/fabric-management.scm:       (list (string-append "CPPFLAGS=-I" (assoc-ref %build-inputs "opensm")
   gnu/packages/game-development.scm:       (list (string-append "CFLAGS=-I"
   gnu/packages/games.scm:       (list (string-append "CXXFLAGS=-I"
   gnu/packages/games.scm:       (list (string-append "CPPFLAGS=-I"
   gnu/packages/games.scm:       (list (string-append "CPPFLAGS=-I"
   gnu/packages/games.scm:       (list (string-append "CPPFLAGS=-I"
   gnu/packages/games.scm:       (list (string-append "CFLAGS=-I"
   gnu/packages/games.scm:             (string-append "CXXFLAGS=-I"
   gnu/packages/games.scm:        (string-append "READLINE_CFLAGS=-I"
   gnu/packages/games.scm:                         (string-append "SDL_CPPFLAGS=-I"
   gnu/packages/guile-xyz.scm:         (list (string-append "CPPFLAGS=-I"
   gnu/packages/image.scm:             (string-append "CFLAGS=-I. -Icommon/include -Iimage/sys -fPIC "
   gnu/packages/image.scm:              (string-append "-DCMAKE_CXX_FLAGS=-I"
   gnu/packages/image-viewers.scm:             (string-append "CPPFLAGS=-I"
   gnu/packages/kde-frameworks.scm:       (list (string-append "-DCMAKE_CXX_FLAGS=-I"
   gnu/packages/kde-multimedia.scm:       (list (string-append "-DCMAKE_CXX_FLAGS=-I"
   gnu/packages/kde-multimedia.scm:              "-DCMAKE_CXX_FLAGS=-I"
   gnu/packages/kde.scm:             (string-append "-DCMAKE_CXX_FLAGS=-I"
   gnu/packages/libreoffice.scm:               (("GPGMEPP_CFLAGS=-I/usr")
   gnu/packages/libreoffice.scm:                (string-append "GPGMEPP_CFLAGS=-I"
   gnu/packages/libusb.scm:          (string-append "ID3TAG_CFLAGS=-I" libid3tag "/include")
   gnu/packages/linux.scm:          (string-append "libuuid_CFLAGS=-I" libuuid "/include/uuid")
   gnu/packages/linux.scm:          (string-append "libblkid_CFLAGS=-I" libuuid "/include/uuid "
   gnu/packages/maths.scm:                               (string-append "CPPFLAGS=-I"
   gnu/packages/messaging.scm:        (string-append "CFLAGS=-I"
   gnu/packages/networking.scm:             (string-append "CFLAGS=-I"
   gnu/packages/photo.scm:        (string-append "-DCMAKE_CXX_FLAGS=-I"
   gnu/packages/qt.scm:          (string-append "-DCMAKE_CXX_FLAGS=-I"
   gnu/packages/raspberry-pi.scm:                           "CFLAGS=-I../common -I../notc/include"))))
   gnu/packages/raspberry-pi.scm:                           "CFLAGS=-Ilib -I. -Iinclude -ffunction-sections -Wall -g -nostdlib -nostartfiles -ffreestanding -DBAREMETAL"))))
   gnu/packages/raspberry-pi.scm:                           "CFLAGS=-Iinclude -g"))))
   gnu/packages/search.scm:                                "CPPFLAGS=-I../src"
   gnu/packages/telephony.scm:                  ,(string-append "UCOMMON_CFLAGS=-I"
   gnu/packages/telephony.scm:                  ,(string-append "LIBOSIP2_CFLAGS=-I"
   gnu/packages/telephony.scm:                  ,(string-append "EXOSIP2_CFLAGS=-I"
   gnu/packages/video.scm:             (string-append "CPPFLAGS=-I"
   gnu/packages/virtualization.scm:              (string-append "CFLAGS=-I" (assoc-ref inputs "cross-libc")
   gnu/packages/web.scm:          (string-append "liblua_CFLAGS=-I" lua "/include")
   gnu/packages/xiph.scm:     `(#:configure-flags (list (string-append "CFLAGS=-I"

> Besides I also opened an issue on [1] though nobody probably even
> noticed :-(

I’m not sure if anyone there is going to be excited about working on an
issue which is Guix-specific. Some months ago, someone on this list
wrote “don’t bother upstream”.

Bye,

WŻ

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 963 bytes --]

Re: Packaging hwinfo

[Tobias Geerinckx-Rice]

Petr,

> Besides I also opened an issue

Thanks!  A good reflex, whether or not upstream cares or notices.

Please share packages in a form that can easily be built by others.  I 
added missing module imports and changed a forgotten /usr/lib to /lib.  
Please also place {native-,}inputs after arguments.  Phases need no 
longer return #t.

> CFLAGS+="-fPIC -I$srcdir/$pkgname/src/hd -w"

This does not look relevant, so I didn't add it anywhere.

> starting phase `build'
> gcc -c -O2 -Wall -Wno-pointer-sign -pipe -g
> -I/tmp/guix-build-hwinfo-21.75.drv-0/source/src/hd  -Lsrc hwinfo.c
> make: *** No rule to make target
> '/tmp/guix-build-hwinfo-21.75.drv-0/source/src/libhd.a', needed by
> 'hwinfo'.  Stop.
> 
> The build continues

And how:

make: *** Waiting for unfinished jobs....
make[1]: warning: jobserver unavailable: using -j1.  Add '+' to parent 
make rule.
make[1]: Entering directory 
'/tmp/guix-build-hwinfo-21.75.drv-0/source/src'
make[2]: Entering directory 
'/tmp/guix-build-hwinfo-21.75.drv-0/source/src/hd'

i.e.: "Oh no, you made some mistake, I can't make libhd!  Anyway, before 
I perish dramatically, let's chdir into this here 'src/hd' directory and 
make whatever's there. ¯\_( ツ )_/¯  I'm sure it won't be relevant lol."

That's one bug that only upstream can fix.  For now, set 
"#:parallel-build? #f" (with a detailed comment!) and you'll be on your 
merry, slow, way.  Alas:

> /gnu/store/74ldv6c74gslx5irsa0a8kn54yi4gc3h-hwinfo-21.75/bin/hwinfo:
> error: depends on 'libhd.so.21', which cannot be found in RUNPATH

Weird, but (search for '-Wl,-rpath' in Guix) it happens.  Let's--no, 
wait a minute, strange things are afoot:

tree /gnu/store/74ldv6c74gslx5irsa0a8kn54yi4gc3h-hwinfo-21.75
├── bin
│   ├── check_hd
│   ├── convert_hd
│   ├── getsysinfo
│   ├── hwinfo
│   └── mk_isdnhwdb
├── gnu
│   └── store
│       └── 74ldv6c74gslx5irsa0a8kn54yi4gc3h-hwinfo-21.75
│           └── lib
│               ├── libhd.so -> libhd.so.21
[...]

Afootness intensifies:

> (string-append "DESTDIR=" %output)

Seeing DESTDIR= is never a good sign (I didn't check whether its use in 
libx86emu is warranted or not).  Let's stick with hwinfo & peer into its 
Makefile again:

> ULIBDIR         = $(LIBDIR)
> [...]
> install -d -m 755 $(DESTDIR)/sbin $(DESTDIR)/usr/sbin \
>         $(DESTDIR)$(ULIBDIR)

Really?  No support for the standard $prefix variable?  Cool.

Making LIBDIR 'relative' works around that.  Not the rpath issue, so:

> "LIBDIR=/lib"
> (string-append "LDFLAGS=-Lsrc -Wl,-rpath=" %output "/lib")

The "-Lsrc" is repeated from upstream, because VARIABLE+= on the make 
command line doesn't actually work.

With that, it finally builds and runs.  Some software just isn't fun to 
package.

 From hwinfo:
> using /var/lib/hardware

This might need fixing.  There may be remaining issues with the 
package(s); I'm out of time.

 From the build log:
> info(FRITZ!Card PCMCIA): This card has multiple drivers, some functions 
> are only available on the binonly driver

We'll need to make sure this package doesn't include or 'promote' binary 
blobs in any way.

Thanks!

T G-R

Sent from a Web browser.  Excuse or enjoy my brevity.

Re: Packaging hwinfo

[phodina]

> Petr,
>
> > Besides I also opened an issue
>
> Thanks! A good reflex, whether or not upstream cares or notices.
>

Well my point here is that the issue concerning the project itself should be reported e.g. what are the instructions to build it by hand.
However, when comes to the packaging then it's obvious not a problem of the project itself and therefore I ask on this mailing list.

> Please share packages in a form that can easily be built by others. I
>
> added missing module imports and changed a forgotten /usr/lib to /lib.
>

Good point and my apologizes for that.

> Please also place {native-,}inputs after arguments. Phases need no
>
> longer return #t.
>

Thanks. Fixed.

> > CFLAGS+="-fPIC -I$srcdir/$pkgname/src/hd -w"
>
> This does not look relevant, so I didn't add it anywhere.
>
> > starting phase `build'
> >
> > gcc -c -O2 -Wall -Wno-pointer-sign -pipe -g
> >
> > -I/tmp/guix-build-hwinfo-21.75.drv-0/source/src/hd -Lsrc hwinfo.c
> >
> > make: *** No rule to make target
> >
> > '/tmp/guix-build-hwinfo-21.75.drv-0/source/src/libhd.a', needed by
> >
> > 'hwinfo'. Stop.
> >
> > The build continues
>
> And how:
>
> make: *** Waiting for unfinished jobs....
>
> make[1]: warning: jobserver unavailable: using -j1. Add '+' to parent
>
> make rule.
>
> make[1]: Entering directory
>
> '/tmp/guix-build-hwinfo-21.75.drv-0/source/src'
>
> make[2]: Entering directory
>
> '/tmp/guix-build-hwinfo-21.75.drv-0/source/src/hd'
>
> i.e.: "Oh no, you made some mistake, I can't make libhd! Anyway, before
>
> I perish dramatically, let's chdir into this here 'src/hd' directory and
>
> make whatever's there. ¯\( ツ )/¯ I'm sure it won't be relevant lol."
>
> That's one bug that only upstream can fix. For now, set
>
> "#:parallel-build? #f" (with a detailed comment!) and you'll be on your

Turning off the parallel build fixes the issue. But it's still a bug as the target src/libhd.a is "not relevant". I'll update the ticket on upstream with the findings. Thanks Tobias.

>
> merry, slow, way. Alas:
>
> > /gnu/store/74ldv6c74gslx5irsa0a8kn54yi4gc3h-hwinfo-21.75/bin/hwinfo:
> >
> > error: depends on 'libhd.so.21', which cannot be found in RUNPATH
>
> Weird, but (search for '-Wl,-rpath' in Guix) it happens. Let's--no,
>
> wait a minute, strange things are afoot:
>
> tree /gnu/store/74ldv6c74gslx5irsa0a8kn54yi4gc3h-hwinfo-21.75
>
> ├── bin
>
> │   ├── check_hd
>
> │   ├── convert_hd
>
> │   ├── getsysinfo
>
> │   ├── hwinfo
>
> │   └── mk_isdnhwdb
>
> ├── gnu
>
> │   └── store
>
> │   └── 74ldv6c74gslx5irsa0a8kn54yi4gc3h-hwinfo-21.75
>
> │   └── lib
>
> │   ├── libhd.so -> libhd.so.21
>
> [...]
>
> Afootness intensifies:
>
> > (string-append "DESTDIR=" %output)
>
> Seeing DESTDIR= is never a good sign (I didn't check whether its use in
>
> libx86emu is warranted or not). Let's stick with hwinfo & peer into its

Nope, but then again let's create a ticket upstream with the fix.

>
> Makefile again:
>
> > ULIBDIR = $(LIBDIR)
> >
> > [...]
> >
> > install -d -m 755 $(DESTDIR)/sbin $(DESTDIR)/usr/sbin \
> >
> > $(DESTDIR)$(ULIBDIR)
>
> Really? No support for the standard $prefix variable? Cool.
>
> Making LIBDIR 'relative' works around that. Not the rpath issue, so:
>
> > "LIBDIR=/lib"
> >
> > (string-append "LDFLAGS=-Lsrc -Wl,-rpath=" %output "/lib")
>
> The "-Lsrc" is repeated from upstream, because VARIABLE+= on the make
>
> command line doesn't actually work.
>
> With that, it finally builds and runs. Some software just isn't fun to
>
> package.

It's a tedious process that often overlooked in distros where all stuff is already packaged.

>
> From hwinfo:
>
> > using /var/lib/hardware
>
> This might need fixing. There may be remaining issues with the
>
> package(s); I'm out of time.
>

Thanks for your time! I made the modification and now it builds. Time to prepare the patches.

> From the build log:
>
> > info(FRITZ!Card PCMCIA): This card has multiple drivers, some functions
> >
> > are only available on the binonly driver
>
> We'll need to make sure this package doesn't include or 'promote' binary
>
> blobs in any way.
>

What do you mean by striping binary blobs?
Don't know exactly how to spot them yet :-)