From mboxrd@z Thu Jan 1 00:00:00 1970 From: =?UTF-8?B?QmrDtnJuIEjDtmZsaW5n?= Subject: How I got stuck (and ways to resolve) Date: Thu, 20 Dec 2018 16:03:00 +0100 Message-ID: <20181220160300.1140d486@alma-ubu> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; boundary="Sig_/DdXV1LsakN+wWh3xCmJlfps"; protocol="application/pgp-signature" Return-path: Received: from eggs.gnu.org ([2001:4830:134:3::10]:48974) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gZzr0-0001De-JL for guix-devel@gnu.org; Thu, 20 Dec 2018 10:03:15 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gZzqv-00080k-IX for guix-devel@gnu.org; Thu, 20 Dec 2018 10:03:14 -0500 Received: from m4s11.vlinux.de ([83.151.27.109]:51344 helo=bjoernhoefling.de) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gZzqr-0007vn-72 for guix-devel@gnu.org; Thu, 20 Dec 2018 10:03:06 -0500 List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+gcggd-guix-devel=m.gmane.org@gnu.org Sender: "Guix-devel" To: guix-devel --Sig_/DdXV1LsakN+wWh3xCmJlfps Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Hi Guix, the task for mentors in this Outreachy week #3 is to give an example of how one got stuck in a problem, and show ways to go on and finally succeed with your problem. So, I write a bit about my problems and encourage everyone else to tell their story of small or big stucks.=20 Our intern in this round is doing video documentation, so her tasks are a bit different than "normal" contributions, anyway I will report about packaging problems as this is what I mostly did for Guix and where I really got stuck and I think everyone of us gets stuck quickly with this. When packaging a new software for Guix, I start with documenting the process: I create a new MarkDown file (I will be very sloppy with the syntax), write the date and package I want to pack and start. Mostly, this is "write-only": I just write down my activity, my problems and the solution. I rarely read that again. But it helps me structure the task and continue on when I put it aside for a while. In general, my problem is that I know nothing (or: always too little) about makefiles, about C and C++, about CMake, Python, Qt, Ruby, and all the rest: I'm a Java expert and I know nothing about all these "strange" languages and their "brainfucked" error messages. So, one of the first "solutions" is to keep calm and read the error message. Read it again. Try to parse it. What is the problem? From which program/compiler/tool did it came? From which dependency? Is really THIS the problem, or is it caused by something ELSE somewhere above? In which build phase did it occur (configure, compile, test, ...)? Let's dig into one from opencv, don't look too much into the details of the error message: ---8<---unaltered-citation::start--8<------------------------------- [...] Now it compiles. But then I stumble over a nasty build error: ``` cd /tmp/guix-build-opencv-3.4.0.drv-0/build/modules/video && /gnu/store/5sv= 5zy2k gg6iaqyv8zw49w4243j0xkd0-gcc-5.4.0/bin/c++ -DCVAPI_EXPORTS -D_USE_MATH_DE= FINES -D__OPENCV_BUILD=3D1 -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__S= TDC_LIM IT_MACROS -I/tmp/guix-build-opencv-3.4.0.drv-0/build -I/tmp/guix-build-open= cv-3. 4.0.drv-0/opencv-3.4.0/modules/video/include -I/tmp/guix-build-opencv-3.4.0= .drv- 0/opencv-3.4.0/modules/video/src -I/tmp/guix-build-opencv-3.4.0.drv-0/build= /modu les/video -I/tmp/guix-build-opencv-3.4.0.drv-0/opencv-3.4.0/modules/core/in= clude -I/tmp/guix-build-opencv-3.4.0.drv-0/opencv-3.4.0/modules/imgproc/include = -fsi gned-char -W -Wall -Werror=3Dreturn-type -Werror=3Dnon-virtual-dtor -Werror= =3Daddress=20 -Werror=3Dsequence-point -Wformat -Werror=3Dformat-security -Wmissing-decla= rations - Wundef -Winit-self -Wpointer-arith -Wshadow -Wsign-promo -Wuninitialized -W= init- self -Wno-narrowing -Wno-delete-non-virtual-dtor -Wno-comment -fdiagnostics= -show -option -Wno-long-long -pthread -fomit-frame-pointer -ffunction-sections -f= data- sections -msse -msse2 -msse3 -fvisibility=3Dhidden -fvisibility-inlines-hi= dden -O 2 -g -DNDEBUG -fPIC -Winvalid-pch -include "/tmp/guix-build-opencv-3.4.= 0.drv -0/build/modules/video/precomp.hpp" -o CMakeFiles/opencv_video.dir/src/ecc.= cpp.o -c /tmp/guix-build-opencv-3.4.0.drv-0/opencv-3.4.0/modules/video/src/ecc.c= pp In file included from /tmp/guix-build-opencv-3.4.0.drv-0/opencv-3.4.0/modul= es/imgcodecs/src/grfmt_exr.hpp:52:0, from /tmp/guix-build-opencv-3.4.0.drv-0/opencv-3.4.0/modul= es/imgcodecs/src/grfmts.hpp:53, from /tmp/guix-build-opencv-3.4.0.drv-0/opencv-3.4.0/modul= es/imgcodecs/src/loadsave.cpp:47: /gnu/store/kikj95f44ygrp3fapd1yybykxl167i0l-openexr-2.2.1/include/OpenEXR/I= mfChromaticities.h:46:22: fatal error: ImathVec.h: No such file or directory compilation terminated. make[2]: *** [modules/imgcodecs/CMakeFiles/opencv_imgcodecs.dir/build.make:= 66: modules/imgcodecs/CMakeFiles/opencv_imgcodecs.dir/src/loadsave.cpp.o] E= rror 1 make[2]: Leaving directory '/tmp/guix-build-opencv-3.4.0.drv-0/build' make[1]: *** [CMakeFiles/Makefile2:4057: modules/imgcodecs/CMakeFiles/openc= v_imgcodecs.dir/all] Error 2 make[1]: *** Waiting for unfinished jobs.... ``` Bug reports maybe related: * https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=3D865862 * https://github.com/ampas/CTL/issues/15 But somehow not. While I was executing `find /gnu/store -name ImathVec.h`, I had that idea: = Look again at the Guix definition of *openexv*. And yes: There is also that package `ilmbase`. And here we go: ``` ./6knqzzds4fp6y4qrzyg2ppx6qmiil7jr-ilmbase-2.2.1/include/OpenEXR/ImathVec.h ``` So, let's add that to the inputs, too! Failed again. Would habe been too nice to work. I found something in graphics.scm: blender has also a workaround for it: ``` #:phases (modify-phases %standard-phases (add-after 'set-paths 'add-ilmbase-include-path (lambda* (#:key inputs #:allow-other-keys) ;; OpenEXR propagates ilmbase, but its include files do not ap= pear ;; in the CPATH, so we need to add "$ilmbase/include/OpenEXR/"= to ;; the CPATH to satisfy the dependency on "ImathVec.h". (setenv "CPATH" (string-append (assoc-ref inputs "ilmbase") "/include/OpenEXR" ":" (or (getenv "CPATH") ""))) #t))))) ``` Yes, that's it. Going to the next error: ---8<---unaltered-citation::end--8<------------------------------- * This was during the compile phase * You see this "Error 2", which is not the error. * The error is above "Error 1", but if you compile in parallel, there could be some garbage in between. Be aware to always search up for "Error 1". * Some header file could not be found. * I didn't know what to do. * So I googled for it. * I found some reports which where not too helpful * Out of desparation, I brute-force searched the store for it. * I somehow had the idea to look closer at/around package "openexv" and found "ilmbase" as the root package. * With that hint, I just added that dependency too. * OK, it would have been too nice if it worked directly... * I searched around and could find another package using it that prepared a CPATH. Copy&paste solved that problem finally. * Surprise, the very next problem is directly coming! What is always annoying are failing tests. What's even more annoying is when you notice that they fail on upstream too and nobody cares. Here is one example out of my log: ---8<---unaltered-citation::start--8<------------------------------- I found out that opencv is using Google Test framework. And you can disable individual tests by adding "DISABLED_" in front of the test name https://github.com/google/googletest/blob/master/googletest/docs/AdvancedGu= ide.md chapter: Temporarily Disabling Tests ``` For example, the following tests won't be run by Google Test, even though t= hey will still be compiled: // Tests that Foo does Abc. TEST(FooTest, DISABLED_DoesAbc) { ... } ``` ---8<---unaltered-citation::end--8<------------------------------- So, here OpenCV is using a specific test framework and I first had to find out how to disable tests in it. In the final package definition, this introduced a new package phase that disables the tests with a substitute* construct to add the "DISABLED_" macro in the C file. Some general ways to resolve: * Look exactly at the error message. * Google it. Is it fixed somewhere else? * Look at packages that use the dependency that causes the problem. Do they have a solution? * Is there a newer version of your software? * Is there documentation about the dependencies? * Maybe you need a SPECIFIC version of one library to build your package? Try to up/downgrade that dependency. * Write down your problem. Be very specific and write it in a way down that others can reproduce it. * Sometimes, just writing helps already. If not, send it out to the mailing list to ask for help. And finally: * Take a break. * Drink a tea. * Get back when you are fresh and concentrated. And if all doesn't help: * Get sarcastic. You are not alone with your problems: There is at least Kenneth Hoste with his great talk "How To Make Package Managers Cry" from FOSDEM 2018: https://archive.fosdem.org/2018/schedule/event/how_to_make_package_managers= _cry/ Bj=C3=B6rn --Sig_/DdXV1LsakN+wWh3xCmJlfps Content-Type: application/pgp-signature Content-Description: OpenPGP digital signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iEYEARECAAYFAlwbryUACgkQvyhstlk+X/0FUACaA1y18/iI2ejPaLnPm5uwUE7e rbMAn2mQEaWkzh89PC2IXfB2QL5xDDbU =VkIh -----END PGP SIGNATURE----- --Sig_/DdXV1LsakN+wWh3xCmJlfps--