Packaging Jami progress

Packaging Jami progress

[Jan Wielkiewicz]

Hello everyone,

With some help from both you and Jami developers, I finally managed
to build pjproject-jami, which means the hardest task has been
already done! Now I have to package restinio, which replaced restbed in
Jami. I have already a sketch of the package, but I encountered a
problem I don't really know the meaning nor know how to fix it:

CMake Error at CMakeLists.txt:84 (add_subdirectory):
  add_subdirectory given source "fmt" which is not an existing directory.


-- Found OpenSSL: /gnu/store/k2m4q2av9hw73hw2jx6qrxqdyh855398-openssl-1.1.1c/lib/libcrypto.so (found version "1.1.1c")  
OpenSSL include dir: /gnu/store/k2m4q2av9hw73hw2jx6qrxqdyh855398-openssl-1.1.1c/include
OpenSSL libraries: /gnu/store/k2m4q2av9hw73hw2jx6qrxqdyh855398-openssl-1.1.1c/lib/libssl.so;/gnu/store/k2m4q2av9hw73hw2jx6qrxqdyh855398-openssl-1.1.1c/lib/libcrypto.so
-- Found PCRE: /gnu/store/5j6w0x3aq0i5r9565w92lrh016vlmv2d-pcre-8.43/lib/libpcre.so  
-- PCRE_LIBRARIES='/gnu/store/5j6w0x3aq0i5r9565w92lrh016vlmv2d-pcre-8.43/lib/libpcre.so'
-- PCRE_INCLUDE_DIRS='/gnu/store/5j6w0x3aq0i5r9565w92lrh016vlmv2d-pcre-8.43/include'
-- Found PCRE2: /gnu/store/4jmnzz60hrm7k1lmn4565sh7xh447q1z-pcre2-10.33/lib/libpcre2-8.so  
-- PCRE2_LIBRARIES='/gnu/store/4jmnzz60hrm7k1lmn4565sh7xh447q1z-pcre2-10.33/lib/libpcre2-8.so'
-- PCRE2_INCLUDE_DIRS='/gnu/store/4jmnzz60hrm7k1lmn4565sh7xh447q1z-pcre2-10.33/include'
-- PCRE2_LIBRARIES='/gnu/store/4jmnzz60hrm7k1lmn4565sh7xh447q1z-pcre2-10.33/lib/libpcre2-8.so'
-- PCRE2_INCLUDE_DIRS='/gnu/store/4jmnzz60hrm7k1lmn4565sh7xh447q1z-pcre2-10.33/include'
CMake Error at CMakeLists.txt:124 (add_subdirectory):
  add_subdirectory given source "so_5" which is not an existing directory.


-- Found ZLIB: /gnu/store/qx7p7hiq90mi7r78hcr9cyskccy2j4bg-zlib-1.2.11/lib/libz.so (found version "1.2.11") 
-- ZLIB_LIBRARIES='/gnu/store/qx7p7hiq90mi7r78hcr9cyskccy2j4bg-zlib-1.2.11/lib/libz.so'
-- ZLIB_INCLUDE_DIRS='/gnu/store/qx7p7hiq90mi7r78hcr9cyskccy2j4bg-zlib-1.2.11/include'
-- Looking for pthread.h
-- Looking for pthread.h - found
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Failed
-- Check if compiler accepts -pthread
-- Check if compiler accepts -pthread - yes
-- Found Threads: TRUE  
-- Configuring incomplete, errors occurred!
See also "/tmp/guix-build-restinio-0.6.0.1.drv-0/source/build/CMakeFiles/CMakeOutput.log".
See also "/tmp/guix-build-restinio-0.6.0.1.drv-0/source/build/CMakeFiles/CMakeError.log".
command "cmake" "../dev" "-DCMAKE_BUILD_TYPE=RelWithDebInfo" "-DCMAKE_INSTALL_PREFIX=/gnu/store/1xjcw3icsrmrzjikbqslr11ihrsz22nj-restinio-0.6.0.1" "-DCMAKE_INSTALL_LIBDIR=lib" "-DCMAKE_INSTALL_RPATH_USE_LINK_PATH=TRUE" "-DCMAKE_INSTALL_RPATH=/gnu/store/1xjcw3icsrmrzjikbqslr11ihrsz22nj-restinio-0.6.0.1/lib" "-DCMAKE_VERBOSE_MAKEFILE=ON" failed with status 1


The package:

(define-public restinio
  (package
    (name "restinio")
    (version "0.6.0.1")
    (source (origin
              (method git-fetch)
              (uri (git-reference
                    (url "https://github.com/Stiffstream/restinio.git")
                    (commit (string-append "v." version))))
              (file-name (git-file-name name version))
              (sha256
               (base32
                "1c25kpx652nng8m1sqf5an2c3c4g3k6zj85mkkaxzk88iwfzq1s8"))))
    (build-system cmake-build-system)
    (inputs
     `(("zlib", zlib)
       ("openssl", openssl)
       ("boost", boost)
       ("asio", asio)
       ("pcre", pcre)
       ("pcre2", pcre2)))
    (propagated-inputs
     `())
    (native-inputs
     `())
    (arguments
     `(#:configure-flags '()
       #:phases
       (modify-phases %standard-phases
		      (add-after 'unpack 'change-directory
				 (lambda _
				   (chdir "dev")
				   #t)))))
    (home-page "https://stiffstream.com/en/products/restinio.html")
    (synopsis "C++14 library that gives you an embedded HTTP/Websocket
server")
    (description "RESTinio is a header-only C++14 library that gives
you an embedded
HTTP/Websocket server. It is based on standalone version of ASIO
and targeted primarily for asynchronous processing of HTTP-requests.")
    (license license:bsd-3)))


I created a new thread, because the old one was too long, which is not
helpful for anyone.


Jan Wielkiewicz

Re: Packaging Jami progress

[Gábor Boskovits]

[-- Attachment #1: Type: text/plain, Size: 5573 bytes --]

Hello,

Jan Wielkiewicz <tona_kosmicznego_smiecia@interia.pl> ezt írta (időpont:
2019. nov. 4., H, 21:48):

> Hello everyone,
>
> With some help from both you and Jami developers, I finally managed
> to build pjproject-jami, which means the hardest task has been
> already done! Now I have to package restinio, which replaced restbed in
> Jami. I have already a sketch of the package, but I encountered a
> problem I don't really know the meaning nor know how to fix it:
>
> CMake Error at CMakeLists.txt:84 (add_subdirectory):
>   add_subdirectory given source "fmt" which is not an existing directory.
>
>
This is most probably because fmt is missing from inputs.


> -- Found OpenSSL:
> /gnu/store/k2m4q2av9hw73hw2jx6qrxqdyh855398-openssl-1.1.1c/lib/libcrypto.so
> (found version "1.1.1c")
> OpenSSL include dir:
> /gnu/store/k2m4q2av9hw73hw2jx6qrxqdyh855398-openssl-1.1.1c/include
> OpenSSL libraries:
> /gnu/store/k2m4q2av9hw73hw2jx6qrxqdyh855398-openssl-1.1.1c/lib/libssl.so;/gnu/store/k2m4q2av9hw73hw2jx6qrxqdyh855398-openssl-1.1.1c/lib/libcrypto.so
> -- Found PCRE:
> /gnu/store/5j6w0x3aq0i5r9565w92lrh016vlmv2d-pcre-8.43/lib/libpcre.so
> --
> PCRE_LIBRARIES='/gnu/store/5j6w0x3aq0i5r9565w92lrh016vlmv2d-pcre-8.43/lib/libpcre.so'
> --
> PCRE_INCLUDE_DIRS='/gnu/store/5j6w0x3aq0i5r9565w92lrh016vlmv2d-pcre-8.43/include'
> -- Found PCRE2:
> /gnu/store/4jmnzz60hrm7k1lmn4565sh7xh447q1z-pcre2-10.33/lib/libpcre2-8.so
> --
> PCRE2_LIBRARIES='/gnu/store/4jmnzz60hrm7k1lmn4565sh7xh447q1z-pcre2-10.33/lib/libpcre2-8.so'
> --
> PCRE2_INCLUDE_DIRS='/gnu/store/4jmnzz60hrm7k1lmn4565sh7xh447q1z-pcre2-10.33/include'
> --
> PCRE2_LIBRARIES='/gnu/store/4jmnzz60hrm7k1lmn4565sh7xh447q1z-pcre2-10.33/lib/libpcre2-8.so'
> --
> PCRE2_INCLUDE_DIRS='/gnu/store/4jmnzz60hrm7k1lmn4565sh7xh447q1z-pcre2-10.33/include'
> CMake Error at CMakeLists.txt:124 (add_subdirectory):
>   add_subdirectory given source "so_5" which is not an existing directory.
>
> This is because SObjectizer is missing from inputs.
You can get further info about this in the cmake file:
https://github.com/Stiffstream/restinio/blob/master/dev/CMakeLists.txt

>
> -- Found ZLIB:
> /gnu/store/qx7p7hiq90mi7r78hcr9cyskccy2j4bg-zlib-1.2.11/lib/libz.so (found
> version "1.2.11")
> --
> ZLIB_LIBRARIES='/gnu/store/qx7p7hiq90mi7r78hcr9cyskccy2j4bg-zlib-1.2.11/lib/libz.so'
> --
> ZLIB_INCLUDE_DIRS='/gnu/store/qx7p7hiq90mi7r78hcr9cyskccy2j4bg-zlib-1.2.11/include'
> -- Looking for pthread.h
> -- Looking for pthread.h - found
> -- Performing Test CMAKE_HAVE_LIBC_PTHREAD
> -- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Failed
> -- Check if compiler accepts -pthread
> -- Check if compiler accepts -pthread - yes
> -- Found Threads: TRUE
> -- Configuring incomplete, errors occurred!
> See also
> "/tmp/guix-build-restinio-0.6.0.1.drv-0/source/build/CMakeFiles/CMakeOutput.log".
> See also
> "/tmp/guix-build-restinio-0.6.0.1.drv-0/source/build/CMakeFiles/CMakeError.log".
> command "cmake" "../dev" "-DCMAKE_BUILD_TYPE=RelWithDebInfo"
> "-DCMAKE_INSTALL_PREFIX=/gnu/store/1xjcw3icsrmrzjikbqslr11ihrsz22nj-restinio-0.6.0.1"
> "-DCMAKE_INSTALL_LIBDIR=lib" "-DCMAKE_INSTALL_RPATH_USE_LINK_PATH=TRUE"
> "-DCMAKE_INSTALL_RPATH=/gnu/store/1xjcw3icsrmrzjikbqslr11ihrsz22nj-restinio-0.6.0.1/lib"
> "-DCMAKE_VERBOSE_MAKEFILE=ON" failed with status 1
>
>
> The package:
>
> (define-public restinio
>   (package
>     (name "restinio")
>     (version "0.6.0.1")
>     (source (origin
>               (method git-fetch)
>               (uri (git-reference
>                     (url "https://github.com/Stiffstream/restinio.git")
>

This does not seem the upstream repo. Could you change it to the upstream
one? (Albeit, it seems to be an official mirror.)
(https://bitbucket.org/sobjectizerteam/restinio/src/default/)
Or if there is a tarball available for the needed version, that would be
even better (

https://bitbucket.org/sobjectizerteam/restinio/downloads/), but I
don't know if these archives are stable.

                    (commit (string-append "v." version))))
>               (file-name (git-file-name name version))
>               (sha256
>                (base32
>                 "1c25kpx652nng8m1sqf5an2c3c4g3k6zj85mkkaxzk88iwfzq1s8"))))
>     (build-system cmake-build-system)
>     (inputs
>      `(("zlib", zlib)
>        ("openssl", openssl)
>        ("boost", boost)
>        ("asio", asio)
>        ("pcre", pcre)
>        ("pcre2", pcre2)))
>     (propagated-inputs
>      `())
>     (native-inputs
>      `())
>     (arguments
>      `(#:configure-flags '()
>        #:phases
>        (modify-phases %standard-phases
>                       (add-after 'unpack 'change-directory
>                                  (lambda _
>                                    (chdir "dev")
>                                    #t)))))
>     (home-page "https://stiffstream.com/en/products/restinio.html")
>     (synopsis "C++14 library that gives you an embedded HTTP/Websocket
> server")
>     (description "RESTinio is a header-only C++14 library that gives
> you an embedded
> HTTP/Websocket server. It is based on standalone version of ASIO
> and targeted primarily for asynchronous processing of HTTP-requests.")
>     (license license:bsd-3)))
>
>
> I created a new thread, because the old one was too long, which is not
> helpful for anyone.
>
>
> Jan Wielkiewicz
>
>
Best regards,
g_bor
-- 
OpenPGP Key Fingerprint: 7988:3B9F:7D6A:4DBF:3719:0367:2506:A96C:CF63:0B21

[-- Attachment #2: Type: text/html, Size: 7659 bytes --]

Re: Packaging Jami progress

[Jan Wielkiewicz]

Dnia 2019-11-04, o godz. 23:48:00
Gábor Boskovits <boskovits@gmail.com> napisał(a):

> Hello,
> 
> This is most probably because fmt is missing from inputs.
> > This is because SObjectizer is missing from inputs.
> You can get further info about this in the cmake file:
> https://github.com/Stiffstream/restinio/blob/master/dev/CMakeLists.txt

I packaged SObjectizer and placed it in the gnu/packages/cpp.scm file.
I also added SObjectizer and fmt as inputs, but the result is the same,
even though I tried removing commands, which add the subdirectories
from CMakeLists.txt. What's the proper way of dealing with situations
like this? Am I missing something important? Should I copy outputs of
fmt and sobjectizer into the directories? If so, how can I do this?

(add-after 'unpack 'do-not-make-directories
				 (lambda _
				   (substitute* "dev/CMakeLists.txt"
						(("add_subdirectory(fmt)")
"") (("add_subdirectory(so_5)") ""))
				   #t))

> 
> This does not seem the upstream repo. Could you change it to the
> upstream one? (Albeit, it seems to be an official mirror.)
> (https://bitbucket.org/sobjectizerteam/restinio/src/default/)
> Or if there is a tarball available for the needed version, that would
> be even better (
> 
> https://bitbucket.org/sobjectizerteam/restinio/downloads/), but I
> don't know if these archives are stable.

On the website, they say the software is hosted on github:
https://stiffstream.com/en/products/restinio.html

Also AFAIK git is always reproducible, whereas tarballs can get
rebuilt, so fetching using git is safer.


Jan Wielkiewicz

Re: Packaging Jami progress

[Gábor Boskovits]

[-- Attachment #1: Type: text/plain, Size: 2392 bytes --]

Hello,

Jan Wielkiewicz <tona_kosmicznego_smiecia@interia.pl> ezt írta (időpont:
2019. nov. 5., Ke 17:50):

> Dnia 2019-11-04, o godz. 23:48:00
> Gábor Boskovits <boskovits@gmail.com> napisał(a):
>
> > Hello,
> >
> > This is most probably because fmt is missing from inputs.
> > > This is because SObjectizer is missing from inputs.
> > You can get further info about this in the cmake file:
> > https://github.com/Stiffstream/restinio/blob/master/dev/CMakeLists.txt
>
> I packaged SObjectizer and placed it in the gnu/packages/cpp.scm file.
> I also added SObjectizer and fmt as inputs, but the result is the same,
> even though I tried removing commands, which add the subdirectories
> from CMakeLists.txt. What's the proper way of dealing with situations
> like this? Am I missing something important? Should I copy outputs of
> fmt and sobjectizer into the directories? If so, how can I do this?
>
The inputs should be detected by cmake, and subdirs should only be used if
not found. You could add a phase with copy-recursive if needed. I am not
sure that we can't just simply copy the header into the output, but it's
true that we would lose the tests then.

>
> (add-after 'unpack 'do-not-make-directories
>                                  (lambda _
>                                    (substitute* "dev/CMakeLists.txt"
>                                                 (("add_subdirectory(fmt)")
> "") (("add_subdirectory(so_5)") ""))
>                                    #t))
>
> >
> > This does not seem the upstream repo. Could you change it to the
> > upstream one? (Albeit, it seems to be an official mirror.)
> > (https://bitbucket.org/sobjectizerteam/restinio/src/default/)
> > Or if there is a tarball available for the needed version, that would
> > be even better (
> >
> > https://bitbucket.org/sobjectizerteam/restinio/downloads/), but I
> > don't know if these archives are stable.
>
> On the website, they say the software is hosted on github:
> https://stiffstream.com/en/products/restinio.html

Nice... and in the documentation section on the same site they say the
mercurial repository is the upstream, and the github one is the mirror...

>
>
> Also AFAIK git is always reproducible, whereas tarballs can get
> rebuilt, so fetching using git is safer.
>
>
> Jan Wielkiewicz
>
Best regards, g_bor

>

[-- Attachment #2: Type: text/html, Size: 4062 bytes --]

Re: Packaging Jami progress

[Pierre Neidhardt]

[-- Attachment #1: Type: text/plain, Size: 132 bytes --]

If you are still stuck, share your progress for restinio, I'll give it a
shot! :)

-- 
Pierre Neidhardt
https://ambrevar.xyz/

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

Re: Packaging Jami progress

[Jan Wielkiewicz]

[-- Attachment #1: Type: text/plain, Size: 617 bytes --]

Dnia 2019-11-06, o godz. 11:30:19
Pierre Neidhardt <mail@ambrevar.xyz> napisał(a):

> If you are still stuck, share your progress for restinio, I'll give
> it a shot! :)
> 

Yes please, I didn't have time nor knowledge to finish this.
Let me know if you'll fix this. I also have to reread the packaging
tutorial, because last time I checked it wasn't that fancy as the
current one in the cookbook is.
I send all patches, because I'm not a git wizard yet.
Do I have to add copyright lines or something? Of course I license my
work under the GPL v3 or any later as the project uses.


Jan Wielkiewicz

[-- Attachment #2: jami-patches-wip-06.11.2019.tar.bz2 --]
[-- Type: application/x-bzip, Size: 6932 bytes --]

Re: Packaging Jami progress

[Pierre Neidhardt]

[-- Attachment #1: Type: text/plain, Size: 661 bytes --]

Thanks, I'll look into it.

> Yes please, I didn't have time nor knowledge to finish this.
> Let me know if you'll fix this. I also have to reread the packaging
> tutorial, because last time I checked it wasn't that fancy as the
> current one in the cookbook is.

Actually... It's should be the same content! ;)

> I send all patches, because I'm not a git wizard yet.
> Do I have to add copyright lines or something? Of course I license my
> work under the GPL v3 or any later as the project uses.

No problem, I'll add your copyright lines wherever needed.

Cheers and thanks for the hard work!

-- 
Pierre Neidhardt
https://ambrevar.xyz/

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

Re: Packaging Jami progress

[Pierre Neidhardt]

[-- Attachment #1: Type: text/plain, Size: 2351 bytes --]

Hi Jan,

Here is a quick 'n' dirty restinio package:

--8<---------------cut here---------------start------------->8---
(define-public restinio
  (package
    (name "restinio")
    (version "0.6.0.1")
    (source (origin
              (method git-fetch)
              (uri (git-reference
                    (url "https://github.com/Stiffstream/restinio.git")
                    (commit (string-append "v." version))))
              (file-name (git-file-name name version))
              (sha256
               (base32
                "1c25kpx652nng8m1sqf5an2c3c4g3k6zj85mkkaxzk88iwfzq1s8"))))
    (build-system cmake-build-system)
    (inputs                             ; TODO: Need to force-keep references on some inputs, e.g. boost.
     `(("zlib" ,zlib)
       ("catch2" ,catch-framework2)
       ("openssl" ,openssl)
       ("fmt" ,fmt)
       ("boost" ,boost)
       ;;("asio", asio) ; TODO: Use external asio?  Need -DRESTINIO_USE_BOOST_ASIO_VALUES=shared.
       ("pcre" ,pcre)
       ("pcre2" ,pcre2)
       ("sobjectizer" ,sobjectizer)))
    (arguments
     `(#:configure-flags '("-DRESTINIO_INSTALL=on")
       #:tests? #f ; TODO: The tests are called from the root CMakelist, need RESTINIO_TEST=on.
       #:phases
       (modify-phases %standard-phases
         (add-after 'unpack 'change-directory
           (lambda _
             (chdir "dev/restinio")
             #t)))))
    (home-page "https://stiffstream.com/en/products/restinio.html")
    (synopsis "C++14 library that gives you an embedded HTTP/Websocket server")
    (description "RESTinio is a header-only C++14 library that gives you an embedded
HTTP/Websocket server.  It is based on standalone version of ASIO
and targeted primarily for asynchronous processing of HTTP-requests.")
    (license license:bsd-3)))
--8<---------------cut here---------------end--------------->8---

The CMakeLists are a bit convoluted, so I simply skipped the root one
and when straight into the restinio subfolder.

I'm too lazy to figure out how to run the tests at the moment.

Note: the comma "," in Scheme means it "unquotes" the following
expression, and thus the space must be put before it, not after.

Wrong:

  ("boost", boost)

Correct:

  ("boost" ,boost)


Cheers!

-- 
Pierre Neidhardt
https://ambrevar.xyz/

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

Re: Packaging Jami progress

[Jan Wielkiewicz]

Dnia 2019-11-07, o godz. 20:02:08
Pierre Neidhardt <mail@ambrevar.xyz> napisał(a):

> Hi Jan,
> 
> Here is a quick 'n' dirty restinio package:
Cool, thanks!

> 
> The CMakeLists are a bit convoluted, so I simply skipped the root one
> and when straight into the restinio subfolder.
>
> I'm too lazy to figure out how to run the tests at the moment.
As far as it works, we can skip this now. All I need is compiling Jami,
then I'll improve the quality of the package definitions.

> 
> Note: the comma "," in Scheme means it "unquotes" the following
> expression, and thus the space must be put before it, not after.
> 
> Wrong:
> 
>   ("boost", boost)
> 
> Correct:
> 
>   ("boost" ,boost)
Right, this explains a lot. This is my brain and Scheme - I actually
read what it is and how it works, I use it in guile repl, but somehow
my brain had treated this comma as if it was C++.
Same thing with the fancy version of the Cookbook :)

> 
> Cheers!
>

Thanks for everything, I'll check how it works with the Jami package
tomorrow.


Jan Wielkiewicz

Re: Packaging Jami progress

[Jan]

Hi again,

Sorry for being impatient, but is it normal for patches to be merged
that long? Is there something stopping the commits?
I need those merged in order to continue working on Jami.


Jan Wielkiewicz

Re: Packaging Jami progress

[Pierre Neidhardt]

[-- Attachment #1: Type: text/plain, Size: 225 bytes --]

I was waiting for people to test the patch, since it does not run
properly on my machine (the UI does not show up).

Have you tried it yourself, Jan?  Does ti work for you?

-- 
Pierre Neidhardt
https://ambrevar.xyz/

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

Re: Packaging Jami progress

[Jan]

On Tue, 26 Nov 2019 11:07:28 +0100
Pierre Neidhardt <mail@ambrevar.xyz> wrote:

> I was waiting for people to test the patch, since it does not run
> properly on my machine (the UI does not show up).
> 
> Have you tried it yourself, Jan?  Does ti work for you?
> 

That's the chicken and the egg problem - I can't try it until it gets
merged - if our changes cause webkit-gtk to rebuild, I can't go any
further without the substitute, because my machine is underpowered and
can't compile the package. I couldn't find any time to install Guix
System on my external hard drive for my more powerful machine, but if
it's necessary to proceed, I'll do it as fast as I can. I'll also ask
the developers if they have any clue what could be the cause.

There could be also a more recent change in Jami - they moved the
webchat from the gnome-client to libringclient - they made it
cross-platform with Qt-something.

P.S. I apply the patches using "patch" command, right?

Jan Wielkiewicz

Re: Packaging Jami progress

[Pierre Neidhardt]

[-- Attachment #1: Type: text/plain, Size: 269 bytes --]

WebKitGTK is not rebuilt with my patch, because I removed the GnuTLS
update.
You can test with substitutes, it should work with no problem.

> P.S. I apply the patches using "patch" command, right?

With `git am`.

-- 
Pierre Neidhardt
https://ambrevar.xyz/

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

Re: Packaging Jami progress

[zimoun]

Hi Jan,

On Tue, 26 Nov 2019 at 20:43, Jan <tona_kosmicznego_smiecia@interia.pl> wrote:

> P.S. I apply the patches using "patch" command, right?

What do you use?

If you use Emacs, you can open Debbugs with: C-u M-x debbugs-gnu then
RET guix-patches n y
Then M-x debbugs-gnu-bugs RET 38211 RET
So far so good.
Select the patch set.
Then M-x shell-command-on-region RET cd /path/to/your/guix/clone && git am RET

Or you can select the patch set then press the two letters O b and it
will download the patches.
Then inside your favorite terminal session: git am < /path/to/patches/*

If you do not use Emacs, you can download the series on the web page
(mbox) but I have never done without Emacs. ;-)


Note that the series will not apply with a fresh "git pull" because
there a conflict with the file cpp.scm. You need to rebase, as I asked
to Pierre. ;-)

Hope that helps.


All the best,
simon

Re: Packaging Jami progress

[Jan]

On Wed, 27 Nov 2019 12:43:19 +0100
zimoun <zimon.toutoune@gmail.com> wrote:

> Hi Jan,
Hello,
> If you use Emacs, you can open Debbugs with: C-u M-x debbugs-gnu then
> RET guix-patches n y
> Then M-x debbugs-gnu-bugs RET 38211 RET
> So far so good.
> Select the patch set.
> Then M-x shell-command-on-region RET cd /path/to/your/guix/clone &&
> git am RET
>
> Or you can select the patch set then press the two letters O b and it
> will download the patches.
> Then inside your favorite terminal session: git am
> < /path/to/patches/*
Well, I actually use Emacs, but not as my OS, just as my text editor :)
It really shows Emacs is a better platform than GNU/Linux, if instead
of using tools provided by the OS, people use Emacs. I have a plan for
fixing this, but it'll take me some time to bring the idea to life. For
now I can use Emacs then :P

> If you do not use Emacs, you can download the series on the web page
> (mbox) but I have never done without Emacs. ;-)
That's fun, tried this and it failed. I'll try downloading just the
mails with the patches for Jami.

> 
> Note that the series will not apply with a fresh "git pull" because
> there a conflict with the file cpp.scm. You need to rebase, as I asked
> to Pierre. ;-)
Rebase? Should I use a different checkout? Sorry, but this git thing
is still a dark magic for me. 

> Hope that helps.
> 
> 
> All the best,
> simon

Jan Wielkiewicz

Re: Packaging Jami progress

[Pierre Neidhardt]

[-- Attachment #1: Type: text/plain, Size: 1059 bytes --]

Jan <tona_kosmicznego_smiecia@interia.pl> writes:

>> Note that the series will not apply with a fresh "git pull" because
>> there a conflict with the file cpp.scm. You need to rebase, as I asked
>> to Pierre. ;-)
> Rebase? Should I use a different checkout? Sorry, but this git thing
> is still a dark magic for me. 

"Rebasing" means "reapply some changes on top of a new commit."

It's an important concept to synchronize changes when working with
multiple people.

The (1)git-rebase man page has some good examples with the drawings,
this could be helpful to understand how it works.

If you don't want to use Emacs and Magit, the man page will give you the
right command line invocation to rebase your branch onto master.

If you decide to go with Emacs and Magit (possibly the easiest way), you
can "rebase the <branch> onto master".  This should trigger a conflict,
which you can then resolve by pressing "e" (ediff) on the unmerged
(i.e. conflicting) changes.

Hope this helps!

-- 
Pierre Neidhardt
https://ambrevar.xyz/

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

Re: Packaging Jami progress

[Jan]

I see the changes got merged. Tested Jami and it works great :)
I guess the wrapper was the issue.
I am going to update it to the latest version, this time I'll try
sending the patches myself in order to learn something new.
I also told the devs about our success:
https://git.jami.net/savoirfairelinux/ring-project/issues/691
Should I ask them to add the option for Guix on this page?
https://jami.net/download-jami-linux/
I mean, do we consider the package to be more or less ready, so they
can show it?


Jan Wielkiewicz

Re: Packaging Jami progress

[Pierre Neidhardt]

[-- Attachment #1: Type: text/plain, Size: 95 bytes --]

If it works for you, then I think it's ready!

-- 
Pierre Neidhardt
https://ambrevar.xyz/

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

Re: Packaging Jami progress

[Jan]

On Sun, 01 Dec 2019 18:32:36 +0100
Pierre Neidhardt <mail@ambrevar.xyz> wrote:

> If it works for you, then I think it's ready!
I'll do some more tests - video/audio conference, but for now
everything works - chat, sending files.

As for updating, Sébastien Blin said that since then they had bumped
opendht and had patched ffmpeg:
https://git.jami.net/savoirfairelinux/ring-project/issues/691#note_16723

The first should be easy, but the second probably requires adding
ffmpeg-jami (if I'm correct).


Jan Wielkiewicz

Re: Packaging Jami progress

[Jan Wielkiewicz]

Hello,
I started working on updating Jami to the latest version and it seems
it needs libnatpmp, because without it, compilation fails during doing
something connected to UPnP.
For that purpose I started packaging libnatpmp, but during the "install"
stage, it fails with the following error:
starting phase `install'
install -p -d /usr/include
install: cannot create directory ‘/usr’: Permission denied
make: *** [Makefile:95: install] Error 1
command "make" "install"
"prefix=/gnu/store/rn7h6irrjfcd5w2s7a1clrq91g8jxjhl-libnatpmp-20150609"
failed with status 2

I tried chmoding and making files writable, but it didn't work or I did
something wrong.
Here's the sketch of the package:

(define-public libnatpmp
  (package
    (name "libnatpmp")
    (version "20150609")
    (source (origin
              (method url-fetch)
              (uri (string-append
		    "http://miniupnp.free.fr/files/"
		    name "-" version ".tar.gz"))
              (sha256
               (base32
                "1c1n8n7mp0amsd6vkz32n8zj3vnsckv308bb7na0dg0r8969rap1"))))
    (build-system gnu-build-system)
    (arguments
     `(#:phases
       (modify-phases %standard-phases
		      (delete 'configure)
		      (delete 'check))
       #:make-flags
       (list (string-append "prefix=" (assoc-ref %outputs "out")))))
    (home-page "http://miniupnp.free.fr/libnatpmp.html")
    (synopsis "C Library implementing NAT-PMP")
    (description
     "libnatpmp is a portable and asynchronous implementaiton of the NAT Port Mapping Protocol (NAT-PMP) written in C.")
    (license license:bsd-3)))

How do we deal with problems like these? I checked the makefile and it
doesn't seem to have the "/usr" path hardcoded - it has the $(PREFIX)
variable.


Jan Wielkiewicz

Re: Packaging Jami progress

[Pierre Neidhardt]

[-- Attachment #1: Type: text/plain, Size: 475 bytes --]

Hi!

I suppose that if you deleted the 'configure phase, it's because there
is no ./configure file.
Is there a bootstrap file?

You probably want to use "PREFIX=" (in all caps).
Also see if the Makefile uses the DESTDIR variable, in which case you
might want to set it to the default output instead of PREFIX.

Look for PREFIX= and DESTDIR= in the Guix packages repository, you'll
find lots of examples.

Cheers!

-- 
Pierre Neidhardt
https://ambrevar.xyz/

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

Re: Packaging Jami progress

[Jan]

On Tue, 03 Dec 2019 17:04:39 +0100
Pierre Neidhardt <mail@ambrevar.xyz> wrote:

> Hi!
> 
> I suppose that if you deleted the 'configure phase, it's because there
> is no ./configure file.
> Is there a bootstrap file?
There's no ./configure, nor ./bootstrap.

> You probably want to use "PREFIX=" (in all caps).
> Also see if the Makefile uses the DESTDIR variable, in which case you
> might want to set it to the default output instead of PREFIX.
> Look for PREFIX= and DESTDIR= in the Guix packages repository, you'll
> find lots of examples.
There's no DESTDIR, but changing "prefix=" to "PREFIX=" fixed the build.
Is this always the case, or is it dependent on how the makefile is
written? I saw an example with "prefix=" in the cookbook. What about
mentioning it in the cookbook?

Eventually it turns out this package wasn't breaking the build, but
libupnp was. I updated it to 1.8.3 (from 1.6.something), changed the
fetch metod from url-fetch from sourceforge (which is being considered
legacy by its developers(that's what they say on the website)) to
git-fetch from github (which is now an official repository). I also
added autotools, etc. because the package seems to require it now.

Anyway, building the latest Jami version works now. I'll send the
patches in my free (as in freedom) time :)
> Cheers!
> 


Jan Wielkiewicz

Re: Packaging Jami progress

[Pierre Neidhardt]

[-- Attachment #1: Type: text/plain, Size: 687 bytes --]

Jan <tona_kosmicznego_smiecia@interia.pl> writes:

> There's no DESTDIR, but changing "prefix=" to "PREFIX=" fixed the build.
> Is this always the case, or is it dependent on how the makefile is
> written? I saw an example with "prefix=" in the cookbook.

Yes, it depends on the Makefile.  The variable names are purely
conventional.
Automake normalizes some variable names, but since it's not used here
anything can happen.

> What about mentioning it in the cookbook?

Sure, will do!

> Anyway, building the latest Jami version works now. I'll send the
> patches in my free (as in freedom) time :)

Fantastic, thanks!

-- 
Pierre Neidhardt
https://ambrevar.xyz/

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

Re: Packaging Jami progress

[Pierre Neidhardt]

[-- Attachment #1: Type: text/plain, Size: 365 bytes --]

Pierre Neidhardt <mail@ambrevar.xyz> writes:

> Sure, will do!

Hmm, just looked at the cookbook and the `prefix=` example does not talk
about the variable itself, so it does not seem like a right fit to
discuss about Makefile variables.

Feel free to send a patch if you can come up with a good suggestion.

-- 
Pierre Neidhardt
https://ambrevar.xyz/

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

Re: Packaging Jami progress

[Jan Wielkiewicz]

Hi,
It turns out ffmpeg needs to be patched, just like pjproject and there
could be more packages to patch. I quickly searched for "*.patch" in
the contrib folder and that's what I got:
./argon2/pkgconfig.patch
./argon2/argon2-vs2017.patch
./pthreads/pthreads-uwp.patch
./pthreads/pthreads-vs2017.patch
./pthreads/pthreads-windows.patch
./vpx/windows-vpx_config.patch
./gnutls/no-create-time-h.patch
./gnutls/mac-keychain-lookup.patch
./gnutls/gnutls-uwp.patch
./gnutls/read-file-limits.h.patch
./gnutls/gnutls-mscver.patch
./gnutls/gnutls-disable-getentropy-osx.patch
./gnutls/gnutls-3.6.7-win32-vs-support.patch
./gnutls/gnutls-no-egd.patch
./gnutls/gnutls-3.6.7-win32-compat.patch
./gnutls/downgrade-gettext-requirement.patch
./gpg-error/gpgerror-android.patch
./gpg-error/missing-unistd-include.patch
./gpg-error/windres-make.patch
./ffmpeg/remove-mjpeg-log.patch
./ffmpeg/windows-configure-libmfx.patch
./ffmpeg/change-RTCP-ratio.patch
./ffmpeg/rtp_ext_abs_send_time.patch
./ffmpeg/windows-configure-ffnvcodec.patch
./ffmpeg/windows-configure.patch
./gmp/gmp_addaddmul_1msb0.patch
./gmp/clock_gettime.patch
./libressl/getpagesize.patch
./libressl/0001-build-don-t-fetch-git-tag-if-openbsd-directory-exist.patch
./iconv/libiconv-win64.patch
./iconv/bins.patch
./iconv/libiconv-winrt.patch
./iconv/libiconv-android-ios.patch
./iconv/win32.patch
./restinio/cmake.patch
./jack/config-osx.patch
./natpmp/natpmp-win32-ssize_t.patch
./x264/0001-use-internal-log2f.patch
./x264/remove-align.patch
./x264/x264-uwp.patch
./asio/no_tests_examples.patch
./asio/asio-vcxproj.patch
./asio/asio-uwp.patch
./uuid/android.patch
./gcrypt/0001-Fix-assembly-division-check.patch
./gcrypt/fix-amd64-assembly-on-solaris.patch
./yaml-cpp/cmake.patch
./media-sdk/windows-static-lib-build.patch
./gsm/include_ios.patch
./gsm/gsm-cross.patch
./upnp/threadpool.patch
./upnp/win_inet_pton.patch
./upnp/libupnp-ipv6.patch
./upnp/miniserver.patch
./upnp/libupnp-windows.patch
./jsoncpp/jsoncpp-vs2017.patch
./pjproject/sip_config.patch
./pjproject/ice_config.patch
./pjproject/fix_first_packet_turn_tcp.patch
./pjproject/disable_local_resolution.patch
./pjproject/fix_ioqueue_ipv6_sendto.patch
./pjproject/rfc2466.patch
./pjproject/fix_assert_on_connection_attempt.patch
./pjproject/pj_ice_sess.patch
./pjproject/fix_turn_connection_failure.patch
./pjproject/multiple_listeners.patch
./pjproject/rfc6544.patch
./pjproject/ignore_ipv6_on_transport_check.patch
./pjproject/uwp_vs.patch
./pjproject/win_vs2017_props.patch
./pjproject/ipv6.patch
./pjproject/win32_vs_gnutls.patch
./pjproject/add_dtls_transport.patch
./pjproject/android.patch
./pjproject/fix_turn_alloc_failure.patch
./pjproject/fix_turn_fallback.patch
./pjproject/fix_ebusy_turn.patch
./pjproject/win_config.patch
./http_parser/http_parser-vs.patch
./openssl/openssl-uwp.patch
./portaudio/dsound_utf8.patch
./portaudio/pa-dsound.patch
./portaudio/pa-vs2017.patch
./portaudio/pa-dsound-aecns.patch
./portaudio/pa-uwp.patch
./dbus-cpp/dbus-c++-writechar.patch
./dbus-cpp/dbus-c++-gcc4.7.patch
./dbus-cpp/dbus-c++-threading.patch
./secp256k1/secp256k1-vs2017.patch

It would be good If there was a procedure (or a macro?) accepting the
name of the dependency and the list of patches, simmilar to this:

(lambda* (#:key inputs #:allow-other-keys)
             (let ((savoir-faire-linux-patches-directory "Savoir-faire Linux patches")
                   (savoir-faire-linux-patches
                    '("fix_turn_alloc_failure"
                      "rfc2466"
                      "ipv6"
                      "multiple_listeners"
                      "pj_ice_sess"
                      "fix_turn_fallback"
                      "fix_ioqueue_ipv6_sendto"
                      "add_dtls_transport"
                      "rfc6544"
                      "ice_config"
                      "sip_config"
                      "fix_first_packet_turn_tcp"
                      "fix_ebusy_turn"
                      "ignore_ipv6_on_transport_check"
                      "fix_turn_connection_failure"
                      "disable_local_resolution")))
               (mkdir-p savoir-faire-linux-patches-directory)
               (invoke "tar" "-xvf" (assoc-ref inputs "savoir-faire-linux-patches")
                       "-C" savoir-faire-linux-patches-directory
                       "--strip-components=5"
                       "ring-project/daemon/contrib/src/pjproject")
               (for-each
                (lambda (file)
                  (invoke "patch" "--force" "-p1" "-i"
                          (string-append savoir-faire-linux-patches-directory "/"
                                         file ".patch")))
                savoir-faire-linux-patches))
             #t)

The question is - if I defined a procedure named "jami-apply-patches" or
whatever, will importing it using #:use-module (gnu packages telephony
jami-apply-patches) work, or do I need to explicitly export it?
Also how to add such procedure to the "add-after" field without causing
an error?


Jan Wielkiewicz

Re: Packaging Jami progress

[Pierre Neidhardt]

[-- Attachment #1: Type: text/plain, Size: 616 bytes --]

> The question is - if I defined a procedure named "jami-apply-patches" or
> whatever, will importing it using #:use-module (gnu packages telephony
> jami-apply-patches) work, or do I need to explicitly export it?

You need to export the symbol.
To do so, you can either specify the symbol in the #:export part of the
module at the top of the file, or simply use `define-public` when
defining the variable.

> Also how to add such procedure to the "add-after" field without causing
> an error?

Can you give an example?  I don't understand what you mean.

-- 
Pierre Neidhardt
https://ambrevar.xyz/

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

Re: Packaging Jami progress

[Jan Wielkiewicz]

Dnia 2019-12-04, o godz. 16:27:26
Pierre Neidhardt <mail@ambrevar.xyz> napisał(a):

> You need to export the symbol.
> To do so, you can either specify the symbol in the #:export part of
> the module at the top of the file, or simply use `define-public` when
> defining the variable.
Okay, thanks.

> Can you give an example?  I don't understand what you mean.
I would like to have something like this:

(define-public jami-apply-dependency-patches
  (lambda* (#:key inputs patches dependency-name #:allow-other-keys)
	   (let ((savoir-faire-linux-patches-directory "Savoir-faire Linux patches"))
	     (mkdir-p savoir-faire-linux-patches-directory)
	     (invoke "tar" "-xvf" (assoc-ref inputs "savoir-faire-linux-patches")
		     "-C" savoir-faire-linux-patches-directory
		     "--strip-components=5"
		     (string-append "ring-project/daemon/contrib/src/"
				    dependency-name))
	     (for-each
	      (lambda (file)
		(invoke "patch" "--force" "-p1" "-i"
			(string-append savoir-faire-linux-patches-directory "/"
				       file ".patch")))
	      patches))
	   #t))

And then invoke it like this
(add-after 'unpack 'apply-patches (jami-apply-dependency-patches
#:dependency-name "pjproject" #:patches '(*the list*) #:inputs inputs)

I know it's a bit vague, don't really know what happens here with this
lambda*. I would like to pass some arguments there, but if I understand
this correctly, Guix does it for me - I give it a procedure and it
executes it.


Thanks in advance
Jan Wielkiewicz

Re: Packaging Jami progress

[Pierre Neidhardt]

[-- Attachment #1: Type: text/plain, Size: 1875 bytes --]

Jan Wielkiewicz <tona_kosmicznego_smiecia@interia.pl> writes:

> Dnia 2019-12-04, o godz. 16:27:26
> Pierre Neidhardt <mail@ambrevar.xyz> napisał(a):
>
>> You need to export the symbol.
>> To do so, you can either specify the symbol in the #:export part of
>> the module at the top of the file, or simply use `define-public` when
>> defining the variable.
> Okay, thanks.
>
>> Can you give an example?  I don't understand what you mean.
> I would like to have something like this:
>
> (define-public jami-apply-dependency-patches
>   (lambda* (#:key inputs patches dependency-name #:allow-other-keys)
> 	   (let ((savoir-faire-linux-patches-directory "Savoir-faire Linux patches"))
> 	     (mkdir-p savoir-faire-linux-patches-directory)
> 	     (invoke "tar" "-xvf" (assoc-ref inputs "savoir-faire-linux-patches")
> 		     "-C" savoir-faire-linux-patches-directory
> 		     "--strip-components=5"
> 		     (string-append "ring-project/daemon/contrib/src/"
> 				    dependency-name))
> 	     (for-each
> 	      (lambda (file)
> 		(invoke "patch" "--force" "-p1" "-i"
> 			(string-append savoir-faire-linux-patches-directory "/"
> 				       file ".patch")))
> 	      patches))
> 	   #t))
>
> And then invoke it like this
> (add-after 'unpack 'apply-patches (jami-apply-dependency-patches
> #:dependency-name "pjproject" #:patches '(*the list*) #:inputs inputs)

You need a lambda here:

--8<---------------cut here---------------start------------->8---
(add-after 'unpack 'apply-patches
  (lambda* (#:key inputs #:allow-other-keys)
    (let ((my-input (assoc-ref inputs "my-input")))
      (jami-apply-dependency-patches #:inputs my-input))))
--8<---------------cut here---------------end--------------->8---
 
See the other packages for many more examples.
Does that make sense?

-- 
Pierre Neidhardt
https://ambrevar.xyz/

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

Re: Packaging Jami progress

[Jan]

On Wed, 04 Dec 2019 17:06:16 +0100
Pierre Neidhardt <mail@ambrevar.xyz> wrote:

> You need a lambda here:
> 
> --8<---------------cut here---------------start------------->8---
> (add-after 'unpack 'apply-patches
>   (lambda* (#:key inputs #:allow-other-keys)
>     (let ((my-input (assoc-ref inputs "my-input")))
>       (jami-apply-dependency-patches #:inputs my-input))))
> --8<---------------cut here---------------end--------------->8---
>  
> See the other packages for many more examples.
> Does that make sense?
> 

Okay, thanks.
There's a problem with the procedure though - mkdir-p, etc. are
unbound in the procedure. Tried using #:use-modules (guix build
utils), but the compiler said that "which" was imported from both
"packages base" and "build utils". If that was a package I could use
(modules '((guix build system))), but it isn't. Using #:prefix doesn't
seem to be a good way of handling this. Can I remove the unnecessary
"which"?
Sorry for asking so many questions, the whole thing is huge...
Hope one day I'll be able to do everything myself.

Jan Wielkiewicz

Re: Packaging Jami progress

[Pierre Neidhardt]

[-- Attachment #1: Type: text/plain, Size: 102 bytes --]

Can you share your current patch of the whole thing?

-- 
Pierre Neidhardt
https://ambrevar.xyz/

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

Re: Packaging Jami progress

[Jan Wielkiewicz]

[-- Attachment #1: Type: text/plain, Size: 164 bytes --]

Dnia 2019-12-04, o godz. 18:01:39
Pierre Neidhardt <mail@ambrevar.xyz> napisał(a):

> Can you share your current patch of the whole thing?
> 

Here you go.

[-- Attachment #2: jami-patches-04-12-2019.tar.bz2 --]
[-- Type: application/x-bzip, Size: 3985 bytes --]

Re: Packaging Jami progress

[Pierre Neidhardt]

[-- Attachment #1: Type: text/plain, Size: 1155 bytes --]

Thanks!

The first 5 commits look good overall.
For the last one, I think there are a few confusions:

- native-inputs takes packages or origins, it does not take a lambda.

- To call a custom function from the builder, you need to include the
  module that defines it for the build system, e.g.

  (arguments
    `(#:modules ((gnu packages telephony)
                 ,@%gnu-build-system-modules)
      ...)

- In this particular case, you could even make a macro that is expanded
  at definition time, so that you would not even need to import the module.
  If you don't know how to write scheme macros, no need to go down this
  road for now.

But before fixing this issue, what are you trying to do exactly?
Do you intend to reuse this `jami-apply-dependency-patches' procedure
somewhere else?


On a different topic, to avoid sending patch archives in the future, you
could use a public repository clone of Guix.
There are a couple of free services out there, like sr.ht, maybe
gogs.io, etc.
This would make it smoother to track your progress and help you in the process.

-- 
Pierre Neidhardt
https://ambrevar.xyz/

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

Re: Packaging Jami progress

[Jan]

On Thu, 05 Dec 2019 15:32:23 +0100
Pierre Neidhardt <mail@ambrevar.xyz> wrote:

> Thanks!
> 
> The first 5 commits look good overall.
Good to know.

> For the last one, I think there are a few confusions:
> 
> - native-inputs takes packages or origins, it does not take a lambda.
> 
> - To call a custom function from the builder, you need to include the
>   module that defines it for the build system, e.g.
> 
>   (arguments
>     `(#:modules ((gnu packages telephony)
>                  ,@%gnu-build-system-modules)
>       ...)
> 
> - In this particular case, you could even make a macro that is
> expanded at definition time, so that you would not even need to
> import the module. If you don't know how to write scheme macros, no
> need to go down this road for now.
I read about macros, but didn't write anything complicated myself. I
would have to reread the manual again.

> But before fixing this issue, what are you trying to do exactly?
> Do you intend to reuse this `jami-apply-dependency-patches' procedure
> somewhere else?
Jami developers apply many patches for different projects -
currently our package code only patches pjproject, but ffmpeg needs
patches for instance for auto bitrate support and I don't really know
what has to be patched next - I listed the output of "find -name
"*.patch"" in the first mail.
I think copying and pasting similar code from pjproject package
definition to ffmpeg definition isn't a good practice, that's why I
decided to create a procedure taking patches from
ring-project/daemon/contrib/<dependency-name> and applying them.

> 
> On a different topic, to avoid sending patch archives in the future,
> you could use a public repository clone of Guix.
> There are a couple of free services out there, like sr.ht, maybe
> gogs.io, etc.
> This would make it smoother to track your progress and help you in
> the process.

I'll check this out, thanks.


Jan Wielkiewicz

Re: Packaging Jami progress

[Pierre Neidhardt]

[-- Attachment #1: Type: text/plain, Size: 762 bytes --]

Jan <tona_kosmicznego_smiecia@interia.pl> writes:

> Jami developers apply many patches for different projects -
> currently our package code only patches pjproject, but ffmpeg needs
> patches for instance for auto bitrate support and I don't really know
> what has to be patched next - I listed the output of "find -name
> "*.patch"" in the first mail.
> I think copying and pasting similar code from pjproject package
> definition to ffmpeg definition isn't a good practice, that's why I
> decided to create a procedure taking patches from
> ring-project/daemon/contrib/<dependency-name> and applying them.

OK I get it now, thanks!
You are right, it's a good idea to have such a procedure then.


-- 
Pierre Neidhardt
https://ambrevar.xyz/

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

Re: Packaging Jami progress

[Jan Wielkiewicz]

Dnia 2019-12-05, o godz. 15:32:23
Pierre Neidhardt <mail@ambrevar.xyz> napisał(a):

> - To call a custom function from the builder, you need to include the
>   module that defines it for the build system, e.g.
> 
>   (arguments
>     `(#:modules ((gnu packages telephony)
>                  ,@%gnu-build-system-modules)
>       ...)
> 
Tried luck with the procedure, but it seems I can't provide the module
I'm currently in to the "arguments" field - Guile said it couldn't find
code for the module: "no code for module (gnu packages jami)". I mean
in the jami.scm file, defining (gnu packages jami) module,
I can't give this module as an argument.
    (arguments
     `(#:modules ((gnu packages jami)
		   ,@%gnu-build-system-modules)
    ...)

By the way, I decided to move all Jami packages, procedures and
dependencies modified by the patches to a separate file -
jami.scm. It'll be much easier to maintain in this way.

The full error message:

The following derivation will be built:
   /gnu/store/an28ny48d6iyfn79j2fhqk7mvd6jmahq-pjproject-jami-2.9.drv
building
/gnu/store/an28ny48d6iyfn79j2fhqk7mvd6jmahq-pjproject-jami-2.9.drv...
Backtrace: 10 (primitive-load "/gnu/store/dj185gjiag94gk1clrj0158xcal?")
In ice-9/eval.scm:
   721:20  9 (primitive-eval (begin (use-modules (gnu # jami) # ?) ?))
In ice-9/psyntax.scm:
  1262:36  8 (expand-top-sequence ((begin (use-modules (gnu ?) ?) ?)) ?)
  1209:24  7 (parse _ (("placeholder" placeholder)) ((top) #(# # ?)) ?)
  1209:24  6 (parse _ (("placeholder" placeholder)) ((top) #(# # ?)) ?)
   285:10  5 (parse _ (("placeholder" placeholder)) (()) _ c&e (eval) ?)
In ice-9/boot-9.scm:
  3377:20  4 (process-use-modules _)
   222:17  3 (map1 (((gnu packages jami)) ((guix build #)) ((# ?)) ?))
  3378:31  2 (_ ((gnu packages jami)))
   2803:6  1 (resolve-interface _ #:select _ #:hide _ #:prefix _ # _ ?)
In unknown file:
           0 (scm-error misc-error #f "~A ~S" ("no code for modu?" ?) ?)

ERROR: In procedure scm-error:
no code for module (gnu packages jami)

Is there some kind of "this" for the module in guile I could use in
order to be able to invoke the jami-apply-dependency-patches procedure?
I exported it correctly using #:export (jami-apply-dependency-patches).


Jan Wielkiewicz

Re: Packaging Jami progress

[Pierre Neidhardt]

[-- Attachment #1: Type: text/plain, Size: 133 bytes --]

Sorry, I don't know how to import a procedure defined in the same file :(
Anyone?

-- 
Pierre Neidhardt
https://ambrevar.xyz/

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

Re: Packaging Jami progress

[Caleb Ristvedt]

[-- Attachment #1: Type: text/plain, Size: 1834 bytes --]

#:modules and #:imported-modules are distinct arguments. #:modules is the
modules that your builder is going to use (as in "they go in a (use-modules
...) form"), while #:imported-modules is the modules that need to be
available
in the build environment. It's complaining at build-time that it can't find
that
module to use, because you haven't told it to include that module in the
build
environment. #:imported-modules should give a superset of what #:modules
gives,
especially if a module in use is going to have indirect
requirements. Thankfully, wrangling together those indirect requirements is
already implemented in (guix modules) as source-module-closure.

Thus, you could conceptually do


(arguments
     `(#:imported-modules (,@(source-module-closure
                  '((gnu packages jami)
                ,@%gnu-build-system-modules)))
       #:modules ((gnu packages jami)
          ,@(@@ (guix build-system gnu) %default-modules))))

And in theory it would work. Note, though, that this would pull in the
entire
module dependency graph of (gnu packages jami), and this may include things
that
source-module-closure would have a hard time detecting and aren't really
needed. Ideally this procedure would be generalized and put in (guix build
<something>), but I can understand if that's not possible. Note also that
you
could simply splice in the definition of your procedure into the builder
manually, like so:

(define my-procedure-code '(lambda (a b c) ...))

(arguments
 `(#:phases (let ((my-procedure ,my-procedure-code)) (modify-phases ...))))

Hope that helps, thanks for the work on Jami y'all!

On Tue, Dec 10, 2019 at 8:57 AM Pierre Neidhardt <mail@ambrevar.xyz> wrote:

> Sorry, I don't know how to import a procedure defined in the same file :(
> Anyone?
>
> --
> Pierre Neidhardt
> https://ambrevar.xyz/
>

[-- Attachment #2: Type: text/html, Size: 2427 bytes --]

Re: Packaging Jami progress

[Pierre Neidhardt]

[-- Attachment #1: Type: text/plain, Size: 441 bytes --]

Wow, thanks for the explanation, it's very enlightening!

This should probably end up in the documentation somewhere.  Maybe as
part of the packaging tutorial?  Or for the long-due "Advanced Packaging
Tutorial"?


> (define my-procedure-code '(lambda (a b c) ...))
>
> (arguments
>  `(#:phases (let ((my-procedure ,my-procedure-code)) (modify-phases ...))))

This is a great tip!

-- 
Pierre Neidhardt
https://ambrevar.xyz/

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

Re: Packaging Jami progress

[Jan Wielkiewicz]

Hi!

I tried both ways - the second works, but the first doesn't.
That's what I have in the file - if I didn't miss something, it is the
same as your example:

#:imported-modules (,@(source-module-closure
	'((gnu packages jami)
	,@%gnu-build-system-modules)))
#:modules ((gnu packages jami)
	,@(@@ (guix build-system gnu) %default-modules))

And I get an ugly backtrace (could this be a bug or am I doing
something wrong?):

The following derivations will be built:
   /gnu/store/xf6b58rlki7sb3k9fj2dxkm4ljiypdc0-pjproject-jami-2.9.drv
   /gnu/store/aaqaz52fhjb86g233ar4ynnyjvrv7xa7-module-import-compiled.drv
building
/gnu/store/aaqaz52fhjb86g233ar4ynnyjvrv7xa7-module-import-compiled.drv...
Backtrace: In ice-9/eval.scm:
   721:20 19 (primitive-eval _)
In ice-9/psyntax.scm:
  1262:36 18 (expand-top-sequence _ _ _ #f _ _ _)
  1209:24 17 (parse _ (("placeholder" placeholder)) ((top) #(# # ?)) ?)
   285:10 16 (parse _ (("placeholder" placeholder)) (()) _ c&e (eval) ?)
In ice-9/eval.scm:
   293:34 15 (_ #<module (#{ g172}#) 7ffff488f5a0>)
In ice-9/boot-9.scm:
   2874:4 14 (define-module* _ #:filename _ #:pure _ #:version _ # _ ?)
  2887:24 13 (_)
   222:29 12 (map1 _)
   222:29 11 (map1 _)
   222:29 10 (map1 _)
   222:29  9 (map1 _)
   222:29  8 (map1 _)
   222:29  7 (map1 (((guix monads)) ((guix records)) ((guix #)) (#) ?))
   222:29  6 (map1 (((guix records)) ((guix base16)) ((guix #)) (#) ?))
   222:29  5 (map1 (((guix base16)) ((guix base32)) ((gcrypt #)) # ?))
   222:29  4 (map1 (((guix base32)) ((gcrypt hash)) ((guix #)) (#) ?))
   222:17  3 (map1 (((gcrypt hash)) ((guix profiling)) ((rnrs #)) # ?))
   2803:6  2 (resolve-interface _ #:select _ #:hide _ #:prefix _ # _ ?)
In unknown file:
           1 (scm-error misc-error #f "~A ~S" ("no code for modu?" ?) ?)
In ice-9/boot-9.scm:
   752:25  0 (dispatch-exception _ _ _)

ice-9/boot-9.scm:752:25: In procedure dispatch-exception:
no code for module (gcrypt hash)
builder for
`/gnu/store/aaqaz52fhjb86g233ar4ynnyjvrv7xa7-module-import-compiled.drv'
failed with exit code 1

Any ideas?

> Hope that helps, thanks for the work on Jami y'all!
No problem.


Dnia 2019-12-10, o godz. 09:59:40
Caleb Ristvedt <caleb.ristvedt@cune.org> napisał(a):

> #:modules and #:imported-modules are distinct arguments. #:modules is
> the modules that your builder is going to use (as in "they go in a
> (use-modules ...) form"), while #:imported-modules is the modules
> that need to be available
> in the build environment. It's complaining at build-time that it
> can't find that
> module to use, because you haven't told it to include that module in
> the build
> environment. #:imported-modules should give a superset of what
> #:modules gives,
> especially if a module in use is going to have indirect
> requirements. Thankfully, wrangling together those indirect
> requirements is already implemented in (guix modules) as
> source-module-closure.
> 
> Thus, you could conceptually do
> 
> 
> (arguments
>      `(#:imported-modules (,@(source-module-closure
>                   '((gnu packages jami)
>                 ,@%gnu-build-system-modules)))
>        #:modules ((gnu packages jami)
>           ,@(@@ (guix build-system gnu) %default-modules))))
> 
> And in theory it would work. Note, though, that this would pull in the
> entire
> module dependency graph of (gnu packages jami), and this may include
> things that
> source-module-closure would have a hard time detecting and aren't
> really needed. Ideally this procedure would be generalized and put in
> (guix build <something>), but I can understand if that's not
> possible. Note also that you
> could simply splice in the definition of your procedure into the
> builder manually, like so:
> 
> (define my-procedure-code '(lambda (a b c) ...))
> 
> (arguments
>  `(#:phases (let ((my-procedure ,my-procedure-code)) (modify-phases
> ...))))
> 
> Hope that helps, thanks for the work on Jami y'all!


Jan Wielkiewicz

Re: Packaging Jami progress

[Caleb Ristvedt]

Jan Wielkiewicz <tona_kosmicznego_smiecia@interia.pl> writes:

> I tried both ways - the second works, but the first doesn't.

That would be the "in theory, it would work" part. On further investigation,
source-module-closure has a #:select? keyword argument, which takes a module
name and returns #f if it shouldn't be included in the closure. By default it's
'guix-module-name?', so it only includes the guix modules, and not, for example,
(gcrypt hash). One might try passing #:select? (const #t), but this would likely
reveal further issues - for example, guile-gcrypt doesn't work without
libgcrypt, but source-module-closure isn't going to pull that in.

The short answer is "yeah, for big closures that reach outside of guix (or
especially that have non-scheme dependencies) source-module-closure isn't
perfect", and build-side code should try to minimize the size of the closure it
pulls in (which, for pretty much any (gnu packages ...) module, is going to be
huge). The second solution is probably the better one here.

There's this sort of awkward middle ground we don't see much where a
build-side procedure has to be specific to some relatively small set of
packages, but still to enough packages that repeating it in the builder
for each of those packages duplicates a lot of code. Splicing the
definition into the builder programmatically is a bit of a hack, as it's
still duplicated between each builder interned in the store, but much
better than copy+pasting manually.

Hope that helps.

- reepca

Re: Packaging Jami progress

[Jan]

Okay, thanks. I'll use the second way then. I could also create a new
file - jami-utils.scm and use it as a module, but a file containing
only one procedure isn't good either.

On Tue, 10 Dec 2019 18:43:58 -0600
Caleb Ristvedt <caleb.ristvedt@cune.org> wrote:

> Jan Wielkiewicz <tona_kosmicznego_smiecia@interia.pl> writes:
> 
> > I tried both ways - the second works, but the first doesn't.  
> 
> That would be the "in theory, it would work" part. On further
> investigation, source-module-closure has a #:select? keyword
> argument, which takes a module name and returns #f if it shouldn't be
> included in the closure. By default it's 'guix-module-name?', so it
> only includes the guix modules, and not, for example, (gcrypt hash).
> One might try passing #:select? (const #t), but this would likely
> reveal further issues - for example, guile-gcrypt doesn't work
> without libgcrypt, but source-module-closure isn't going to pull that
> in.
> 
> The short answer is "yeah, for big closures that reach outside of
> guix (or especially that have non-scheme dependencies)
> source-module-closure isn't perfect", and build-side code should try
> to minimize the size of the closure it pulls in (which, for pretty
> much any (gnu packages ...) module, is going to be huge). The second
> solution is probably the better one here.
> 
> There's this sort of awkward middle ground we don't see much where a
> build-side procedure has to be specific to some relatively small set
> of packages, but still to enough packages that repeating it in the
> builder for each of those packages duplicates a lot of code. Splicing
> the definition into the builder programmatically is a bit of a hack,
> as it's still duplicated between each builder interned in the store,
> but much better than copy+pasting manually.
> 
> Hope that helps.
> 
> - reepca


Jan Wielkiewicz

Re: Packaging Jami progress

[zimoun]

Hi,

Thank you for working that.



On Mon, 25 Nov 2019 at 22:15, Jan <tona_kosmicznego_smiecia@interia.pl> wrote:

> Sorry for being impatient, but is it normal for patches to be merged
> that long? Is there something stopping the commits?
> I need those merged in order to continue working on Jami.

Two weeks are not that long. ;-)
If you feel frustrating -- and I understand, I feel time to time the
same :-) -- my advice is to help to decrease the stack by reviewing
(building, testing, asking the status) of old patches [1] and/or by
trying to close old bugs [2].

[1] https://debbugs.gnu.org/cgi/pkgreport.cgi?repeatmerged=0;max-bugs=100;base-order=1;package=guix-patches;bug-rev=1;ordering=normal;archive=0;page=4
[2] https://debbugs.gnu.org/cgi/pkgreport.cgi?package=guix;max-bugs=100;archive=0;ordering=normal;base-order=1;repeatmerged=0;bug-rev=1;page=9


Why do you need these merged in master to continue working on Jami?


In any case, the patch "[PATCH 2/9] gnu: Add sobjectizer." does not
seem to apply because the commit 7e08be71ac39f (pushed the day after
Pierre submitted the series :-))


Pierre, could you advice me with your Magit' power to easily apply and
test the series? Or the only way is to rebase against the parent
commit of the Mathieu's commit?


Cheers,
simon

Re: Packaging Jami progress

[Pierre Neidhardt]

[-- Attachment #1: Type: text/plain, Size: 162 bytes --]

I'm not sure how to apply conflicting patches with Magit.  Good
question, I'd like to know how to do it too! :)

-- 
Pierre Neidhardt
https://ambrevar.xyz/

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

Re: Packaging Jami progress

[Pierre Neidhardt]

[-- Attachment #1: Type: text/plain, Size: 932 bytes --]

Jan, can you resend the your patch set, I think you forgot a commit.
The first patch does not apply, in particular this hunk:

--8<---------------cut here---------------start------------->8---
 		  ;; Things we don't need:
 		  ;; BaseClasses - contains libraries from Windows SDK
 		  ;; we don't need it, at least not now.
-                  (list "BaseClasses" "bin" "g7221" "ilbc" "milenage"
+                  (list "BaseClasses" "g7221" "ilbc" "milenage"
 			"speex" "threademulation" "yuv" "bdsound"
-			"gsm" "lib" "mp3" "resample" "srtp" "webrtc"
+			"gsm" "mp3" "resample" "srtp" "webrtc"
                         ;; Keep only resample, build and README.txt.
                         "build/baseclasses" "build/g7221" "build/gsm"
 			"build/ilbc" "build/milenage" "build/resample"
--8<---------------cut here---------------end--------------->8---

Cheers!

-- 
Pierre Neidhardt
https://ambrevar.xyz/

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

Re: Packaging Jami progress

[Jan Wielkiewicz]

Dnia 2019-11-07, o godz. 20:10:01
Pierre Neidhardt <mail@ambrevar.xyz> napisał(a):

> Jan, can you resend the your patch set, I think you forgot a commit.
> The first patch does not apply, in particular this hunk:
> 
> --8<---------------cut here---------------start------------->8---
>  		  ;; Things we don't need:
>  		  ;; BaseClasses - contains libraries from Windows
> SDK ;; we don't need it, at least not now.
> -                  (list "BaseClasses" "bin" "g7221" "ilbc" "milenage"
> +                  (list "BaseClasses" "g7221" "ilbc" "milenage"
>  			"speex" "threademulation" "yuv" "bdsound"
> -			"gsm" "lib" "mp3" "resample" "srtp" "webrtc"
> +			"gsm" "mp3" "resample" "srtp" "webrtc"
>                          ;; Keep only resample, build and README.txt.
>                          "build/baseclasses" "build/g7221" "build/gsm"
>  			"build/ilbc" "build/milenage"
> "build/resample" --8<---------------cut
> here---------------end--------------->8---
> 
> Cheers!
> 

I sent everything I had, so don't know what's wrong. Can we skip this
now and I'll make a new commit (don't know how intelligent git is)?
There's a chance I modified and something, then ran "git checkout
previous_commit_number", and then commited, but I'm not sure, since
this is the first time I use git for something other than "git clone".
I also ran a strange command - "git cherrypick", when I was trying to
figure out how to patch my commits.


Jan Wielkiewicz

Re: Packaging Jami progress

[Pierre Neidhardt]

[-- Attachment #1: Type: text/plain, Size: 585 bytes --]

I think you produced the patchset against the wrong commit.
Try 

  git format-patch origin

or

  git format-patch LAST-UPSTREAM-COMMIT

where you replace LAST-UPSTREAM-COMMIT as appropriate.

A pitfall is to write

  git format-patch YOUR-FIRST-COMMIT

then you'd omit your first commit, which is probably what happened.

> I also ran a strange command - "git cherrypick", when I was trying to
> figure out how to patch my commits.

Git cherrypick allows you to apply specific commits on top of the
current HEAD.

-- 
Pierre Neidhardt
https://ambrevar.xyz/

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

Re: Packaging Jami progress

[Jan Wielkiewicz]

[-- Attachment #1: Type: text/plain, Size: 897 bytes --]

Dnia 2019-11-07, o godz. 21:37:52
Pierre Neidhardt <mail@ambrevar.xyz> napisał(a):

> I think you produced the patchset against the wrong commit.
> Try 
> 
>   git format-patch origin
> 
> or
> 
>   git format-patch LAST-UPSTREAM-COMMIT
Okay, I did the first one and I'm attaching the patches again, hope
this time it'll work.

> where you replace LAST-UPSTREAM-COMMIT as appropriate.
> 
> A pitfall is to write
> 
>   git format-patch YOUR-FIRST-COMMIT
> 
> then you'd omit your first commit, which is probably what happened.
> 
> > I also ran a strange command - "git cherrypick", when I was trying
> > to figure out how to patch my commits.
> 
> Git cherrypick allows you to apply specific commits on top of the
> current HEAD.
Good to know, thanks.

I am going to apply your patch for restinio and try to get Jami to work
this weekend.



Jan Wielkiewicz


[-- Attachment #2: jami-wip-08.11.2019.tar.bz2 --]
[-- Type: application/x-bzip, Size: 8549 bytes --]

Re: Packaging Jami progress

[Pierre Neidhardt]

[-- Attachment #1: Type: text/plain, Size: 406 bytes --]

Jan Wielkiewicz <tona_kosmicznego_smiecia@interia.pl> writes:

> Okay, I did the first one and I'm attaching the patches again, hope
> this time it'll work.

It worked, thanks!

> I am going to apply your patch for restinio and try to get Jami to work
> this weekend.

OK, let me know how it goes and don't hesitate to ask for help!

Good luck!

-- 
Pierre Neidhardt
https://ambrevar.xyz/

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

Re: Packaging Jami progress

[Jan Wielkiewicz]

[-- Attachment #1: Type: text/plain, Size: 1842 bytes --]

Dnia 2019-11-11, o godz. 09:38:49
Pierre Neidhardt <mail@ambrevar.xyz> napisał(a):

> Jan Wielkiewicz <tona_kosmicznego_smiecia@interia.pl> writes:
> 
> > Okay, I did the first one and I'm attaching the patches again, hope
> > this time it'll work.
> 
> It worked, thanks!
> 
> > I am going to apply your patch for restinio and try to get Jami to
> > work this weekend.
> 
> OK, let me know how it goes and don't hesitate to ask for help!
I fixed some small problems since then: I added a newer version of
curl, because the build of something failed with an old version (didn't
change the main package though, because I didn't want to break
something); I removed restbed from the list of dependencies of opendht
and libring; I updated gstreamer to the version not requiring the
security patch anymore (1.16.1) and updated the gst-plugins-base to
1.16.1.

Is it normal guix ran from "./pre-inst-env" builds everything from the
source code? I ran "./pre-inst-env guix build jami --cores=2
--max-jobs=2" and Guix started to build webkitgtk, even though the
substitute is available. Or did I change something that causes webkit
to get rebuild. This made checking if Jami builds properly not possible,
because my machine has only 3,1 GB of RAM.
I could set up Guix on my second, more powerfull machine, but last time
I tried, the installer script didn't work on Devuan. I don't know if
it's a bug, or is the system not supported. I could install Guix System
there, but the machine is highly proprietary and Guix isn't yet fully
ready for the desktop use for me (not every package I use, including
Jami, is available yet).

But I suppose Jami will work this time, there's not much (or nothing) to
fix now. I'm attaching all patches, including the most recent.

> Good luck!
> 

Thanks,

Jan Wielkiewicz

[-- Attachment #2: jami-wip-11-11-2019.tar.bz2 --]
[-- Type: application/x-bzip, Size: 10286 bytes --]

Re: Packaging Jami progress

[Pierre Neidhardt]

[-- Attachment #1: Type: text/plain, Size: 1415 bytes --]

Jan Wielkiewicz <tona_kosmicznego_smiecia@interia.pl> writes:

> Is it normal guix ran from "./pre-inst-env" builds everything from the
> source code? I ran "./pre-inst-env guix build jami --cores=2
> --max-jobs=2" and Guix started to build webkitgtk, even though the
> substitute is available. Or did I change something that causes webkit
> to get rebuild.

If you changed any (recursive) input of webkitgtk, then yes, it will
trigger a rebuilt.

If you didn't, but if you "guix pulled" and upstream changed an input
recently while the build farm hasn't had time to produce a substitute,
then you'll have to build webkitgtk yourself, or wait for the build farm.

> This made checking if Jami builds properly not possible,
> because my machine has only 3,1 GB of RAM.
> I could set up Guix on my second, more powerfull machine, but last time
> I tried, the installer script didn't work on Devuan. I don't know if
> it's a bug, or is the system not supported. I could install Guix System
> there, but the machine is highly proprietary and Guix isn't yet fully
> ready for the desktop use for me (not every package I use, including
> Jami, is available yet).

> But I suppose Jami will work this time, there's not much (or nothing) to
> fix now. I'm attaching all patches, including the most recent.

Thanks a lot, I'll give it a shot then.

-- 
Pierre Neidhardt
https://ambrevar.xyz/

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

Re: Packaging Jami progress

[Pierre Neidhardt]

[-- Attachment #1: Type: text/plain, Size: 265 bytes --]

The big rebuild is triggered by the gnutls version update.
Is the update really necessary at the moment?

If you want to know how many packages depend on gnutls, you can run

  guix refresh -l gnutls

Cheers!

-- 
Pierre Neidhardt
https://ambrevar.xyz/

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

Re: Packaging Jami progress

[Jan]

On Mon, 11 Nov 2019 16:04:30 +0100
Pierre Neidhardt <mail@ambrevar.xyz> wrote:

> The big rebuild is triggered by the gnutls version update.
> Is the update really necessary at the moment?
The version of gnutls has been bumped in Jami, so I guess that's
necessary. If that's a problem I can add gnutls-jami.

> If you want to know how many packages depend on gnutls, you can run
> 
>   guix refresh -l gnutls
Will use next time, thanks.
I'm also installing Guix System on an external hard drive and I'll be
able to compile everything with 6 cores (12 threads) on my second
machine.

> Cheers!
> 


Jan Wielkiewicz

Re: Packaging Jami progress

[Pierre Neidhardt]

[-- Attachment #1: Type: text/plain, Size: 126 bytes --]

Update: I've managed to build libring.
Jami should follow soon, stay tuned.

-- 
Pierre Neidhardt
https://ambrevar.xyz/

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

Re: Packaging Jami progress

[Pierre Neidhardt]

[-- Attachment #1: Type: text/plain, Size: 94 bytes --]

See https://issues.guix.gnu.org/issue/38211.

-- 
Pierre Neidhardt
https://ambrevar.xyz/

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

Re: Packaging Jami progress

[Jan]

On Thu, 14 Nov 2019 19:07:56 +0100
Pierre Neidhardt <mail@ambrevar.xyz> wrote:

> See https://issues.guix.gnu.org/issue/38211.
> 

Cool, thanks for building it! 
As for the issue, I don't know if the Jami version I use is
stable, unfortunately some versions from the tarball repo contain bugs.
I can ask the devs about it.

I wanted to check if Jami compiles at all with the current state of
work, not to break already working things. Now, because we know it
compiles, I can improve the quality of the package and its
dependencies, and of course keep it updated.

My ideas:
- what about handling the package in a modular way and instead of
  having three separated packages, can we have one package with many
  outputs? For example jami:libring, jami:client-gnome,
  jami:libringclient, etc.
- finishing pjproject - the current package doesn't work at all, what
  works is pjproject-jami, because we disable some features. To prevent
  further breaking and not to keep a broken package it would be a right
  thing to do to finish it.
- support for other operating systems (???) Guix probably doesn't
  support this now, but we could build every Jami client in a
  reproducible way. 
- easy to use Jami packages with guix pack for other distributions,
  using different init systems (support for autostart)? Or we can
  skip this and hope people will install Guix on their systems :)
- Jami daemon (libring) service for the Shepherd? I could do this, but
  I would have to learn about the Shepherd and daemons, since I know
  almost nothing.
- a small tutorial about maintaining a package, from the perspective of
  someone, who didn't know much about build systems and packaging.
  Things like where do you look for the needed dependencies, etc.


Jan Wielkiewicz

Re: Packaging Jami progress

[Pierre Neidhardt]

[-- Attachment #1: Type: text/plain, Size: 2451 bytes --]

Jan <tona_kosmicznego_smiecia@interia.pl> writes:

> On Thu, 14 Nov 2019 19:07:56 +0100
> Pierre Neidhardt <mail@ambrevar.xyz> wrote:
>
>> See https://issues.guix.gnu.org/issue/38211.
>> 
>
> Cool, thanks for building it! 
> As for the issue, I don't know if the Jami version I use is
> stable, unfortunately some versions from the tarball repo contain bugs.

I don't think it's a Jami bug: as I mentioned the same bug affects the
old package so I believe it comes from us.

> - what about handling the package in a modular way and instead of
>   having three separated packages, can we have one package with many
>   outputs? For example jami:libring, jami:client-gnome,
>   jami:libringclient, etc.

The thing is that those are 3 different build processes, so having 3
packages makes it easier to package.  I believe what we have is the most
natural way to go.

> - support for other operating systems (???) Guix probably doesn't
>   support this now, but we could build every Jami client in a
>   reproducible way.

What do you mean?  Guix can be installed on foreign distribution and
Jami would work there the same way it works on Guix System.

> - easy to use Jami packages with guix pack for other distributions,
>   using different init systems (support for autostart)? Or we can
>   skip this and hope people will install Guix on their systems :)

It's the job of the init system to decide what to start.  The Guix pack
does not embed any init system information, I don't know if it can or
even if that would make sense.

> - Jami daemon (libring) service for the Shepherd? I could do this, but
>   I would have to learn about the Shepherd and daemons, since I know
>   almost nothing.

Wouldn't it be a user service?

> - a small tutorial about maintaining a package, from the perspective of
>   someone, who didn't know much about build systems and packaging.
>   Things like where do you look for the needed dependencies, etc.

I'm not sure there is much more we can do here: regarding the packaging
process, we already have a tutorial (cf. the cookbook) that covers
everything needed to package Jami.  (As tough as Jami may be to package...)

Regarding the dependencies, it belongs to upstream
to tell which deps Jami uses; sadly this is not done very well in
their current documentation.  We could open an issue I suppose.

Cheers!

-- 
Pierre Neidhardt
https://ambrevar.xyz/

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

Re: Packaging Jami progress

[Jan]

On Thu, 14 Nov 2019 22:54:15 +0100
Pierre Neidhardt <mail@ambrevar.xyz> wrote:

> I don't think it's a Jami bug: as I mentioned the same bug affects the
> old package so I believe it comes from us.
Sorry, didn't read the entire message, I'm a bit tired, because of
exams. 

> The thing is that those are 3 different build processes, so having 3
> packages makes it easier to package.  I believe what we have is the
> most natural way to go.
Okay then.

> What do you mean?  Guix can be installed on foreign distribution and
> Jami would work there the same way it works on Guix System.
I mean Windows (ReactOS in the future?), OSX, iOS, Android - Jami is an
universal platform. AFAIK there's the option to build with
--target=mingw64 or something like this. Could Guix support other
targets as well? Some people in the Jami community want to have
reproducible builds for Jami, no matter on what distribution. 
Providing reproducible builds for other systems is going to improve
overall security of the platform. 

> It's the job of the init system to decide what to start.  The Guix
> pack does not embed any init system information, I don't know if it
> can or even if that would make sense.

> Wouldn't it be a user service?
I'm not sure if I understand this correctly, but autostart on GNU/Linux
is handled by different init systems/process supervisors. How is the
autostart handled on Guix System? When I searched for the way of adding
a program to autostart on Debian-based distributions, I often found
tutorials showing how to make an init script or systemd-something,
anyway I thought that if libring is a daemon, then there needs to be
something written for the Shepherd, but if it's handled as an user
service, then it's fine.

> I'm not sure there is much more we can do here: regarding the
> packaging process, we already have a tutorial (cf. the cookbook) that
> covers everything needed to package Jami.  (As tough as Jami may be
> to package...)
I found some things hard/unintuitive, but maybe that's because I didn't
entirely read the tutorial. Anyway if I find something worth improving,
I'll tell.

> Regarding the dependencies, it belongs to upstream
> to tell which deps Jami uses; sadly this is not done very well in
> their current documentation.  We could open an issue I suppose.
I'll ask about this.
> Cheers!
> 


Jan Wielkiewicz

Re: Packaging Jami progress

[Pierre Neidhardt]

[-- Attachment #1: Type: text/plain, Size: 2693 bytes --]

Hi Jan,

>> What do you mean?  Guix can be installed on foreign distribution and
>> Jami would work there the same way it works on Guix System.
> I mean Windows (ReactOS in the future?), OSX, iOS, Android - Jami is an
> universal platform. AFAIK there's the option to build with
> --target=mingw64 or something like this. Could Guix support other
> targets as well? Some people in the Jami community want to have
> reproducible builds for Jami, no matter on what distribution. 
> Providing reproducible builds for other systems is going to improve
> overall security of the platform. 

The --target is for the _hardware_ platform, not the OS.  In Guix, there
is no concept of an "OS", it's just Guix.  So if you want the result to
run on a different OS, it just need to support the executable format,
e.g. ELF.

Currently Guix runs on GNU/Linux.  In the future, support may be added
for the *BSD, macOS, etc.  Time will tell.

To answer your question, we cannot make a "Jami package for <some-OS>"
specifically, but we can add OS support for Guix as a whole.

>> Wouldn't it be a user service?
> I'm not sure if I understand this correctly, but autostart on GNU/Linux
> is handled by different init systems/process supervisors. How is the
> autostart handled on Guix System? When I searched for the way of adding
> a program to autostart on Debian-based distributions, I often found
> tutorials showing how to make an init script or systemd-something,
> anyway I thought that if libring is a daemon, then there needs to be
> something written for the Shepherd, but if it's handled as an user
> service, then it's fine.

In my understanding, Jami does not run as a system service but
per-user.  Actually, I'm not sure I would even call it a service, it's
more of an automatically-started program, so this is something you would
configure with you desktop environment.  GNOME supports this pretty well
as far as I know.

>> I'm not sure there is much more we can do here: regarding the
>> packaging process, we already have a tutorial (cf. the cookbook) that
>> covers everything needed to package Jami.  (As tough as Jami may be
>> to package...)
> I found some things hard/unintuitive, but maybe that's because I didn't
> entirely read the tutorial. Anyway if I find something worth improving,
> I'll tell.

If you are new to packaging in Guix, then you picked the wrong package
I'm afraid: Jami is really hard! :p This is something that has to do
with Jami itself, not so much with Guix.

That said, feel free to send suggestions to improve the cookbook.  The
clearer the better!

Cheers!

-- 
Pierre Neidhardt
https://ambrevar.xyz/

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

Re: Packaging Jami progress

[Jan]

On Fri, 15 Nov 2019 10:07:38 +0100
Pierre Neidhardt <mail@ambrevar.xyz> wrote:

Hello,
When can I expect our changes to be merged? There's a new Jami version
(20191115.5.b0a579d), I would like to update it to, but a strange
conflict stops me from running "git pull". I also can't investigate the
issue, until webkit-gtk substitute is available, because my laptop is
underpowered.


Jan Wielkiewicz

Re: Packaging Jami progress

[Jan Wielkiewicz]

Hi all,

I made some progress with applying patches for dependencies. The
function works now, I use it with pjproject-jami correctly.
Tried luck with ffmpeg-jami, the compilation process works, but one
test fails:

TEST    lavf-mkv_attachment
TEST    lavf-mov
TEST    lavf-mov_rtphint
--- ./tests/ref/lavf/mov_rtphint	1970-01-01 00:00:01.000000000
+0000 +++ tests/data/fate/lavf-mov_rtphint	2019-12-15
20:04:09.880137614 +0000 @@ -1,3 +1,3 @@
-7014419d8267c2751314303a8fb303c1 *tests/data/lavf/lavf.mov_rtphint
-366449 tests/data/lavf/lavf.mov_rtphint
+872f923297706823384923086147e2b4 *tests/data/lavf/lavf.mov_rtphint
+370877 tests/data/lavf/lavf.mov_rtphint
 tests/data/lavf/lavf.mov_rtphint CRC=0xbb2b949b
Test lavf-mov_rtphint failed. Look at
tests/data/fate/lavf-mov_rtphint.err for details. make: ***
[tests/Makefile:241: fate-lavf-mov_rtphint] Error 1 make: *** Waiting
for unfinished jobs....

Could it be changing the date we do in Guix makes a test fail?
Any ideas what could be wrong? Sould I skip the test?


Jan Wielkiewicz

Re: Packaging Jami progress

[Ricardo Wurmus]

Hi Jan,

> TEST    lavf-mov_rtphint
> --- ./tests/ref/lavf/mov_rtphint	1970-01-01 00:00:01.000000000
> +0000 +++ tests/data/fate/lavf-mov_rtphint	2019-12-15
> 20:04:09.880137614 +0000 @@ -1,3 +1,3 @@
> -7014419d8267c2751314303a8fb303c1 *tests/data/lavf/lavf.mov_rtphint
> -366449 tests/data/lavf/lavf.mov_rtphint
> +872f923297706823384923086147e2b4 *tests/data/lavf/lavf.mov_rtphint
> +370877 tests/data/lavf/lavf.mov_rtphint
>  tests/data/lavf/lavf.mov_rtphint CRC=0xbb2b949b
> Test lavf-mov_rtphint failed. Look at
> tests/data/fate/lavf-mov_rtphint.err for details. make: ***
> [tests/Makefile:241: fate-lavf-mov_rtphint] Error 1 make: *** Waiting
> for unfinished jobs....

The test appears to compare the hash of this file with the hash of a
known good file.  What are the contents?  Can we adjust the test to test
for what is actually of interest here?

Otherwise it should be fine to disable it.

-- 
Ricardo

Re: Packaging Jami progress

[Jan Wielkiewicz]

On Sun, 15 Dec 2019 22:46:06 +0100
Ricardo Wurmus <rekado@elephly.net> wrote:

> Hi Jan,
> 
> The test appears to compare the hash of this file with the hash of a
> known good file.  What are the contents?  Can we adjust the test to
> test for what is actually of interest here?
> 
> Otherwise it should be fine to disable it.
> 

Actually I have no idea, I downloaded the source code now and I
couldn't even find the file, the closest one was "lavf-mov", but its
contents doesn't look human-readable. I'll better ask Jami devs about
it.


Jan Wielkiewicz

Re: Packaging Jami progress

[Jan Wielkiewicz]

Dnia 2019-12-15, o godz. 22:46:06
Ricardo Wurmus <rekado@elephly.net> napisał(a):

> The test appears to compare the hash of this file with the hash of a
> known good file.  What are the contents?  Can we adjust the test to
> test for what is actually of interest here?
> 
> Otherwise it should be fine to disable it.
> 

Okay, I haven't got any answer from the devs so far and I'm looking for
a way of disabling only the test that fails.
I tried to use substitute* to remove 
"include $(SRC_PATH)/tests/fate/lavf-container.mak"
from the tests/Makefile, but it didn't work (don't know why) and tried
removing "FATE_LAVF_CONTAINER-$(call ENCDEC2, MPEG4,      PCM_ALAW,
MOV)                += mov mov_rtphint ismv" 
and
"fate-lavf-mov_rtphint: CMD = lavf_container "" "-movflags +rtphint
-c:a pcm_alaw -c:v mpeg4 -threads 1 -f mov""
from the "tests/fate/lavf-container.mak" file.

The fragment of code I wrote looks like this:

(add-before 'check 'skip-test
  (lambda _
    (substitute* "tests/Makefile"
	(("include $(SRC_PATH)/tests/fate/lavf-container.mak")
			"")) #t))

I also made the git checkout writeable.
Am I doing something wrong? Should I try deleting the test using
snippet or just skip all tests using #:tests? #f?


Jan Wielkiewicz

Re: Packaging Jami progress

[Ricardo Wurmus]

Jan Wielkiewicz <tona_kosmicznego_smiecia@interia.pl> writes:

> The fragment of code I wrote looks like this:
>
> (add-before 'check 'skip-test
>   (lambda _
>     (substitute* "tests/Makefile"
> 	(("include $(SRC_PATH)/tests/fate/lavf-container.mak")
> 			"")) #t))
>
> I also made the git checkout writeable.
> Am I doing something wrong? Should I try deleting the test using
> snippet or just skip all tests using #:tests? #f?

The problem here is that “substitute*” expects regular expressions and
“$”, “(”, “)”, and “.” all have special meaning in a regular expression
context, so they need to be escaped.  Escaping is done with a backslash,
but using a backslash in a string in Guile requires another layer of
escaping, so we end up with this expression:

    (add-before 'check 'skip-test
      (lambda _
        (substitute* "tests/Makefile"
          (("include \\$\\(SRC_PATH\\)/tests/fate/lavf-container.mak") ""))
        #t))

(I did not escap the dot here because it can only match a single
character, a literal dot in this case.)

--
Ricardo

Re: Packaging Jami progress

[Jan]

On Sun, 22 Dec 2019 08:48:31 +0100
Ricardo Wurmus <rekado@elephly.net> wrote:

> The problem here is that “substitute*” expects regular expressions and
> “$”, “(”, “)”, and “.” all have special meaning in a regular
> expression context, so they need to be escaped.  Escaping is done
> with a backslash, but using a backslash in a string in Guile requires
> another layer of escaping, so we end up with this expression:
> 
>     (add-before 'check 'skip-test
>       (lambda _
>         (substitute* "tests/Makefile"
>           (("include
> \\$\\(SRC_PATH\\)/tests/fate/lavf-container.mak") "")) #t))
> 
> (I did not escap the dot here because it can only match a single
> character, a literal dot in this case.)
> 
> --
> Ricardo
> 
Thanks, this solved the issue! ffmpeg-jami now builds correctly,
but there's one more issue. In the "origin" field I use the exact
commit number needed in for applying the patches correctly. Downloading
the source code takes about a minute and then it throws an error saying
that the given commit number is unadvertised by the server and then it
switches to the commit. How can I prevent the source code from being
downloaded for so long?

Besides this, I think the work is finished for now and I would like to
send the patches myself this time, but it seems I used "git pull"
somewhere between my changes. How do I make a patch containing only my
changes?
What about the copyright lines? Should I add them? I also moved Jami
and its modified dependencies to a new file - jami.scm. I know for sure
that's Pierre's and my work, but I don't know if someone else have
modified the code.



Jan Wielkiewicz

Re: Packaging Jami progress

[Jan]

Never mind the question about removing not my commits - I rediscovered
the guix pull --rebase option, Pierre showed me previously.

There's also a problem - don't know if it's a bug or not, but Jami
client encounters an error during screen sharing. I suspect it could be
caused by the gnutls version in our repo - 3.6.9, whereas Jami
developers told me the unpatched 3.6.10 should work finely.
Is there anyone maintaining gnutls? I could try updating it, but for
now I can't resolve some failing tests on 3.6.10 and I would like not
to touch it at all, considering it is a really crucial for security
package - I don't want to make the whole community vulnerable due to a
stupid mistake I could make.


Jan Wielkiewicz

Re: Packaging Jami progress

[Efraim Flashner]

[-- Attachment #1: Type: text/plain, Size: 1239 bytes --]

On Wed, Dec 25, 2019 at 02:34:16AM +0100, Jan wrote:
> Never mind the question about removing not my commits - I rediscovered
> the guix pull --rebase option, Pierre showed me previously.
> 
> There's also a problem - don't know if it's a bug or not, but Jami
> client encounters an error during screen sharing. I suspect it could be
> caused by the gnutls version in our repo - 3.6.9, whereas Jami
> developers told me the unpatched 3.6.10 should work finely.
> Is there anyone maintaining gnutls? I could try updating it, but for
> now I can't resolve some failing tests on 3.6.10 and I would like not
> to touch it at all, considering it is a really crucial for security
> package - I don't want to make the whole community vulnerable due to a
> stupid mistake I could make.
> 

gnutls has too many dependant packages to be updated on the master
branch. One option is to create a gnutls-3.6.10 package which inherits
from our normal gnutls package and use that until gnutls gets updated on
core-updates.

-- 
Efraim Flashner   <efraim@flashner.co.il>   אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Packaging Jami progress

[Jan Wielkiewicz]

Tested Jami with gnutls 3.6.10, but I get the same bug. I reported it
to Jami developers, but they can't reproduce the bug.
Here's more info, if anyone has any idea what could be the cause, then
please tell me: 
https://git.jami.net/savoirfairelinux/ring-client-gnome/issues/1123

I need to find out what's wrong with our package. How do I debug
something on Guix System? I used "guix environment guix", then
"./pre-inst-env guix environment jami --ad-hoc gdb", then after running
gdb and passing the proper path to it and pressing "r", it displays the
following error:

Reading symbols from
/gnu/store/30jjbf7jkddw6z679c0h8zvifwaaakn0-jami-20191224.1.5c0154a/bin/.jami-gnome-real...
(No debugging symbols found in
/gnu/store/30jjbf7jkddw6z679c0h8zvifwaaakn0-jami-20191224.1.5c0154a/bin/.jami-gnome-real)
(gdb) r Starting program:
/gnu/store/30jjbf7jkddw6z679c0h8zvifwaaakn0-jami-20191224.1.5c0154a/bin/.jami-gnome-real
[Thread debugging using libthread_db enabled] Using host libthread_db
library
"/gnu/store/ahqgl4h89xqj695lgqvsaf6zh2nhy4pj-glibc-2.29/lib/libthread_db.so.1".
[New Thread 0x7fffee145700 (LWP 4795)] [New Thread 0x7fffed944700 (LWP
4796)] ** Message: 19:45:55.037: Jami GNOME client version:
501cb99929f171ede62a96c675d51ffe0581ce5c ** Message: 19:45:55.038: git
ref: unknown [New Thread 0x7fffeca91700 (LWP 4797)] No migration
required
/gnu/store/30jjbf7jkddw6z679c0h8zvifwaaakn0-jami-20191224.1.5c0154a/bin/.jami-gnome-real:
symbol lookup error:
/gnu/store/371gzl7v7c8p0waasm4kwkalvgqmskav-qtbase-5.12.6/lib/qt5/plugins/sqldrivers/libqsqlite.so:
undefined symbol: sqlite3_column_table_name16 [Thread 0x7fffeca91700
(LWP 4797) exited] [Thread 0x7fffed944700 (LWP 4796) exited] [Thread
0x7fffee145700 (LWP 4795) exited] [Inferior 1 (process 4789) exited
with code 0177] (gdb) 


Jan Wielkiewicz

Re: Packaging Jami progress

[Gábor Boskovits]

Hello Jan,

Thanks for working on this.

Jan Wielkiewicz <tona_kosmicznego_smiecia@interia.pl> ezt írta
(időpont: 2019. dec. 27., P, 20:11):
>
> Tested Jami with gnutls 3.6.10, but I get the same bug. I reported it
> to Jami developers, but they can't reproduce the bug.
> Here's more info, if anyone has any idea what could be the cause, then
> please tell me:
> https://git.jami.net/savoirfairelinux/ring-client-gnome/issues/1123
>
> I need to find out what's wrong with our package. How do I debug
> something on Guix System? I used "guix environment guix", then
> "./pre-inst-env guix environment jami --ad-hoc gdb", then after running
> gdb and passing the proper path to it and pressing "r", it displays the
> following error:
>
> Reading symbols from
> /gnu/store/30jjbf7jkddw6z679c0h8zvifwaaakn0-jami-20191224.1.5c0154a/bin/.jami-gnome-real...
> (No debugging symbols found in
> /gnu/store/30jjbf7jkddw6z679c0h8zvifwaaakn0-jami-20191224.1.5c0154a/bin/.jami-gnome-real)
> (gdb) r Starting program:
> /gnu/store/30jjbf7jkddw6z679c0h8zvifwaaakn0-jami-20191224.1.5c0154a/bin/.jami-gnome-real
> [Thread debugging using libthread_db enabled] Using host libthread_db
> library
> "/gnu/store/ahqgl4h89xqj695lgqvsaf6zh2nhy4pj-glibc-2.29/lib/libthread_db.so.1".
> [New Thread 0x7fffee145700 (LWP 4795)] [New Thread 0x7fffed944700 (LWP
> 4796)] ** Message: 19:45:55.037: Jami GNOME client version:
> 501cb99929f171ede62a96c675d51ffe0581ce5c ** Message: 19:45:55.038: git
> ref: unknown [New Thread 0x7fffeca91700 (LWP 4797)] No migration
> required
> /gnu/store/30jjbf7jkddw6z679c0h8zvifwaaakn0-jami-20191224.1.5c0154a/bin/.jami-gnome-real:
> symbol lookup error:
> /gnu/store/371gzl7v7c8p0waasm4kwkalvgqmskav-qtbase-5.12.6/lib/qt5/plugins/sqldrivers/libqsqlite.so:
> undefined symbol: sqlite3_column_table_name16 [Thread 0x7fffeca91700
> (LWP 4797) exited] [Thread 0x7fffed944700 (LWP 4796) exited] [Thread
> 0x7fffee145700 (LWP 4795) exited] [Inferior 1 (process 4789) exited
> with code 0177] (gdb)

You should look for packages with debug output. That is the way the
debugging symbol files are generated
for a package. Currently not too many packages define it, but from an
example you can easily find out how to
include that into the interesting package. You can find further
infomation here:
https://guix.gnu.org/manual/en/html_node/Installing-Debugging-Files.html

>
>
> Jan Wielkiewicz
>

Best regards,
g_bor
-- 
OpenPGP Key Fingerprint: 7988:3B9F:7D6A:4DBF:3719:0367:2506:A96C:CF63:0B21

Re: Packaging Jami progress

[Jan Wielkiewicz]

Dnia 2019-12-27, o godz. 21:32:10
Gábor Boskovits <boskovits@gmail.com> napisał(a):

> Hello Jan,
> 
> Thanks for working on this.
Jami is really painful to package :D
> 
> You should look for packages with debug output. That is the way the
> debugging symbol files are generated
> for a package. Currently not too many packages define it, but from an
> example you can easily find out how to
> include that into the interesting package. You can find further
> infomation here:
> https://guix.gnu.org/manual/en/html_node/Installing-Debugging-Files.html
> 
Tried adding some configure flags for enabling debugging, but it seems
it's not the cause.

But there's a comment, I guess Pierre left, showing a similar error:
       
;; TODO: We must wrap ring-client-gnome to force using the
;; `sqlite-with-column-metadata' package instead of `sqlite' or else it
;; fails with:
;;
;;   /gnu/store/...-qtbase-5.11.2/lib/qt5/plugins/sqldrivers/libqsqlite.so:
;;   undefined symbol: sqlite3_column_table_name16
;;
;; qtbase is built against sqlite-with-column-metadata but somehow
;; jami-client-gnome ends up with both `sqlite' and
;; `sqlite-with-column-metadata' as inputs and it seems that
;; libqsqlite.so gets confused.

I have no idea what does it means though. Could someone explain to me
what needs to be done in *simple language* please?

> 
> Best regards,
> g_bor


Jan Wielkiewicz

Re: Packaging Jami progress

[Pierre Neidhardt]

[-- Attachment #1: Type: text/plain, Size: 275 bytes --]

This is indeed your problem, I suspect that something broke the wrapper
somehow.
Can you share your patch set again?  I can have a look.

P.S.:  Have you managed to set up a public Git clone of Guix to share
you patches?

-- 
Pierre Neidhardt
https://ambrevar.xyz/

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

Re: Packaging Jami progress

[Jan]

On Sat, 28 Dec 2019 10:40:09 +0100
Pierre Neidhardt <mail@ambrevar.xyz> wrote:

> This is indeed your problem, I suspect that something broke the
> wrapper somehow.
> Can you share your patch set again?  I can have a look.
I didn't really break anything - it works when launching Jami normally
or using the ring.cx script or the jami-gnome script manually. ring.cx
starts both the daemon and the client using "jami-gnome", jami-gnome is
also a script, which exports the path and then starts
".jami-gnome-real". Jami works during normal use, but I couldn't debug
like this, because by passing only ".jami-gnome-real" which is a
binary, to gdb, the needed path wasn't exported.

> P.S.:  Have you managed to set up a public Git clone of Guix to share
> you patches?
> 

Not yet, but can try today.


Jan Wielkiewicz

Re: Packaging Jami progress

[Ricardo Wurmus]

Jan <tona_kosmicznego_smiecia@interia.pl> writes:

> On Sat, 28 Dec 2019 10:40:09 +0100
> Pierre Neidhardt <mail@ambrevar.xyz> wrote:
>
>> This is indeed your problem, I suspect that something broke the
>> wrapper somehow.
>> Can you share your patch set again?  I can have a look.
> I didn't really break anything - it works when launching Jami normally
> or using the ring.cx script or the jami-gnome script manually. ring.cx
> starts both the daemon and the client using "jami-gnome", jami-gnome is
> also a script, which exports the path and then starts
> ".jami-gnome-real". Jami works during normal use, but I couldn't debug
> like this, because by passing only ".jami-gnome-real" which is a
> binary, to gdb, the needed path wasn't exported.

You can run “gdb --args /bin/sh /path/to/wrapper” and then "set
follow-fork-mode child" in gdb.

--
Ricardo

Re: Packaging Jami progress

[Jan]

On Sat, 28 Dec 2019 10:40:09 +0100
Pierre Neidhardt <mail@ambrevar.xyz> wrote:

> P.S.:  Have you managed to set up a public Git clone of Guix to share
> you patches?
> 

Okay, good news: I finally have a public repo at
https://notabug.org/kromka_chleba/guix-jami/src/jami-wip
My changes are on the jami-wip branch. Would be nice if someone checked
the ffmpeg-jami package - I have there some conditionals adding
configure flags depending on the architecture and system, and don't know
if what I did is fine.

Something like this:
(if (string-contains (%current-system) "linux")
	'("--enable-pic"
...

Bad news:
I still haven't got any response that would solve the bug present only
in our package:
https://git.jami.net/savoirfairelinux/ring-client-gnome/issues/1123
I have not much experience with debugging and reading backtraces, but
could it be there's something wrong with our glibc package?



Jan Wielkiewicz

Re: Packaging Jami progress

[Ricardo Wurmus]

Jan <tona_kosmicznego_smiecia@interia.pl> writes:

> Bad news:
> I still haven't got any response that would solve the bug present only
> in our package:
> https://git.jami.net/savoirfairelinux/ring-client-gnome/issues/1123
> I have not much experience with debugging and reading backtraces, but
> could it be there's something wrong with our glibc package?

That’s very unlikely as we would probably see errors like this in most
packages then.

cogl issues an optimized instruction (__memcpy_ssse3), which then fails.

I’m just guessing, but I wonder if this comment is a hint at what’s
wrong here:

    ;; NOTE: mutter exports a bundled fork of cogl, so when making changes to
    ;; cogl, corresponding changes may be appropriate in mutter as well.

cogl is up to date, but mutter is not (it’s tied to the current Gnome
version in Guix, which is a few releases behind).

It may also be interesting to see if this can be reproduced on Wayland.

--
Ricardo

Re: Packaging Jami progress

[Jan]

On Fri, 03 Jan 2020 07:33:38 +0100
Ricardo Wurmus <rekado@elephly.net> wrote:

> That’s very unlikely as we would probably see errors like this in most
> packages then.
> 
> cogl issues an optimized instruction (__memcpy_ssse3), which then
> fails.
> 
> I’m just guessing, but I wonder if this comment is a hint at what’s
> wrong here:
> 
>     ;; NOTE: mutter exports a bundled fork of cogl, so when making
> changes to ;; cogl, corresponding changes may be appropriate in
> mutter as well.
> 
> cogl is up to date, but mutter is not (it’s tied to the current Gnome
> version in Guix, which is a few releases behind).
> 
> It may also be interesting to see if this can be reproduced on
> Wayland.
> 
> --
> Ricardo
> 

I'll check it on wayland then. Anyway the bug is also present in our
current version, can I send paches containing my current changes then?


Jan Wielkiewicz

Re: Packaging Jami progress

[Pierre Neidhardt]

[-- Attachment #1: Type: text/plain, Size: 60 bytes --]

Please do!

-- 
Pierre Neidhardt
https://ambrevar.xyz/

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

Re: Packaging Jami progress

[Jan]

Dnia 2020-01-04, o godz. 16:37:53
Pierre Neidhardt <mail@ambrevar.xyz> napisał(a):

> Please do!
> 

Okay, don't know what was that, but I got an smtp error 250 during
sending patches aaand the mailing list looks like a mess now. I closed
four visible issues (out of 22), but don't know what's with the rest.
I think the tutorial for sending patches should be in a step-by-step
form.

Jan Wielkiewicz

Re: Packaging Jami progress

[Jan]

I just closed all 22 issues that I improperly started. How should I
send the patches to create just one issue? When git send-email asks me
if I want to send the patches, should I send only one and then wait?
How long? I tried sending the rest a few seconds later, but it didn't
work as indended. Or should I send one mail to guix-patches@gnu.org and
then send the rest to NNN@debbugs.gnu.org?


Jan Wielkiewicz

Re: Packaging Jami progress

[Jack Hill]

On Mon, 6 Jan 2020, Jan wrote:

> Or should I send one mail to guix-patches@gnu.org and
> then send the rest to NNN@debbugs.gnu.org?

Yes, I believe that sending to guix-patches first and then to NNN@debbugs 
is the recommended way.

Best,
Jack

Re: Packaging Jami progress

[zimoun]

Hi Jan,

Thank you for working on Jami. Cool!


On Mon, 6 Jan 2020 at 20:09, Jan <tona_kosmicznego_smiecia@interia.pl> wrote:
>
> I just closed all 22 issues that I improperly started. How should I
> send the patches to create just one issue? When git send-email asks me
> if I want to send the patches, should I send only one and then wait?
> How long? I tried sending the rest a few seconds later, but it didn't
> work as indended. Or should I send one mail to guix-patches@gnu.org and
> then send the rest to NNN@debbugs.gnu.org?


I copy/paste what I wrote here [1], waiting to improve the manual or
the cookbook.


--8<---------------cut here---------------start------------->8---
The next step is to commit the changes. In this case, three commits
(one per package) seem nice. Give a look to previous commits as
example of commit message (ChangeLog format, etc.). Now, it is time to
prepare the submission:

  git format-patch --cover-letter -o patches master

this will create the 3 patches in the folder patches/ and one cover
letter. Edit the cover letter to describe what the patches are about
then submit it to the bug tracker:

  git send-email --to=guix-patches@gnu.org patches/0000-cover-letter.patch

Wait the answer to the bug tracker.
You should receive an email (if your .gitconfig is ok) with the bug
number. Last submit the patches:

  git send-email --to=ABCDEF@debbugs.gnu.org patches/000{1,2,3}-*

where ABCDEF is the bug number.
--8<---------------cut here---------------stop------------->8---

You can also run:

  rm patches/0000-cover-letter.patch
  git send-email --to=ABCDEF@debbugs.gnu.org patches/00*

to send all the patches to the bug number ABCDEF. Be careful to not
send twice the same patch. :-)
Then you can also send normal email to ABCDEF@debbugs.gnu.org to
discuss the patch.
And another version (say version 2) of the patch series is produced
with the git option --reroll-count=2 (in short -v2):

  git rebase -i / git amend / do your change
  git format-patch -v2
  git send-email --to=ABCDEF@debbugs.gnu.org patches/v2-000*


Note that you need to "guix install git:send-email".


Hope that helps.

All the best,
simon

[1] https://lists.gnu.org/archive/html/guix-devel/2019-10/msg00093.html

Re: Packaging Jami progress

[Pierre Neidhardt]

[-- Attachment #1: Type: text/plain, Size: 258 bytes --]

Of course it'd be ideal if you can get the email workflow to work, but
if anything else fails, attach the patch set to an email (maybe a new
thread considering this one is getting way too big) :)

Cheers!

-- 
Pierre Neidhardt
https://ambrevar.xyz/

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

Re: Packaging Jami progress

[Jan Wielkiewicz]

I managed to debug jami client properly, the cause of the gdb error was
the "jami-gnome" file, which is acutally just a bash script, exporting
the path:

#!/gnu/store/29jhbbg1hf557x8j53f9sxd9imlmf02a-bash-minimal-5.0.7/bin/bash
export LD_LIBRARY_PATH="/gnu/store/dha6b5g3kjqw2vfdbhv43sfnpa5d0m5v-sqlite-with-column-metadata-3.28.0/lib${LD_LIBRARY_PATH:+:}$LD_LIBRARY_PATH"
exec -a "$0" "/gnu/store/ab4wy29g68vgvcjv1x54bm4yklrvbw2q-jami-20191224.1.5c0154a/bin/.jami-gnome-real" "$@"

I had to paste the export line before debugging using gdb, and
everything went well. I'm confused with this. Is it also the reason of
the bug, from Pierre's comment? I think we should probably get rid of
"jami-gnome" file. My brain is spaghetti now, thanks to this error :P


Jan Wielkiewicz

Re: Packaging Jami progress

[Pierre Neidhardt]

[-- Attachment #1: Type: text/plain, Size: 777 bytes --]

Allow me to explain a little more:  

Binaries embed a value called RPATH which points to the locations where
to load dynamic libraries (also called "shared objects").

When building a binary, Guix automatically sets the RPATH to that of the
required inputs.

Jami (indirectly) depends on both sqlite and sqlite-with-column-metadata.

When the binary is started, the loader finds both "sqlite" shared
objects in the RPATH, but it's not very clear which is one is loaded
first.  Hence my comment.

The wrapper aims to fix this issue by prepending
sqlite-with-column-metadata to LD_LIBRARY_PATH, which has higher
priority than the RPATH.  This makes sure the right library is loaded.

Does that make more sense?

-- 
Pierre Neidhardt
https://ambrevar.xyz/

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

Re: Packaging Jami progress

[Jan]

On Sat, 28 Dec 2019 10:53:33 +0100
Pierre Neidhardt <mail@ambrevar.xyz> wrote:

> Allow me to explain a little more:  
> 
> Binaries embed a value called RPATH which points to the locations
> where to load dynamic libraries (also called "shared objects").
> 
> When building a binary, Guix automatically sets the RPATH to that of
> the required inputs.
> 
> Jami (indirectly) depends on both sqlite and
> sqlite-with-column-metadata.
> 
> When the binary is started, the loader finds both "sqlite" shared
> objects in the RPATH, but it's not very clear which is one is loaded
> first.  Hence my comment.
> 
> The wrapper aims to fix this issue by prepending
> sqlite-with-column-metadata to LD_LIBRARY_PATH, which has higher
> priority than the RPATH.  This makes sure the right library is loaded.
> 
> Does that make more sense?
> 

Yes, this is understandable.



Jan Wielkiewicz

Re: Packaging Jami progress

[Jan Wielkiewicz]

[-- Attachment #1: Type: text/plain, Size: 61 bytes --]

Forgot to add the error file.
Here it goes.

Jan Wielkiewicz

[-- Attachment #2: lavf-mov_rtphint.err --]
[-- Type: application/octet-stream, Size: 6583 bytes --]

ffmpeg version 4.2.git Copyright (c) 2000-2019 the FFmpeg developers
  built with gcc 7.4.0 (GCC)
  configuration: --prefix=/gnu/store/bgrmfv2sh04hkf4jnnwnww9b6xr40pf7-ffmpeg-jami-4.2.1 --extra-ldflags='-Wl,-rpath=/gnu/store/bgrmfv2sh04hkf4jnnwnww9b6xr40pf7-ffmpeg-jami-4.2.1/lib' --enable-avresample --enable-gpl --enable-shared --enable-frei0r --enable-fontconfig --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libcaca --enable-libcdio --enable-libdav1d --enable-libfreetype --enable-libmp3lame --enable-libopus --enable-libpulse --enable-libsoxr --enable-libspeex --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libxvid --enable-libx264 --enable-libx265 --enable-openal --enable-opengl --enable-libdrm --enable-runtime-cpudetect --disable-htmlpages --disable-static --disable-mips32r2 --disable-mipsdsp --disable-mipsdspr2 --disable-mipsfpu
  libavutil      56. 33.100 / 56. 33.100
  libavcodec     58. 55.101 / 58. 55.101
  libavformat    58. 31.104 / 58. 31.104
  libavdevice    58.  9.100 / 58.  9.100
  libavfilter     7. 58.101 /  7. 58.101
  libavresample   4.  0.  0 /  4.  0.  0
  libswscale      5.  6.100 /  5.  6.100
  libswresample   3.  6.100 /  3.  6.100
  libpostproc    55.  6.100 / 55.  6.100
Input #0, image2, from '/tmp/guix-build-ffmpeg-jami-4.2.1.drv-0/source/tests/vsynth1/%02d.pgm':
  Duration: 00:00:02.00, start: 0.000000, bitrate: N/A
    Stream #0:0: Video: pgmyuv, yuv420p, 352x288, 25 fps, 25 tbr, 25 tbn, 25 tbc
[s16le @ 0x47d640] Estimating duration from bitrate, this may be inaccurate
Guessed Channel Layout for Input Stream #1.0 : mono
Input #1, s16le, from '/tmp/guix-build-ffmpeg-jami-4.2.1.drv-0/source/tests/data/asynth1.sw':
  Duration: 00:00:12.00, bitrate: 705 kb/s
    Stream #1:0: Audio: pcm_s16le, 44100 Hz, mono, s16, 705 kb/s
Codec AVOption idct (select IDCT implementation) specified for input file #1 (/tmp/guix-build-ffmpeg-jami-4.2.1.drv-0/source/tests/data/asynth1.sw) has not been used for any stream. The most likely reason is either wrong type (e.g. a video option with no video streams) or that it is a private option of some decoder which was not actually used for any stream.
Stream mapping:
  Stream #0:0 -> #0:0 (pgmyuv (native) -> mpeg4 (native))
  Stream #1:0 -> #0:1 (pcm_s16le (native) -> pcm_alaw (native))
Output #0, mov, to '/tmp/guix-build-ffmpeg-jami-4.2.1.drv-0/source/tests/data/lavf/lavf.mov_rtphint':
  Metadata:
    title           : lavftest
    Stream #0:0: Video: mpeg4 (mp4v / 0x7634706D), yuv420p(progressive), 352x288, q=2-31, 200 kb/s, 25 fps, 12800 tbn, 25 tbc
    Metadata:
      encoder         : Lavc mpeg4
    Side data:
      cpb: bitrate max/min/avg: 0/0/200000 buffer size: 0 vbv_delay: 18446744073709551615
    Stream #0:1: Audio: pcm_alaw (alaw / 0x77616C61), 44100 Hz, mono, s16, 352 kb/s
    Metadata:
      encoder         : Lavc pcm_alaw
[image2 @ 0x476500] Thread message queue blocking; consider raising the thread_queue_size option (current value: 8)
frame=   25 fps=0.0 q=10.0 Lsize=     362kB time=00:00:01.00 bitrate=2967.0kbits/s speed=9.38x    
video:304kB audio:43kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 4.427094%
./tests/fate-run.sh: line 299: test: =: unary operator expected
ffmpeg version 4.2.git Copyright (c) 2000-2019 the FFmpeg developers
  built with gcc 7.4.0 (GCC)
  configuration: --prefix=/gnu/store/bgrmfv2sh04hkf4jnnwnww9b6xr40pf7-ffmpeg-jami-4.2.1 --extra-ldflags='-Wl,-rpath=/gnu/store/bgrmfv2sh04hkf4jnnwnww9b6xr40pf7-ffmpeg-jami-4.2.1/lib' --enable-avresample --enable-gpl --enable-shared --enable-frei0r --enable-fontconfig --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libcaca --enable-libcdio --enable-libdav1d --enable-libfreetype --enable-libmp3lame --enable-libopus --enable-libpulse --enable-libsoxr --enable-libspeex --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libxvid --enable-libx264 --enable-libx265 --enable-openal --enable-opengl --enable-libdrm --enable-runtime-cpudetect --disable-htmlpages --disable-static --disable-mips32r2 --disable-mipsdsp --disable-mipsdspr2 --disable-mipsfpu
  libavutil      56. 33.100 / 56. 33.100
  libavcodec     58. 55.101 / 58. 55.101
  libavformat    58. 31.104 / 58. 31.104
  libavdevice    58.  9.100 / 58.  9.100
  libavfilter     7. 58.101 /  7. 58.101
  libavresample   4.  0.  0 /  4.  0.  0
  libswscale      5.  6.100 /  5.  6.100
  libswresample   3.  6.100 /  3.  6.100
  libpostproc    55.  6.100 / 55.  6.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '/tmp/guix-build-ffmpeg-jami-4.2.1.drv-0/source/tests/data/lavf/lavf.mov_rtphint':
  Metadata:
    major_brand     : qt  
    minor_version   : 512
    compatible_brands: qt  
    title           : lavftest
  Duration: 00:00:01.00, start: 0.000000, bitrate: 2967 kb/s
    Stream #0:0: Video: mpeg4 (Simple Profile) (mp4v / 0x7634706D), yuv420p, 352x288 [SAR 1:1 DAR 11:9], 2488 kb/s, 25 fps, 25 tbr, 12800 tbn, 25 tbc (default)
    Metadata:
      handler_name    : VideoHandler
      encoder         : Lavc mpeg4
    Stream #0:1: Audio: pcm_alaw (alaw / 0x77616C61), 44100 Hz, mono, s16, 352 kb/s (default)
    Metadata:
      handler_name    : SoundHandler
    Stream #0:2(eng): Data: none (rtp  / 0x20707472), 85 kb/s
    Metadata:
      handler_name    : HintHandler
    Stream #0:3(eng): Data: none (rtp  / 0x20707472), 16 kb/s
    Metadata:
      handler_name    : HintHandler
Stream mapping:
  Stream #0:0 -> #0:0 (mpeg4 (native) -> rawvideo (native))
  Stream #0:1 -> #0:1 (pcm_alaw (native) -> pcm_s16le (native))
Output #0, crc, to '/tmp/guix-build-ffmpeg-jami-4.2.1.drv-0/source/tests/data/lavf-mov_rtphint.lavf.crc':
  Metadata:
    major_brand     : qt  
    minor_version   : 512
    compatible_brands: qt  
    title           : lavftest
    encoder         : Lavf58.31.104
    Stream #0:0: Video: rawvideo (I420 / 0x30323449), yuv420p(progressive), 352x288 [SAR 1:1 DAR 11:9], q=2-31, 30412 kb/s, 25 fps, 25 tbn, 25 tbc (default)
    Metadata:
      handler_name    : VideoHandler
      encoder         : Lavc58.55.101 rawvideo
    Stream #0:1: Audio: pcm_s16le, 44100 Hz, mono, s16, 705 kb/s (default)
    Metadata:
      handler_name    : SoundHandler
      encoder         : Lavc58.55.101 pcm_s16le
frame=   25 fps=0.0 q=-0.0 Lsize=       0kB time=00:00:01.00 bitrate=   0.1kbits/s speed=38.3x    
video:3712kB audio:86kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown