[PATCH 0/3] add c-ares and aria2

[PATCH 0/3] add c-ares and aria2

[Efraim Flashner]

This patch has driven me crazy a bit. C-ares is a library for asynchronous DNS
lookups, and aria2 is a cli program to download multiple files/torrents
simultaneously, and also a git-annex dependency. C-ares builds without any
problems, but aria2 was a different story. The first problem is that the tests
seem to need network connectivity, and it tries to access files outside the
build chroot. Disabling the tests was the easy part. The second problem was that
pkg-config wouldn't find c-ares, so it kept on building without it. Looking at
patch 3, where I finally got the flags correct to bypass pkg-config and declare
where to find c-ares, I got errors right at the end of the build phase where it
couldn't find certain features of c-ares. As the patches are, patch 1 and 2 work
well, patch 3 doesn't build, but I've included it in case anyone has an idea of
something I can try. As it currently stands, nothing is currently set to depend
on c-ares, and aria2 works without it.

Efraim Flashner (3):
  gnu: Add c-ares.
  gnu: Add aria2.
  gnu: aria2: Build with c-ares.

 gnu-system.am          |   1 +
 gnu/packages/aria2.scm | 109 +++++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 110 insertions(+)
 create mode 100644 gnu/packages/aria2.scm

-- 
2.6.2

[PATCH 1/3] gnu: Add c-ares.

[Efraim Flashner]

* gnu/packages/aria2.scm: New file.
* gnu-system.am (GNU_SYSTEM_MODULES): Add it.
---
 gnu-system.am          |  1 +
 gnu/packages/aria2.scm | 59 ++++++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 60 insertions(+)
 create mode 100644 gnu/packages/aria2.scm

diff --git a/gnu-system.am b/gnu-system.am
index f140ab6..4884680 100644
--- a/gnu-system.am
+++ b/gnu-system.am
@@ -35,6 +35,7 @@ GNU_SYSTEM_MODULES =				\
   gnu/packages/aidc.scm				\
   gnu/packages/apl.scm				\
   gnu/packages/apr.scm				\
+  gnu/packages/aria2.scm			\
   gnu/packages/asciidoc.scm			\
   gnu/packages/aspell.scm			\
   gnu/packages/attr.scm				\
diff --git a/gnu/packages/aria2.scm b/gnu/packages/aria2.scm
new file mode 100644
index 0000000..86497f2
--- /dev/null
+++ b/gnu/packages/aria2.scm
@@ -0,0 +1,59 @@
+;;; GNU Guix --- Functional package management for GNU
+;;; Copyright © 2015 Efraim Flashner <efraim@flashner.co.il>
+;;;
+;;; This file is part of GNU Guix.
+;;;
+;;; GNU Guix is free software; you can redistribute it and/or modify it
+;;; under the terms of the GNU General Public License as published by
+;;; the Free Software Foundation; either version 3 of the License, or (at
+;;; your option) any later version.
+;;;
+;;; GNU Guix is distributed in the hope that it will be useful, but
+;;; WITHOUT ANY WARRANTY; without even the implied warranty of
+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+;;; GNU General Public License for more details.
+;;;
+;;; You should have received a copy of the GNU General Public License
+;;; along with GNU Guix.  If not, see <http://www.gnu.org/licenses/>.
+
+(define-module (gnu packages aria2)
+  #:use-module ((guix licenses) #:prefix license:)
+  #:use-module (guix download)
+  #:use-module (guix packages)
+  #:use-module (guix build-system gnu)
+  #:use-module (gnu packages autotools)
+  #:use-module (gnu packages pkg-config))
+
+(define-public c-ares
+  (package
+    (name "c-ares")
+    (version "1.10.0")
+    (source (origin
+              (method url-fetch)
+              (uri (string-append
+                    "https://github.com/bagder/c-ares/archive/cares-"
+                    (string-map (lambda (c) (if (char=? c #\.) #\_ c)) version)
+                    ".tar.gz"))
+              (sha256
+               (base32
+                "0cpzf7v91j6rb85pxn98k9mpw880b0hbw4axrvydvgiq043qm4gy"))))
+    (build-system gnu-build-system)
+    (arguments
+     `(#:phases
+       (modify-phases %standard-phases
+         (add-after 'unpack 'autoreconf
+                    (lambda _ (zero? (system* "autoreconf" "-vfi")))))))
+    (native-inputs
+     `(("autoconf" ,autoconf)
+       ("automake" ,automake)
+       ("libtool" ,libtool)
+       ("pkg-config" ,pkg-config)))
+    (home-page "http://c-ares.haxx.se/")
+    (synopsis "C library for asynchronous DNS requests")
+    (description
+      "C-ares is a C library that performs DNS requests and name resolution
+asynchronously.  It is intended for applications which need to perform DNS
+queries without blocking, or need to perform multiple DNS queries in parallel.
+The primary examples of such applications are servers which communicate with
+multiple clients and programs with graphical user interfaces.")
+    (license license:x11)))
-- 
2.6.2

Re: [PATCH 1/3] gnu: Add c-ares.

[Ludovic Courtès]

Efraim Flashner <efraim@flashner.co.il> skribis:

> * gnu/packages/aria2.scm: New file.
> * gnu-system.am (GNU_SYSTEM_MODULES): Add it.

[...]

> +    (source (origin
> +              (method url-fetch)
> +              (uri (string-append
> +                    "https://github.com/bagder/c-ares/archive/cares-"
> +                    (string-map (lambda (c) (if (char=? c #\.) #\_ c)) version)
> +                    ".tar.gz"))

Please use the standalone tarball from
http://c-ares.haxx.se/download/c-ares-1.10.0.tar.gz and remove the
autotools from ‘native-inputs’ and the modified phases.

> +    (license license:x11)))

There’s no top-level license file, but the license text in source files
is:

   * Permission to use, copy, modify, and distribute this
   * software and its documentation for any purpose and without
   * fee is hereby granted, provided that the above copyright
   * notice appear in all copies and that both that copyright
   * notice and this permission notice appear in supporting
   * documentation, and that the name of M.I.T. not be used in
   * advertising or publicity pertaining to distribution of the
   * software without specific, written prior permission.
   * M.I.T. makes no representations about the suitability of
   * this software for any purpose.  It is provided "as is"
   * without express or implied warranty.

This is neither <http://directory.fsf.org/wiki/License:Expat> nor
<http://directory.fsf.org/wiki/License:X11>.

Unless someone has a better idea, I would make it:

  (license:x11-style "http://c-ares.haxx.se/license.html")

Could you send an updated patch?

Thanks,
Ludo’.

[PATCH 2/3] gnu: Add aria2.

[Efraim Flashner]

* gnu/packages/aria2.scm (aria2): New variable.
---
 gnu/packages/aria2.scm | 43 ++++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 42 insertions(+), 1 deletion(-)

diff --git a/gnu/packages/aria2.scm b/gnu/packages/aria2.scm
index 86497f2..dacf4e2 100644
--- a/gnu/packages/aria2.scm
+++ b/gnu/packages/aria2.scm
@@ -22,7 +22,14 @@
   #:use-module (guix packages)
   #:use-module (guix build-system gnu)
   #:use-module (gnu packages autotools)
-  #:use-module (gnu packages pkg-config))
+  #:use-module (gnu packages compression)
+  #:use-module (gnu packages databases)
+  #:use-module (gnu packages nettle)
+  #:use-module (gnu packages tls)
+  #:use-module (gnu packages multiprecision)
+  #:use-module (gnu packages pkg-config)
+  #:use-module (gnu packages ssh)
+  #:use-module (gnu packages xml))
 
 (define-public c-ares
   (package
@@ -57,3 +64,37 @@ queries without blocking, or need to perform multiple DNS queries in parallel.
 The primary examples of such applications are servers which communicate with
 multiple clients and programs with graphical user interfaces.")
     (license license:x11)))
+
+(define-public aria2
+  (package
+    (name "aria2")
+    (version "1.19.2")
+    (source (origin
+              (method url-fetch)
+              (uri (string-append "https://github.com/tatsuhiro-t/aria2/"
+                                  "releases/download/release-" version "/"
+                                  name "-" version ".tar.xz"))
+              (sha256
+               (base32
+                "0gnm1b7yp5q6fcajz1ln2f1rv64p6dv0nz9bcwpqrkcmsinlh19n"))))
+    (build-system gnu-build-system)
+    (arguments
+     `(#:configure-flags '("--enable-libaria2")
+       #:tests? #f)) ; tests require network connectivity
+    (native-inputs
+     `(("pkg-config", pkg-config)))
+    (inputs
+     `(("gnutls" ,gnutls)
+       ("gmp" ,gmp)
+       ("libssh2" ,libssh2)
+       ("libxml2" ,libxml2)
+       ("nettle" ,nettle)
+       ("sqlite" ,sqlite)
+       ("zlib" ,zlib)))
+    (home-page "http://aria2.sourceforge.net/")
+    (synopsis "Utility for parallel downloading files")
+    (description
+      "Aria2 is a lightweight, multi-protocol & multi-source command-line
+downloadu tility.  It supports HTTP/HTTPS, FTP, SFTP, BitTorrent and Metalink.
+Aria2 can be manipulated via built-in JSON-RPC and XML-RPC interfaces.")
+    (license license:gpl2+)))
-- 
2.6.2

[PATCH 3/3] gnu: aria2: Build with c-ares.

[Efraim Flashner]

* gnu/packages/aria2.scm (aria2)[inputs]: Add c-ares.
  [arguments]: Add environmental variables for c-ares.
---
 gnu/packages/aria2.scm | 13 +++++++++++--
 1 file changed, 11 insertions(+), 2 deletions(-)

diff --git a/gnu/packages/aria2.scm b/gnu/packages/aria2.scm
index dacf4e2..debe53a 100644
--- a/gnu/packages/aria2.scm
+++ b/gnu/packages/aria2.scm
@@ -79,12 +79,21 @@ multiple clients and programs with graphical user interfaces.")
                 "0gnm1b7yp5q6fcajz1ln2f1rv64p6dv0nz9bcwpqrkcmsinlh19n"))))
     (build-system gnu-build-system)
     (arguments
-     `(#:configure-flags '("--enable-libaria2")
+     `(#:configure-flags '("--enable-libaria2" "--with-libcares")
+       #:phases
+        (modify-phases %standard-phases
+         (add-before 'configure 'with-libcares
+                     (lambda _ 
+                       (setenv "LIBCARES_LIBS" (string-append "-L" (assoc-ref
+                               %build-inputs "c-ares") "/lib"))
+                       (setenv "LIBCARES_CFLAGS" (string-append "-I" (assoc-ref
+                               %build-inputs "c-ares") "/include")))))
        #:tests? #f)) ; tests require network connectivity
     (native-inputs
      `(("pkg-config", pkg-config)))
     (inputs
-     `(("gnutls" ,gnutls)
+     `(("c-ares" ,c-ares)
+       ("gnutls" ,gnutls)
        ("gmp" ,gmp)
        ("libssh2" ,libssh2)
        ("libxml2" ,libxml2)
-- 
2.6.2

Re: [PATCH 0/3] add c-ares and aria2

[Efraim Flashner]

[-- Attachment #1: Type: text/plain, Size: 2427 bytes --]

On Mon,  2 Nov 2015 09:15:20 +0200
Efraim Flashner <efraim@flashner.co.il> wrote:

> This patch has driven me crazy a bit. C-ares is a library for asynchronous DNS
> lookups, and aria2 is a cli program to download multiple files/torrents
> simultaneously, and also a git-annex dependency. C-ares builds without any
> problems, but aria2 was a different story. The first problem is that the tests
> seem to need network connectivity, and it tries to access files outside the
> build chroot. Disabling the tests was the easy part. The second problem was that
> pkg-config wouldn't find c-ares, so it kept on building without it. Looking at
> patch 3, where I finally got the flags correct to bypass pkg-config and declare
> where to find c-ares, I got errors right at the end of the build phase where it
> couldn't find certain features of c-ares. As the patches are, patch 1 and 2 work
> well, patch 3 doesn't build, but I've included it in case anyone has an idea of
> something I can try. As it currently stands, nothing is currently set to depend
> on c-ares, and aria2 works without it.

I meant to include the error at the end of the build phase:
  CXXLD    aria2c
./.libs/libaria2.so: undefined reference to `ares_fds'
./.libs/libaria2.so: undefined reference to `ares_getsock'
./.libs/libaria2.so: undefined reference to `ares_destroy'
./.libs/libaria2.so: undefined reference to `ares_process'
./.libs/libaria2.so: undefined reference to `ares_gethostbyname'
./.libs/libaria2.so: undefined reference to `ares_library_init'
./.libs/libaria2.so: undefined reference to `ares_strerror'
./.libs/libaria2.so: undefined reference to `ares_process_fd'
./.libs/libaria2.so: undefined reference to `ares_init'
./.libs/libaria2.so: undefined reference to `ares_library_cleanup'
collect2: error: ld returned 1 exit status
Makefile:1796: recipe for target 'aria2c' failed

> Efraim Flashner (3):
>   gnu: Add c-ares.
>   gnu: Add aria2.
>   gnu: aria2: Build with c-ares.
> 
>  gnu-system.am          |   1 +
>  gnu/packages/aria2.scm | 109 +++++++++++++++++++++++++++++++++++++++++++++++++
>  2 files changed, 110 insertions(+)
>  create mode 100644 gnu/packages/aria2.scm
> 

-- 
Efraim Flashner   <efraim@flashner.co.il>   אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: [PATCH 0/3] add c-ares and aria2

[Ludovic Courtès]

Efraim Flashner <efraim@flashner.co.il> skribis:

> This patch has driven me crazy a bit. C-ares is a library for asynchronous DNS
> lookups, and aria2 is a cli program to download multiple files/torrents
> simultaneously, and also a git-annex dependency. C-ares builds without any
> problems, but aria2 was a different story. The first problem is that the tests
> seem to need network connectivity, and it tries to access files outside the
> build chroot. Disabling the tests was the easy part.

It’s cheating!  ;-)  Could you investigate and see whether/how tests
could run in the build environment?  A common problem wrt. networking in
host name lookups.  In the build environment, only “localhost” can be
found.

> The second problem was that pkg-config wouldn't find c-ares, so it
> kept on building without it.

In that case, I would do this:

  guix build -K aria2
  # … build fails
  cd /tmp/nix-build*aria2*
  source environment-variables
  pkg-config c-ares --libs

(Replace “c-ares” with the name corresponding to the actual .pc file
provided by c-ares.)

It might be that this command fails because of a missing dependency that
the .pc file requires.

Ludo’.

Re: [PATCH 0/3] add c-ares and aria2

[Efraim Flashner]

[-- Attachment #1.1: Type: text/plain, Size: 1877 bytes --]

On Tue, 03 Nov 2015 14:35:07 +0100
ludo@gnu.org (Ludovic Courtès) wrote:

> Efraim Flashner <efraim@flashner.co.il> skribis:
> 
> > This patch has driven me crazy a bit. C-ares is a library for asynchronous DNS
> > lookups, and aria2 is a cli program to download multiple files/torrents
> > simultaneously, and also a git-annex dependency. C-ares builds without any
> > problems, but aria2 was a different story. The first problem is that the tests
> > seem to need network connectivity, and it tries to access files outside the
> > build chroot. Disabling the tests was the easy part.  
> 
> It’s cheating!  ;-)  Could you investigate and see whether/how tests
> could run in the build environment?  A common problem wrt. networking in
> host name lookups.  In the build environment, only “localhost” can be
> found.
> 
> > The second problem was that pkg-config wouldn't find c-ares, so it
> > kept on building without it.  
> 
> In that case, I would do this:
> 
>   guix build -K aria2
>   # … build fails
>   cd /tmp/nix-build*aria2*
>   source environment-variables
>   pkg-config c-ares --libs
> 
> (Replace “c-ares” with the name corresponding to the actual .pc file
> provided by c-ares.)
> 
> It might be that this command fails because of a missing dependency that
> the .pc file requires.
> 
> Ludo’.

In the end rebuilding c-ares with the release tarball and not the github
tarball fixed the issues with aria2 not recognizing c-ares. I wasn't able to
get anything more out of aria2, more than it not finding the locations and
leaving me with with variables pointing to /path/to/dir and the like.

-- 
Efraim Flashner   <efraim@flashner.co.il>   אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1.2: 0001-gnu-Add-c-ares.patch --]
[-- Type: text/x-patch, Size: 3220 bytes --]

From 1f601d7707623ff21da6567eec7f57bef133c3f5 Mon Sep 17 00:00:00 2001
From: Efraim Flashner <efraim@flashner.co.il>
Date: Thu, 29 Oct 2015 16:18:36 +0200
Subject: [PATCH 1/2] gnu: Add c-ares.

* gnu/packages/aria2.scm: New file.
* gnu-system.am (GNU_SYSTEM_MODULES): Add it.
---
 gnu-system.am          |  1 +
 gnu/packages/aria2.scm | 49 +++++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 50 insertions(+)
 create mode 100644 gnu/packages/aria2.scm

diff --git a/gnu-system.am b/gnu-system.am
index 946b77e..881ee35 100644
--- a/gnu-system.am
+++ b/gnu-system.am
@@ -35,6 +35,7 @@ GNU_SYSTEM_MODULES =				\
   gnu/packages/aidc.scm				\
   gnu/packages/apl.scm				\
   gnu/packages/apr.scm				\
+  gnu/packages/aria2.scm			\
   gnu/packages/asciidoc.scm			\
   gnu/packages/aspell.scm			\
   gnu/packages/attr.scm				\
diff --git a/gnu/packages/aria2.scm b/gnu/packages/aria2.scm
new file mode 100644
index 0000000..707a3ce
--- /dev/null
+++ b/gnu/packages/aria2.scm
@@ -0,0 +1,49 @@
+;;; GNU Guix --- Functional package management for GNU
+;;; Copyright © 2015 Efraim Flashner <efraim@flashner.co.il>
+;;;
+;;; This file is part of GNU Guix.
+;;;
+;;; GNU Guix is free software; you can redistribute it and/or modify it
+;;; under the terms of the GNU General Public License as published by
+;;; the Free Software Foundation; either version 3 of the License, or (at
+;;; your option) any later version.
+;;;
+;;; GNU Guix is distributed in the hope that it will be useful, but
+;;; WITHOUT ANY WARRANTY; without even the implied warranty of
+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+;;; GNU General Public License for more details.
+;;;
+;;; You should have received a copy of the GNU General Public License
+;;; along with GNU Guix.  If not, see <http://www.gnu.org/licenses/>.
+
+(define-module (gnu packages aria2)
+  #:use-module ((guix licenses) #:prefix license:)
+  #:use-module (guix download)
+  #:use-module (guix packages)
+  #:use-module (guix build-system gnu)
+  #:use-module (gnu packages pkg-config))
+
+(define-public c-ares
+  (package
+    (name "c-ares")
+    (version "1.10.0")
+    (source (origin
+              (method url-fetch)
+              (uri (string-append
+                    "http://c-ares.haxx.se/download/" name "-" version
+                    ".tar.gz"))
+              (sha256
+               (base32
+                "1nyka87yf2jfd0y6sspll0yxwb8zi7kyvajrdbjmh4axc5s1cw1x"))))
+    (build-system gnu-build-system)
+    (native-inputs
+     `(("pkg-config" ,pkg-config)))
+    (home-page "http://c-ares.haxx.se/")
+    (synopsis "C library for asynchronous DNS requests")
+    (description
+      "C-ares is a C library that performs DNS requests and name resolution
+asynchronously.  It is intended for applications which need to perform DNS
+queries without blocking, or need to perform multiple DNS queries in parallel.
+The primary examples of such applications are servers which communicate with
+multiple clients and programs with graphical user interfaces.")
+    (license (license:x11-style "http://c-ares.haxx.se/license.html"))))
-- 
2.6.2


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1.3: 0002-gnu-Add-aria2.patch --]
[-- Type: text/x-patch, Size: 2734 bytes --]

From ef12a054927c42442ba0806caabee75be0d6f88f Mon Sep 17 00:00:00 2001
From: Efraim Flashner <efraim@flashner.co.il>
Date: Tue, 3 Nov 2015 22:05:47 +0200
Subject: [PATCH 2/2] gnu: Add aria2.

* gnu/packages/aria2.scm (aria2): New variable.
---
 gnu/packages/aria2.scm | 44 +++++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 43 insertions(+), 1 deletion(-)

diff --git a/gnu/packages/aria2.scm b/gnu/packages/aria2.scm
index 707a3ce..5a28a77 100644
--- a/gnu/packages/aria2.scm
+++ b/gnu/packages/aria2.scm
@@ -21,7 +21,14 @@
   #:use-module (guix download)
   #:use-module (guix packages)
   #:use-module (guix build-system gnu)
-  #:use-module (gnu packages pkg-config))
+  #:use-module (gnu packages compression)
+  #:use-module (gnu packages databases)
+  #:use-module (gnu packages nettle)
+  #:use-module (gnu packages tls)
+  #:use-module (gnu packages multiprecision)
+  #:use-module (gnu packages pkg-config)
+  #:use-module (gnu packages ssh)
+  #:use-module (gnu packages xml))
 
 (define-public c-ares
   (package
@@ -47,3 +54,38 @@ queries without blocking, or need to perform multiple DNS queries in parallel.
 The primary examples of such applications are servers which communicate with
 multiple clients and programs with graphical user interfaces.")
     (license (license:x11-style "http://c-ares.haxx.se/license.html"))))
+
+(define-public aria2
+  (package
+    (name "aria2")
+    (version "1.19.2")
+    (source (origin
+              (method url-fetch)
+              (uri (string-append "https://github.com/tatsuhiro-t/aria2/"
+                                  "releases/download/release-" version "/"
+                                  name "-" version ".tar.xz"))
+              (sha256
+               (base32
+                "0gnm1b7yp5q6fcajz1ln2f1rv64p6dv0nz9bcwpqrkcmsinlh19n"))))
+    (build-system gnu-build-system)
+    (arguments
+     `(#:configure-flags '("--enable-libaria2")
+       #:tests? #f)) ; tests require network connectivity
+    (native-inputs
+     `(("pkg-config", pkg-config)))
+    (inputs
+     `(("c-ares" ,c-ares)
+       ("gnutls" ,gnutls)
+       ("gmp" ,gmp)
+       ("libssh2" ,libssh2)
+       ("libxml2" ,libxml2)
+       ("nettle" ,nettle)
+       ("sqlite" ,sqlite)
+       ("zlib" ,zlib)))
+    (home-page "http://aria2.sourceforge.net/")
+    (synopsis "Utility for parallel downloading files")
+    (description
+      "Aria2 is a lightweight, multi-protocol & multi-source command-line
+downloadu tility.  It supports HTTP/HTTPS, FTP, SFTP, BitTorrent and Metalink.
+Aria2 can be manipulated via built-in JSON-RPC and XML-RPC interfaces.")
+    (license license:gpl2+)))
-- 
2.6.2


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: [PATCH 0/3] add c-ares and aria2

[Efraim Flashner]

[-- Attachment #1: Type: text/plain, Size: 2164 bytes --]

On Thu, 5 Nov 2015 11:19:53 +0200
Efraim Flashner <efraim@flashner.co.il> wrote:

> On Tue, 03 Nov 2015 14:35:07 +0100
> ludo@gnu.org (Ludovic Courtès) wrote:
> 
> > Efraim Flashner <efraim@flashner.co.il> skribis:
> >   
> > > This patch has driven me crazy a bit. C-ares is a library for asynchronous DNS
> > > lookups, and aria2 is a cli program to download multiple files/torrents
> > > simultaneously, and also a git-annex dependency. C-ares builds without any
> > > problems, but aria2 was a different story. The first problem is that the tests
> > > seem to need network connectivity, and it tries to access files outside the
> > > build chroot. Disabling the tests was the easy part.    
> > 
> > It’s cheating!  ;-)  Could you investigate and see whether/how tests
> > could run in the build environment?  A common problem wrt. networking in
> > host name lookups.  In the build environment, only “localhost” can be
> > found.
> >   
--
> > 
> > Ludo’.  
> 
> In the end rebuilding c-ares with the release tarball and not the github
> tarball fixed the issues with aria2 not recognizing c-ares. I wasn't able to
> get anything more out of aria2, more than it not finding the locations and
> leaving me with with variables pointing to /path/to/dir and the like.
> 

here's the end of the log file:

##Failure Location unknown## : Error
Test name: N5aria224LpdMessageDispatcherTestE::testSendMessage
uncaught exception of type N5aria29DlAbortExE
- Failed to set a socket option, cause: No such device

LpdMessageReceiverTest.cc:34:Assertion
Test name: N5aria222LpdMessageReceiverTestE::testReceiveMessage
assertion failed
- Expression: rcv.init("")

Failures !!!
Run: 979   Failure total: 2   Failures: 1   Errors: 1
FAIL aria2c (exit status: 1)

the way it's presented in the command line, however, is that there was 1 test, and 1 fail, making it harder to figure out what was going on.

-- 
Efraim Flashner   <efraim@flashner.co.il>   אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: [PATCH 0/3] add c-ares and aria2

[Ludovic Courtès]

Efraim Flashner <efraim@flashner.co.il> skribis:

>> In the end rebuilding c-ares with the release tarball and not the github
>> tarball fixed the issues with aria2 not recognizing c-ares. I wasn't able to
>> get anything more out of aria2, more than it not finding the locations and
>> leaving me with with variables pointing to /path/to/dir and the like.
>> 
>
> here's the end of the log file:
>
> ##Failure Location unknown## : Error
> Test name: N5aria224LpdMessageDispatcherTestE::testSendMessage
> uncaught exception of type N5aria29DlAbortExE
> - Failed to set a socket option, cause: No such device

Could you check what this test does?  The “No such device” is likely due
to our build environment not corresponding to the assumptions made by
the test wrt. available networking capabilities.

> LpdMessageReceiverTest.cc:34:Assertion
> Test name: N5aria222LpdMessageReceiverTestE::testReceiveMessage
> assertion failed
> - Expression: rcv.init("")

And this one is likely related to the one above.

TIA,
Ludo’.

Re: [PATCH 0/3] add c-ares and aria2

[Efraim Flashner]

[-- Attachment #1.1: Type: text/plain, Size: 1960 bytes --]

On Thu, 12 Nov 2015 14:05:31 +0100
ludo@gnu.org (Ludovic Courtès) wrote:

> Efraim Flashner <efraim@flashner.co.il> skribis:
> 
> >> In the end rebuilding c-ares with the release tarball and not the github
> >> tarball fixed the issues with aria2 not recognizing c-ares. I wasn't able to
> >> get anything more out of aria2, more than it not finding the locations and
> >> leaving me with with variables pointing to /path/to/dir and the like.
> >>   
> >
> > here's the end of the log file:
> >
> > ##Failure Location unknown## : Error
> > Test name: N5aria224LpdMessageDispatcherTestE::testSendMessage
> > uncaught exception of type N5aria29DlAbortExE
> > - Failed to set a socket option, cause: No such device  
> 
> Could you check what this test does?  The “No such device” is likely due
> to our build environment not corresponding to the assumptions made by
> the test wrt. available networking capabilities.

here's the failing line:
  std::shared_ptr<SocketCore> recvsock(new SocketCore(SOCK_DGRAM));
new SocketCore(SOCK_DGRAM) occurs only in LpdMessage{Dispatcher,Reciever}Test.cc and in SocketCoreTest.cc

> > LpdMessageReceiverTest.cc:34:Assertion
> > Test name: N5aria222LpdMessageReceiverTestE::testReceiveMessage
> > assertion failed
> > - Expression: rcv.init("")  
> 
> And this one is likely related to the one above.

Looking at LpdMessageReceiver.h, I'm not sure that this is connected to the other failed test, or really why it failed. Or maybe it was supposed to fail for not passing an address, or it was supposed to be commented out since the line before it was created with an address and port.

> TIA,
> Ludo’.

I'll keep staring at it, and hopefully it'll make sense soon :)

-- 
Efraim Flashner   <efraim@flashner.co.il>   אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1.2: LpdMessageDispatcherTest.cc --]
[-- Type: text/x-c++src, Size: 2603 bytes --]

#include "LpdMessageDispatcher.h"

#include <cstring>
#include <sstream>

#include <cppunit/extensions/HelperMacros.h>

#include "TestUtil.h"
#include "Exception.h"
#include "util.h"
#include "LpdMessageDispatcher.h"
#include "SocketCore.h"
#include "BtConstants.h"

namespace aria2 {

class LpdMessageDispatcherTest:public CppUnit::TestFixture {

  CPPUNIT_TEST_SUITE(LpdMessageDispatcherTest);
  CPPUNIT_TEST(testCreateLpdRequest);
  CPPUNIT_TEST(testSendMessage);
  CPPUNIT_TEST_SUITE_END();
public:
  void testCreateLpdRequest();
  void testSendMessage();
};


CPPUNIT_TEST_SUITE_REGISTRATION(LpdMessageDispatcherTest);

void LpdMessageDispatcherTest::testCreateLpdRequest()
{
  std::string infoHashString = "cd41c7fdddfd034a15a04d7ff881216e01c4ceaf";
  CPPUNIT_ASSERT_EQUAL
    (std::string("BT-SEARCH * HTTP/1.1\r\n"
                 "Host: 239.192.152.143:6771\r\n"
                 "Port: 6000\r\n"
                 "Infohash: cd41c7fdddfd034a15a04d7ff881216e01c4ceaf\r\n"
                 "\r\n\r\n"),
     bittorrent::createLpdRequest("239.192.152.143", 6771,
                                  fromHex(infoHashString), 6000));
}

void LpdMessageDispatcherTest::testSendMessage()
{
  std::shared_ptr<SocketCore> recvsock(new SocketCore(SOCK_DGRAM));
#ifdef __MINGW32__
  recvsock->bindWithFamily(LPD_MULTICAST_PORT, AF_INET);
#else // !__MINGW32__
  recvsock->bind(LPD_MULTICAST_ADDR, LPD_MULTICAST_PORT, AF_INET);
#endif // !__MINGW32__
  recvsock->joinMulticastGroup(LPD_MULTICAST_ADDR, LPD_MULTICAST_PORT, "");
  recvsock->setNonBlockingMode();

  LpdMessageDispatcher d("cd41c7fdddfd034a15a04d7ff881216e01c4ceaf", 6000,
                         LPD_MULTICAST_ADDR, LPD_MULTICAST_PORT);
  d.init("", 0, 1);

  CPPUNIT_ASSERT(d.sendMessage());

  unsigned char buf[200];

  std::pair<std::string, uint16_t> peer;
  ssize_t nbytes;
  int trycnt;
  for(trycnt = 0; trycnt < 5; ++trycnt) {
    nbytes = recvsock->readDataFrom(buf, sizeof(buf), peer);
    if(nbytes == 0) {
      util::sleep(1);
    } else {
      break;
    }
  }
  if(trycnt == 5) {
    CPPUNIT_FAIL("[TIMEOUT] No Multicast packet received.");
  }
  buf[nbytes] = '\0';
  std::stringstream temp;
  temp << "BT-SEARCH * HTTP/1.1\r\n"
       << "Host: " << LPD_MULTICAST_ADDR << ":" << LPD_MULTICAST_PORT << "\r\n"
       << "Port: " << d.getPort() << "\r\n"
       << "Infohash: " << util::toHex(d.getInfoHash()) << "\r\n"
       << "\r\n\r\n";
  CPPUNIT_ASSERT_EQUAL(temp.str(), std::string(&buf[0], &buf[nbytes]));
}

} // namespace aria2

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1.3: LpdMessageReceiverTest.cc --]
[-- Type: text/x-c++src, Size: 2758 bytes --]

#include "LpdMessageReceiver.h"

#include <cstring>

#include <cppunit/extensions/HelperMacros.h>

#include "TestUtil.h"
#include "Exception.h"
#include "util.h"
#include "LpdMessageReceiver.h"
#include "SocketCore.h"
#include "BtConstants.h"
#include "LpdMessage.h"
#include "LpdMessageDispatcher.h"
#include "Peer.h"

namespace aria2 {

class LpdMessageReceiverTest:public CppUnit::TestFixture {

  CPPUNIT_TEST_SUITE(LpdMessageReceiverTest);
  CPPUNIT_TEST(testReceiveMessage);
  CPPUNIT_TEST_SUITE_END();
public:
  void testReceiveMessage();
};


CPPUNIT_TEST_SUITE_REGISTRATION(LpdMessageReceiverTest);

void LpdMessageReceiverTest::testReceiveMessage()
{
  LpdMessageReceiver rcv(LPD_MULTICAST_ADDR, LPD_MULTICAST_PORT);
  CPPUNIT_ASSERT(rcv.init(""));

  std::shared_ptr<SocketCore> sendsock(new SocketCore(SOCK_DGRAM));
  sendsock->create(AF_INET);
  // Mingw32 build needs to set interface explicitly.
  sendsock->setMulticastInterface("");
  sendsock->setMulticastTtl(0);

  std::string infoHashString = "cd41c7fdddfd034a15a04d7ff881216e01c4ceaf";
  std::string infoHash = fromHex(infoHashString);
  std::string request =
    bittorrent::createLpdRequest(LPD_MULTICAST_ADDR, LPD_MULTICAST_PORT,
                                 infoHash,
                                 6000);

  sendsock->writeData(request.c_str(), request.size(),
                      LPD_MULTICAST_ADDR, LPD_MULTICAST_PORT);

  rcv.getSocket()->isReadable(5);
  auto msg = rcv.receiveMessage();
  CPPUNIT_ASSERT(msg);
  CPPUNIT_ASSERT_EQUAL(std::string("cd41c7fdddfd034a15a04d7ff881216e01c4ceaf"),
                       util::toHex(msg->infoHash));
  CPPUNIT_ASSERT_EQUAL((uint16_t)6000, msg->peer->getPort());

  // Bad infohash
  std::string badInfoHashString = "cd41c7fdddfd034a15a04d7ff881216e01c4ce";
  request =
    bittorrent::createLpdRequest(LPD_MULTICAST_ADDR, LPD_MULTICAST_PORT,
                                 fromHex(badInfoHashString),
                                 6000);
  sendsock->writeData(request.c_str(), request.size(),
                     LPD_MULTICAST_ADDR, LPD_MULTICAST_PORT);

  rcv.getSocket()->isReadable(5);
  msg = rcv.receiveMessage();
  CPPUNIT_ASSERT(!msg);

  // Bad port
  request =
    bittorrent::createLpdRequest(LPD_MULTICAST_ADDR, LPD_MULTICAST_PORT,
                                 infoHash,
                                 0);
  sendsock->writeData(request.c_str(), request.size(),
                     LPD_MULTICAST_ADDR, LPD_MULTICAST_PORT);

  rcv.getSocket()->isReadable(5);
  msg = rcv.receiveMessage();
  CPPUNIT_ASSERT(!msg);

  // No data available
  msg = rcv.receiveMessage();
  CPPUNIT_ASSERT(!msg);
}

} // namespace aria2

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1.4: LpdMessageReceiver.h --]
[-- Type: text/x-chdr, Size: 2723 bytes --]

/*
 * aria2 - The high speed download utility
 *
 * Copyright (C) 2010 Tatsuhiro Tsujikawa
 *
 * This program is free software; you can redistribute it and/or modify
 * it under the terms of the GNU General Public License as published by
 * the Free Software Foundation; either version 2 of the License, or
 * (at your option) any later version.
 *
 * This program is distributed in the hope that it will be useful,
 * but WITHOUT ANY WARRANTY; without even the implied warranty of
 * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 * GNU General Public License for more details.
 *
 * You should have received a copy of the GNU General Public License
 * along with this program; if not, write to the Free Software
 * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
 *
 * In addition, as a special exception, the copyright holders give
 * permission to link the code of portions of this program with the
 * OpenSSL library under certain conditions as described in each
 * individual source file, and distribute linked combinations
 * including the two.
 * You must obey the GNU General Public License in all respects
 * for all of the code used other than OpenSSL.  If you modify
 * file(s) with this exception, you may extend this exception to your
 * version of the file(s), but you are not obligated to do so.  If you
 * do not wish to do so, delete this exception statement from your
 * version.  If you delete this exception statement from all source
 * files in the program, then also delete it here.
 */
#ifndef D_LPD_MESSAGE_RECEIVER_H
#define D_LPD_MESSAGE_RECEIVER_H

#include "common.h"

#include <string>
#include <memory>

namespace aria2 {

class SocketCore;
struct LpdMessage;

class LpdMessageReceiver {
private:
  std::shared_ptr<SocketCore> socket_;
  std::string multicastAddress_;
  uint16_t multicastPort_;
  std::string localAddress_;
public:
  // Currently only IPv4 multicastAddresses are supported.
  LpdMessageReceiver
  (const std::string& multicastAddress, uint16_t multicastPort);

  ~LpdMessageReceiver();

  // No throw.
  bool init(const std::string& localAddr);

  // Receives LPD message and returns LpdMessage which contains
  // sender(peer) and infohash. If no data is available on socket,
  // returns std::shared_ptr<LpdMessage>().  If received data is bad,
  // they are just skipped.
  std::unique_ptr<LpdMessage> receiveMessage();

  const std::shared_ptr<SocketCore>& getSocket() const
  {
    return socket_;
  }

  const std::string& getLocalAddress() const
  {
    return localAddress_;
  }
};

} // namespace aria2

#endif // D_LPD_MESSAGE_RECEIVER_H

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: [PATCH 0/3] add c-ares and aria2

[Ludovic Courtès]

Efraim Flashner <efraim@flashner.co.il> skribis:

> On Thu, 12 Nov 2015 14:05:31 +0100
> ludo@gnu.org (Ludovic Courtès) wrote:
>
>> Efraim Flashner <efraim@flashner.co.il> skribis:

[...]

>> > ##Failure Location unknown## : Error
>> > Test name: N5aria224LpdMessageDispatcherTestE::testSendMessage
>> > uncaught exception of type N5aria29DlAbortExE
>> > - Failed to set a socket option, cause: No such device  
>> 
>> Could you check what this test does?  The “No such device” is likely due
>> to our build environment not corresponding to the assumptions made by
>> the test wrt. available networking capabilities.
>
> here's the failing line:
>   std::shared_ptr<SocketCore> recvsock(new SocketCore(SOCK_DGRAM));

How do you know it’s this line?

The “Failed to set a socket option” would make me think it’s about a
failing setsockopt(2) call, but it’s not supposed to return ENODEV.
socket(2) is not supposed to return ENODEV either.

Could you keep the failed build directory (with ‘-K’), and from there
run the test in ‘strace’ to see exactly what’s failing?  (With the
caveat that it may succeed outside the build environment; did you try
running ‘make check’ from the failed build tree, outside the build
environment?)

> I'll keep staring at it, and hopefully it'll make sense soon :)

It will!  :-)

Cheers,
Ludo’.

Re: [PATCH 0/3] add c-ares and aria2

[Ludovic Courtès]

Efraim Flashner <efraim@flashner.co.il> skribis:

> From 1f601d7707623ff21da6567eec7f57bef133c3f5 Mon Sep 17 00:00:00 2001
> From: Efraim Flashner <efraim@flashner.co.il>
> Date: Thu, 29 Oct 2015 16:18:36 +0200
> Subject: [PATCH 1/2] gnu: Add c-ares.
>
> * gnu/packages/aria2.scm: New file.
> * gnu-system.am (GNU_SYSTEM_MODULES): Add it.

LGTM, thanks!

Ludo’.