How to build GNUTLS Guile bindings on Xubuntu

How to build GNUTLS Guile bindings on Xubuntu

[Zelphir Kaltstahl]

Hi,

I read in the docs at
https://www.gnu.org/software/guile/manual/html_node/Web-Client.html that
Guile will dynamically make use of GNUTLS for HTTPS requests. However, I
don't seem to have the bindings installed. There is a link to a guide on
the same docs page, but the link is dead:
https://www.gnu.org/software/guile/manual/gnutls-guile/Guile-Preparations.html#Guile-Preparations
. More searching showed me the following:

There seems to have been some bug which prevented some tests on Debian
from completing:

Bug discussions seem to indicate the bug was fixed?:

https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=821457
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=805863

Then there is a commit removing Guile bindings from the gnutls-dev
package on Debian:

https://salsa.debian.org/gnutls-team/gnutls/commit/ebb7130b47dc08311c1de2c189758a73bbaeca27#58ef006ab62b83b4bec5d81fe5b32c3b4c2d1cc2_35_35


Discussion about adding it again:

https://groups.google.com/forum/#!topic/linux.debian.bugs.dist/3DTLVkTvME8 

Apart from that, I could not find a download link or repository of the
bindings, like I could for other libraries I so far used in Guile and
run configure make make install in it. So I do not know how to get the
bindings for my system, so that my Guile will be able to make requests
using HTTPS as well as HTTP.

The bindings seem to be available on GUIX:

https://gitlab.digitalcourage.de/htgoebel/guix/commit/1dbe3a8db0a3e5a8e5f9b30e6f6a6bbfb699275b

How can I make it work on a Xubuntu 18.04?

Regards,

Zelphir

Re: How to build GNUTLS Guile bindings on Xubuntu

[Mark H Weaver]

Hi Zelphir,

Zelphir Kaltstahl <zelphirkaltstahl@gmail.com> writes:

> I read in the docs at
> https://www.gnu.org/software/guile/manual/html_node/Web-Client.html that
> Guile will dynamically make use of GNUTLS for HTTPS requests. However, I
> don't seem to have the bindings installed. There is a link to a guide on
> the same docs page, but the link is dead:
> https://www.gnu.org/software/guile/manual/gnutls-guile/Guile-Preparations.html#Guile-Preparations
> . More searching showed me the following:
>
> There seems to have been some bug which prevented some tests on Debian
> from completing:
>
> Bug discussions seem to indicate the bug was fixed?:
>
> https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=821457
> https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=805863
>
> Then there is a commit removing Guile bindings from the gnutls-dev
> package on Debian:
>
> https://salsa.debian.org/gnutls-team/gnutls/commit/ebb7130b47dc08311c1de2c189758a73bbaeca27#58ef006ab62b83b4bec5d81fe5b32c3b4c2d1cc2_35_35

If you look closely, those two bugs were "fixed" by the above commit,
i.e. by simply disabling the Guile bindings in Debian's GnuTLS package.

> Apart from that, I could not find a download link or repository of the
> bindings, like I could for other libraries I so far used in Guile and
> run configure make make install in it. So I do not know how to get the
> bindings for my system, so that my Guile will be able to make requests
> using HTTPS as well as HTTP.

The bindings in question are not a separate package.  They are built and
installed by GnuTLS itself, unless explicitly disabled by passing
"--disable-guile" to the GnuTLS configure script, as Debian now does.

> The bindings seem to be available on GUIX:
>
> https://gitlab.digitalcourage.de/htgoebel/guix/commit/1dbe3a8db0a3e5a8e5f9b30e6f6a6bbfb699275b

Note that the repository above is not the official Guix repository, but
rather someone's personal variant.  Here's the same commit in the
official Guix repository:

  https://git.sv.gnu.org/cgit/guix.git/commit/?id=1dbe3a8db0a3e5a8e5f9b30e6f6a6bbfb699275b

> How can I make it work on a Xubuntu 18.04?

Yes, an easy solution would be to install Guix on top of Xubuntu, and
installing the 'guile' and 'gnutls' packages from Guix.  Note that the
binaries from Guix are self-contained and independent of the host
system, in the sense that they are installed in different directories
(within /gnu/store) and don't use any libraries from the host system,
not even the C library.  Guix installs only in /gnu, /var/guix,
/var/log/guix, /etc/guix, ~/.config/guix, and ~/.guix-profile.  You must
set your environment variables to point within ~/.guix-profile, e.g. add
~/.guix-profile/bin to your PATH, to enable the use of software from
Guix.  If you'd like to try it, the instructions are here:

  https://www.gnu.org/software/guix/manual/en/html_node/Installation.html

      Mark

Re: How to build GNUTLS Guile bindings on Xubuntu

[Zelphir Kaltstahl]

Hi Mark,

I took another look at the page you linked to. The issue is, that I
would like to try Guix package manager to install GNUTLS + Guile
bindings, but Guix itself has the requirement of GNUTLS + Guile
bindings. See:
https://www.gnu.org/software/guix/manual/en/html_node/Requirements.html#Requirements

So I guess I would need to somehow get GNUTLS and Guile bindings working
before getting Guix and then when I have Guix, I can probably somehow
link to the Guix installed GNUTLS and Guile bindings.

Now I don't know where I would get the Guile bindings from. Are they
included in the releases on https://gnutls.org/ ?

Regards,

Zelphir

On 15.11.18 08:28, Mark H Weaver wrote:
> Hi Zelphir,
>
> Zelphir Kaltstahl <zelphirkaltstahl@gmail.com> writes:
>
>> I read in the docs at
>> https://www.gnu.org/software/guile/manual/html_node/Web-Client.html that
>> Guile will dynamically make use of GNUTLS for HTTPS requests. However, I
>> don't seem to have the bindings installed. There is a link to a guide on
>> the same docs page, but the link is dead:
>> https://www.gnu.org/software/guile/manual/gnutls-guile/Guile-Preparations.html#Guile-Preparations
>> . More searching showed me the following:
>>
>> There seems to have been some bug which prevented some tests on Debian
>> from completing:
>>
>> Bug discussions seem to indicate the bug was fixed?:
>>
>> https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=821457
>> https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=805863
>>
>> Then there is a commit removing Guile bindings from the gnutls-dev
>> package on Debian:
>>
>> https://salsa.debian.org/gnutls-team/gnutls/commit/ebb7130b47dc08311c1de2c189758a73bbaeca27#58ef006ab62b83b4bec5d81fe5b32c3b4c2d1cc2_35_35
> If you look closely, those two bugs were "fixed" by the above commit,
> i.e. by simply disabling the Guile bindings in Debian's GnuTLS package.
>
>> Apart from that, I could not find a download link or repository of the
>> bindings, like I could for other libraries I so far used in Guile and
>> run configure make make install in it. So I do not know how to get the
>> bindings for my system, so that my Guile will be able to make requests
>> using HTTPS as well as HTTP.
> The bindings in question are not a separate package.  They are built and
> installed by GnuTLS itself, unless explicitly disabled by passing
> "--disable-guile" to the GnuTLS configure script, as Debian now does.
>
>> The bindings seem to be available on GUIX:
>>
>> https://gitlab.digitalcourage.de/htgoebel/guix/commit/1dbe3a8db0a3e5a8e5f9b30e6f6a6bbfb699275b
> Note that the repository above is not the official Guix repository, but
> rather someone's personal variant.  Here's the same commit in the
> official Guix repository:
>
>   https://git.sv.gnu.org/cgit/guix.git/commit/?id=1dbe3a8db0a3e5a8e5f9b30e6f6a6bbfb699275b
>
>> How can I make it work on a Xubuntu 18.04?
> Yes, an easy solution would be to install Guix on top of Xubuntu, and
> installing the 'guile' and 'gnutls' packages from Guix.  Note that the
> binaries from Guix are self-contained and independent of the host
> system, in the sense that they are installed in different directories
> (within /gnu/store) and don't use any libraries from the host system,
> not even the C library.  Guix installs only in /gnu, /var/guix,
> /var/log/guix, /etc/guix, ~/.config/guix, and ~/.guix-profile.  You must
> set your environment variables to point within ~/.guix-profile, e.g. add
> ~/.guix-profile/bin to your PATH, to enable the use of software from
> Guix.  If you'd like to try it, the instructions are here:
>
>   https://www.gnu.org/software/guix/manual/en/html_node/Installation.html
>
>       Mark

Re: How to build GNUTLS Guile bindings on Xubuntu

[Mark H Weaver]

Zelphir Kaltstahl <zelphirkaltstahl@gmail.com> writes:

> I took another look at the page you linked to. The issue is, that I
> would like to try Guix package manager to install GNUTLS + Guile
> bindings, but Guix itself has the requirement of GNUTLS + Guile
> bindings. See:
> https://www.gnu.org/software/guix/manual/en/html_node/Requirements.html#Requirements
>
> So I guess I would need to somehow get GNUTLS and Guile bindings working
> before getting Guix and then when I have Guix, I can probably somehow
> link to the Guix installed GNUTLS and Guile bindings.

If you install Guix using the binary installation method, this will not
be an issue, because the binary install tarball is self-contained, i.e.
it includes all of Guix's dependencies, including Guile, GnuTLS,
sqlite3, guile-sqlite3, guile-json, libgc, libffi, etc, all the way down
to the C library, even low-level things like the dynamic linker, all in
/gnu/store.  You should be able to unpack it on any system with a Linux
kernel on one of our supported architectures, no matter how bare its
userspace environment is, follow the remaining install steps described
on the following page, and it should just work.

https://www.gnu.org/software/guix/manual/en/html_node/Binary-Installation.html

       Mark

Re: How to build GNUTLS Guile bindings on Xubuntu

[Alex Vong]

[-- Attachment #1: Type: text/plain, Size: 4847 bytes --]

Hi Zelphir,

Zelphir Kaltstahl <zelphirkaltstahl@gmail.com> writes:

> Hi Mark,
>
> I took another look at the page you linked to. The issue is, that I
> would like to try Guix package manager to install GNUTLS + Guile
> bindings, but Guix itself has the requirement of GNUTLS + Guile
> bindings. See:
> https://www.gnu.org/software/guix/manual/en/html_node/Requirements.html#Requirements
>
> So I guess I would need to somehow get GNUTLS and Guile bindings working
> before getting Guix and then when I have Guix, I can probably somehow
> link to the Guix installed GNUTLS and Guile bindings.
>
> Now I don't know where I would get the Guile bindings from. Are they
> included in the releases on https://gnutls.org/ ?
>
As Mark has pointed out, if you use the binary installation of guix,
then you can install gnutls using guix. However, I prefer to build guix
from source. Therefore, I use the standard "configure, make, make
install" method with appropriate CPPFLAGS, CFLAGS, CXXFLAGS and LDFLAGS
(I include some hardending flags[0][1]):

  ./configure --with-included-libtasn1 --with-included-unistring --with-guile-site-dir=/usr/local/share/guile/site/2.2
  make
  make check
  make install

The above works in Debian. Apart from it, you also have to build
scheme-bytestructure, guile-git and guile-sqlite3 from source using the
same method. Besides, it is important to keep gnutls up to date (since
it is a piece of security-sensitive software).

> Regards,
>
> Zelphir
>
Cheers,
Alex

[0]: https://wiki.debian.org/Hardening
[1]: https://security.stackexchange.com/questions/24444/what-is-the-most-hardened-set-of-options-for-gcc-compiling-c-c

> On 15.11.18 08:28, Mark H Weaver wrote:
>> Hi Zelphir,
>>
>> Zelphir Kaltstahl <zelphirkaltstahl@gmail.com> writes:
>>
>>> I read in the docs at
>>> https://www.gnu.org/software/guile/manual/html_node/Web-Client.html that
>>> Guile will dynamically make use of GNUTLS for HTTPS requests. However, I
>>> don't seem to have the bindings installed. There is a link to a guide on
>>> the same docs page, but the link is dead:
>>> https://www.gnu.org/software/guile/manual/gnutls-guile/Guile-Preparations.html#Guile-Preparations
>>> . More searching showed me the following:
>>>
>>> There seems to have been some bug which prevented some tests on Debian
>>> from completing:
>>>
>>> Bug discussions seem to indicate the bug was fixed?:
>>>
>>> https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=821457
>>> https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=805863
>>>
>>> Then there is a commit removing Guile bindings from the gnutls-dev
>>> package on Debian:
>>>
>>> https://salsa.debian.org/gnutls-team/gnutls/commit/ebb7130b47dc08311c1de2c189758a73bbaeca27#58ef006ab62b83b4bec5d81fe5b32c3b4c2d1cc2_35_35
>> If you look closely, those two bugs were "fixed" by the above commit,
>> i.e. by simply disabling the Guile bindings in Debian's GnuTLS package.
>>
>>> Apart from that, I could not find a download link or repository of the
>>> bindings, like I could for other libraries I so far used in Guile and
>>> run configure make make install in it. So I do not know how to get the
>>> bindings for my system, so that my Guile will be able to make requests
>>> using HTTPS as well as HTTP.
>> The bindings in question are not a separate package.  They are built and
>> installed by GnuTLS itself, unless explicitly disabled by passing
>> "--disable-guile" to the GnuTLS configure script, as Debian now does.
>>
>>> The bindings seem to be available on GUIX:
>>>
>>> https://gitlab.digitalcourage.de/htgoebel/guix/commit/1dbe3a8db0a3e5a8e5f9b30e6f6a6bbfb699275b
>> Note that the repository above is not the official Guix repository, but
>> rather someone's personal variant.  Here's the same commit in the
>> official Guix repository:
>>
>>   https://git.sv.gnu.org/cgit/guix.git/commit/?id=1dbe3a8db0a3e5a8e5f9b30e6f6a6bbfb699275b
>>
>>> How can I make it work on a Xubuntu 18.04?
>> Yes, an easy solution would be to install Guix on top of Xubuntu, and
>> installing the 'guile' and 'gnutls' packages from Guix.  Note that the
>> binaries from Guix are self-contained and independent of the host
>> system, in the sense that they are installed in different directories
>> (within /gnu/store) and don't use any libraries from the host system,
>> not even the C library.  Guix installs only in /gnu, /var/guix,
>> /var/log/guix, /etc/guix, ~/.config/guix, and ~/.guix-profile.  You must
>> set your environment variables to point within ~/.guix-profile, e.g. add
>> ~/.guix-profile/bin to your PATH, to enable the use of software from
>> Guix.  If you'd like to try it, the instructions are here:
>>
>>   https://www.gnu.org/software/guix/manual/en/html_node/Installation.html
>>
>>       Mark

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 227 bytes --]

Re: How to build GNUTLS Guile bindings on Xubuntu

[Mark H Weaver]

Hi Alex,

Alex Vong <alexvong1995@gmail.com> writes:

> Zelphir Kaltstahl <zelphirkaltstahl@gmail.com> writes:
>
>> I took another look at the page you linked to. The issue is, that I
>> would like to try Guix package manager to install GNUTLS + Guile
>> bindings, but Guix itself has the requirement of GNUTLS + Guile
>> bindings. See:
>> https://www.gnu.org/software/guix/manual/en/html_node/Requirements.html#Requirements
>>
>> So I guess I would need to somehow get GNUTLS and Guile bindings working
>> before getting Guix and then when I have Guix, I can probably somehow
>> link to the Guix installed GNUTLS and Guile bindings.
>>
>> Now I don't know where I would get the Guile bindings from. Are they
>> included in the releases on https://gnutls.org/ ?
>>
> As Mark has pointed out, if you use the binary installation of guix,
> then you can install gnutls using guix. However, I prefer to build guix
> from source. Therefore, I use the standard "configure, make, make
> install" method with appropriate CPPFLAGS, CFLAGS, CXXFLAGS and LDFLAGS
> (I include some hardending flags[0][1]):
>
>   ./configure --with-included-libtasn1 --with-included-unistring --with-guile-site-dir=/usr/local/share/guile/site/2.2
>   make
>   make check
>   make install
>
> The above works in Debian. Apart from it, you also have to build
> scheme-bytestructure, guile-git and guile-sqlite3 from source using the
> same method. Besides, it is important to keep gnutls up to date (since
> it is a piece of security-sensitive software).

Interesting.  I haven't tried building Guix from source on Debian in a
few years, but it's not clear to me how your suggestion above addresses
the original problem that Zelphir mentioned, namely that Guix requires
the Guile bindings for GnuTLS, which are not included in Debian.  I
don't see any mention of building those bindings above.  Am I missing
something?

       Mark

Re: How to build GNUTLS Guile bindings on Xubuntu

[Alex Vong]

[-- Attachment #1: Type: text/plain, Size: 2407 bytes --]

Hello Mark,

Mark H Weaver <mhw@netris.org> writes:

> Hi Alex,
>
> Alex Vong <alexvong1995@gmail.com> writes:
>
>> Zelphir Kaltstahl <zelphirkaltstahl@gmail.com> writes:
>>
>>> I took another look at the page you linked to. The issue is, that I
>>> would like to try Guix package manager to install GNUTLS + Guile
>>> bindings, but Guix itself has the requirement of GNUTLS + Guile
>>> bindings. See:
>>> https://www.gnu.org/software/guix/manual/en/html_node/Requirements.html#Requirements
>>>
>>> So I guess I would need to somehow get GNUTLS and Guile bindings working
>>> before getting Guix and then when I have Guix, I can probably somehow
>>> link to the Guix installed GNUTLS and Guile bindings.
>>>
>>> Now I don't know where I would get the Guile bindings from. Are they
>>> included in the releases on https://gnutls.org/ ?
>>>
>> As Mark has pointed out, if you use the binary installation of guix,
>> then you can install gnutls using guix. However, I prefer to build guix
>> from source. Therefore, I use the standard "configure, make, make
>> install" method with appropriate CPPFLAGS, CFLAGS, CXXFLAGS and LDFLAGS
>> (I include some hardending flags[0][1]):
>>
>>   ./configure --with-included-libtasn1 --with-included-unistring
>> --with-guile-site-dir=/usr/local/share/guile/site/2.2
>>   make
>>   make check
>>   make install
>>
>> The above works in Debian. Apart from it, you also have to build
>> scheme-bytestructure, guile-git and guile-sqlite3 from source using the
>> same method. Besides, it is important to keep gnutls up to date (since
>> it is a piece of security-sensitive software).
>
> Interesting.  I haven't tried building Guix from source on Debian in a
> few years, but it's not clear to me how your suggestion above addresses
> the original problem that Zelphir mentioned, namely that Guix requires
> the Guile bindings for GnuTLS, which are not included in Debian.  I
> don't see any mention of building those bindings above.  Am I missing
> something?
>
Maybe what I wrote is not clear. What I mean is that since Debian build
of gnutls does not include the guile bindings, we have to build gnutls
from the source tarball ourselves.

The flag "--with-guile-site-dir=/usr/local/share/guile/site/2.2"
instructs the build script to install the guile bindings into
"/usr/local/share/guile/site/2.2".

>        Mark

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 227 bytes --]

Re: How to build GNUTLS Guile bindings on Xubuntu

[Mark H Weaver]

Hi Alex,

Alex Vong <alexvong1995@gmail.com> writes:

> Maybe what I wrote is not clear. What I mean is that since Debian build
> of gnutls does not include the guile bindings, we have to build gnutls
> from the source tarball ourselves.
>
> The flag "--with-guile-site-dir=/usr/local/share/guile/site/2.2"
> instructs the build script to install the guile bindings into
> "/usr/local/share/guile/site/2.2".

Sure.  This is reasonable, but for the sake of completeness, I'll point
out two disadvantages with this approach, namely that (1) it entails
effectively overriding Debian's GnuTLS library with your manually built
version, which potentially affects the operation of any Debian package
that links to GnuTLS and (2) it means staying on top of security updates
yourself, i.e. recompiling and installing new versions of GnuTLS or the
bundled copies of libtasn1 and libunistring when security flaws are
discovered in those versions.  Ditto for the other packages that you
build and install manually.

     Regards,
       Mark

Re: How to build GNUTLS Guile bindings on Xubuntu

[Alex Vong]

[-- Attachment #1: Type: text/plain, Size: 1564 bytes --]

Mark H Weaver <mhw@netris.org> writes:

> Hi Alex,
>
> Alex Vong <alexvong1995@gmail.com> writes:
>
>> Maybe what I wrote is not clear. What I mean is that since Debian build
>> of gnutls does not include the guile bindings, we have to build gnutls
>> from the source tarball ourselves.
>>
>> The flag "--with-guile-site-dir=/usr/local/share/guile/site/2.2"
>> instructs the build script to install the guile bindings into
>> "/usr/local/share/guile/site/2.2".
>
> Sure.  This is reasonable, but for the sake of completeness, I'll point
> out two disadvantages with this approach, namely that (1) it entails
> effectively overriding Debian's GnuTLS library with your manually built
> version, which potentially affects the operation of any Debian package
> that links to GnuTLS and (2) it means staying on top of security updates
> yourself, i.e. recompiling and installing new versions of GnuTLS or the
> bundled copies of libtasn1 and libunistring when security flaws are
> discovered in those versions.  Ditto for the other packages that you
> build and install manually.
>
Agree, while (1) seems to not causing problems for me in practice, (2)
is a very important point to keep in mind. Btw, this bug report[0]
explains why the guile bindings were removed. Although it's mark as
wontfix, should we encourage the maintainers to re-enable the guile
bindings in the experimental repository? The problems could perhaps be
solved by building the bindings with guile 2.2.

[0]: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=863147

>      Regards,
>        Mark

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 227 bytes --]

Re: How to build GNUTLS Guile bindings on Xubuntu

[Mark H Weaver]

Hi Alex,

Alex Vong <alexvong1995@gmail.com> writes:
> Btw, this bug report[0]
> explains why the guile bindings were removed. Although it's mark as
> wontfix, should we encourage the maintainers to re-enable the guile
> bindings in the experimental repository?

Based on:

  https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=863147#10

I think the Debian maintainers have made it clear that they will not
re-enable the Guile bindings until the underlying bug is fixed, which
apparently causes intermittent failures in GnuTLS's test suite when
Guile is built with certain compile flags, as described here:

  https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=805863#84

> The problems could perhaps be solved by building the bindings with
> guile 2.2.

Sure, this would be a worthwhile thing to try, but we should probably
try it ourselves before asking the Debian developers to spend time on
it.  I can't try it easily because I use GuixSD, but if there are
interested users of Debian and Guile here, it would be a great help to
try reproducing the problem described in Debian bug 805863, and seeing
if the problem can be avoided by building GnuTLS with Debian's Guile-2.2
package.

      Regards,
        Mark


> [0]: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=863147

Re: How to build GNUTLS Guile bindings on Xubuntu

[Ludovic Courtès]

Hello,

Zelphir Kaltstahl <zelphirkaltstahl@gmail.com> skribis:

> Now I don't know where I would get the Guile bindings from. Are they
> included in the releases on https://gnutls.org/ ?

Yes, the Guile bindings are part of GnuTLS itself.  See also the
instructions at
<https://gnutls.org/manual/gnutls-guile/Guile-Preparations.html>.

Now, as Mark notes, if you want to install Guix, I strongly recommend
using the binary installation tarball.

Thanks,
Ludo’.