General Question: Execution of binaries not built for GuixSD

General Question: Execution of binaries not built for GuixSD

[christoph vogelsberg]

Hello

I have a fresh installation of GuixSD and I'm keen of learning more. I struggle with a concept and don't know how to circumvent best:

Binaries built for ordinary Linux's (as, in my case, Firefox Developer Edition) expect libraries on certain standard pathes which doesn't exist on GuixSD. I found three possibilities that might work:

* use fakechroot to make my user profile a pseudo root (disadvantage: I have no experience, it seams there is no package for that)
* create symbolic links (disadvantage: new builts get a new path with a hash value and must be re-linked properly)
* install a minimal Linux and use Guix instead of GuixSD (disadvantage: this would the point where I had capitulated)

My question: is there an already elabored way, documentation and so on, for this specific problem?

so long
cvog (Christoph Vogelsberg)

Re: General Question: Execution of binaries not built for GuixSD

[Florian Paul Schmidt]

On 22.06.2016 01:21, christoph vogelsberg wrote:
> Hello
>
> I have a fresh installation of GuixSD and I'm keen of learning more. I struggle with a concept and don't know how to circumvent best:
>
> Binaries built for ordinary Linux's (as, in my case, Firefox Developer Edition) expect libraries on certain standard pathes which doesn't exist on GuixSD. I found three possibilities that might work:
>
> * use fakechroot to make my user profile a pseudo root (disadvantage: I have no experience, it seams there is no package for that)
> * create symbolic links (disadvantage: new builts get a new path with a hash value and must be re-linked properly)
> * install a minimal Linux and use Guix instead of GuixSD (disadvantage: this would the point where I had capitulated)
>
> My question: is there an already elabored way, documentation and so on, for this specific problem?

There's a tool that you can use to hack library paths in ELF binaries. I 
once packaged the binary releases of firefox and thunderbird using that 
way, but I took the repos containing the package definitions offline due 
to licensing issues. You could use those as a starting point. If you 
wish I can dig out my local copies of the repos once I get home..

Regards,
Flo

Re: General Question: Execution of binaries not built for GuixSD

[Florian Paul Schmidt]

[-- Attachment #1.1.1: Type: text/plain, Size: 1563 bytes --]

See the attached file for reference. Put it into a guix checkout under

nonfree/packages/

Does that help as a starting point?

Regards,
Flo

On 06/22/2016 12:50 PM, Florian Paul Schmidt wrote:
> 
> 
> On 22.06.2016 01:21, christoph vogelsberg wrote:
>> Hello
>>
>> I have a fresh installation of GuixSD and I'm keen of learning more. I
>> struggle with a concept and don't know how to circumvent best:
>>
>> Binaries built for ordinary Linux's (as, in my case, Firefox Developer
>> Edition) expect libraries on certain standard pathes which doesn't
>> exist on GuixSD. I found three possibilities that might work:
>>
>> * use fakechroot to make my user profile a pseudo root (disadvantage:
>> I have no experience, it seams there is no package for that)
>> * create symbolic links (disadvantage: new builts get a new path with
>> a hash value and must be re-linked properly)
>> * install a minimal Linux and use Guix instead of GuixSD
>> (disadvantage: this would the point where I had capitulated)
>>
>> My question: is there an already elabored way, documentation and so
>> on, for this specific problem?
> 
> There's a tool that you can use to hack library paths in ELF binaries. I
> once packaged the binary releases of firefox and thunderbird using that
> way, but I took the repos containing the package definitions offline due
> to licensing issues. You could use those as a starting point. If you
> wish I can dig out my local copies of the repos once I get home..
> 
> Regards,
> Flo
> 


-- 
https://fps.io

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1.1.2: mozilla.scm --]
[-- Type: text/x-scheme; name="mozilla.scm", Size: 9767 bytes --]

(define-module (nonfree packages mozilla)
  #:use-module (guix packages)
  #:use-module (guix download)
  #:use-module (guix build-system trivial)
  #:use-module (guix licenses)
  #:use-module (gnu packages base)
  #:use-module (gnu packages xorg)
  #:use-module (gnu packages elf)
  #:use-module (gnu packages gcc)
  #:use-module (gnu packages linux)  
  #:use-module (gnu packages compression)
  #:use-module (gnu packages fontutils)
  #:use-module (gnu packages glib)
  #:use-module (gnu packages gtk)
  #:use-module (gnu packages bash))

(define-public firefox
  (let*
      ((archive-base-name
        "firefox-42.0")
       (archive-name
        (string-append archive-base-name ".tar.bz2"))
       (archive-uri
        (string-append "https://download-installer.cdn.mozilla.net/pub/firefox/releases/42.0/linux-x86_64/en-US/" archive-name)))
    (package
      (name "firefox")
      (version "42.0")
      (source
       (origin
         (method url-fetch)
         (uri archive-uri)
         (sha256
          (base32
           "1mrghfdkzyxpainnqgl76m08cs6cwy4l89iamnn58q94malpq1z3"))))
      (supported-systems '("x86_64-linux"))
      (inputs
       `(("tar" ,tar)
         ("bzip2" ,bzip2)
         ("coreutils" ,coreutils)
         ("glibc" ,glibc)
         ("gcc" ,gcc-4.9 ,"lib")
         ("libstdc++" ,libstdc++-4.9)
         ("libx11" ,libx11)
         ("libxext" ,libxext)
         ("libxdamage" ,libxdamage)
         ("libxfixes" ,libxfixes)
         ("libxcomposite" ,libxcomposite)
         ("libxrender" ,libxrender)
         ("libxt" ,libxt)
         ("alsa-lib" ,alsa-lib)
         ("freetype" ,freetype)
         ("fontconfig" ,fontconfig)
         ("dbus-glib" ,dbus-glib)
         ("glib" ,glib)
         ("gtk+" ,gtk+-2)
         ("atk" ,atk)
         ("pango" ,pango)
         ("cairo" ,cairo)
         ("gdk-pixbuf" ,gdk-pixbuf)
         ("dbus" ,dbus)
         ("patchelf" ,patchelf)))
      (build-system trivial-build-system)
      (arguments
       '(#:modules
         ((guix build utils))
         #:builder
         (begin
           (use-modules (guix build utils))
           (let*
               ((out (assoc-ref %outputs "out"))
                (source (assoc-ref %build-inputs "source"))
                (tar (string-append (assoc-ref %build-inputs "tar") "/bin/tar"))
                (patchelf (string-append (assoc-ref %build-inputs "patchelf") "/bin/patchelf"))
                (ld (string-append (assoc-ref %build-inputs "glibc") "/lib/"))
                (PATH
                 (string-append
                  (assoc-ref %build-inputs "bzip2")
                  "/bin"
                  ":"
                  (assoc-ref %build-inputs "tar")
                  "/bin"
                  ))
                (RPATH
                 (string-append
                  (assoc-ref %build-inputs "freetype") "/lib" ":"
                  (assoc-ref %build-inputs "fontconfig") "/lib" ":"
                  (assoc-ref %build-inputs "libx11") "/lib" ":"
                  (assoc-ref %build-inputs "libxrender") "/lib" ":"
                  (assoc-ref %build-inputs "libxext") "/lib" ":"
                  (assoc-ref %build-inputs "libxdamage") "/lib" ":"
                  (assoc-ref %build-inputs "libxfixes") "/lib" ":"
                  (assoc-ref %build-inputs "libxt") "/lib" ":"
                  (assoc-ref %build-inputs "libxcomposite") "/lib" ":"
                  (assoc-ref %build-inputs "alsa-lib") "/lib" ":"
                  (assoc-ref %build-inputs "dbus-glib") "/lib" ":"
                  (assoc-ref %build-inputs "glib") "/lib" ":"
                  (assoc-ref %build-inputs "gtk+") "/lib" ":"
                  (assoc-ref %build-inputs "atk") "/lib" ":"
                  (assoc-ref %build-inputs "pango") "/lib" ":"
                  (assoc-ref %build-inputs "cairo") "/lib" ":"
                  (assoc-ref %build-inputs "gdk-pixbuf") "/lib" ":"
                  (assoc-ref %build-inputs "dbus") "/lib" ":"
                  (assoc-ref %build-inputs "libstdc++") "/lib" ":"
                  (assoc-ref %build-inputs "glibc") "/lib" ":"
                  (assoc-ref %build-inputs "gcc") "/lib"))
                (ln (string-append (assoc-ref %build-inputs "coreutils") "/bin/ln")))
             (mkdir-p out)
             (mkdir-p (string-append out "/bin"))
             (with-directory-excursion out
               (setenv "PATH" PATH)
               (system* tar "xf" source)
               (system* patchelf "--set-interpreter" (string-append ld "/ld-linux-x86-64.so.2") (string-append out "/firefox/firefox"))
               (system* patchelf "--set-rpath" RPATH (string-append out "/firefox/firefox"))
               (system* patchelf "--set-rpath" RPATH (string-append out "/firefox/libxul.so"))
               (system* ln "-s" (string-append out "/firefox/firefox") (string-append out "/bin/firefox")))))))
      (synopsis "Firefox")
      (description "Pretty cool")
      (home-page "http://mozilla.com/firefox")
      (license "somewhat shaky"))))

(define-public thunderbird
  (let*
      ((archive-base-name
        "thunderbird-38.4.0")
       (archive-name
        (string-append archive-base-name ".tar.bz2"))
       (archive-uri
        (string-append "http://download.cdn.mozilla.net/pub/thunderbird/releases/38.4.0/linux-x86_64/en-US/" archive-name)))
    (package
      (name "thunderbird")
      (version "38.4.0")
      (source
       (origin
         (method url-fetch)
         (uri archive-uri)
         (sha256
          (base32
           "10bpvajf4r7588pgzh7kalzanwsxcmp10wk3xkpr3nx66was7ng8"))))
      (supported-systems '("x86_64-linux"))
      (inputs
       `(("tar" ,tar)
         ("bzip2" ,bzip2)
         ("coreutils" ,coreutils)
         ("glibc" ,glibc)
         ("gcc" ,gcc-4.9 ,"lib")
         ("libstdc++" ,libstdc++-4.9)
         ("libx11" ,libx11)
         ("libxext" ,libxext)
         ("libxdamage" ,libxdamage)
         ("libxfixes" ,libxfixes)
         ("libxcomposite" ,libxcomposite)
         ("libxrender" ,libxrender)
         ("libxt" ,libxt)
         ("alsa-lib" ,alsa-lib)
         ("freetype" ,freetype)
         ("fontconfig" ,fontconfig)
         ("dbus-glib" ,dbus-glib)
         ("glib" ,glib)
         ("gtk+" ,gtk+-2)
         ("atk" ,atk)
         ("pango" ,pango)
         ("cairo" ,cairo)
         ("gdk-pixbuf" ,gdk-pixbuf)
         ("dbus" ,dbus)
         ("patchelf" ,patchelf)))
      (build-system trivial-build-system)
      (arguments
       '(#:modules
         ((guix build utils))
         #:builder
         (begin
           (use-modules (guix build utils))
           (let*
               ((out (assoc-ref %outputs "out"))
                (source (assoc-ref %build-inputs "source"))
                (tar (string-append (assoc-ref %build-inputs "tar") "/bin/tar"))
                (patchelf (string-append (assoc-ref %build-inputs "patchelf") "/bin/patchelf"))
                (ld (string-append (assoc-ref %build-inputs "glibc") "/lib/"))
                (PATH
                 (string-append
                  (assoc-ref %build-inputs "bzip2")
                  "/bin"
                  ":"
                  (assoc-ref %build-inputs "tar")
                  "/bin"
                  ))
                (RPATH
                 (string-append
                  (assoc-ref %build-inputs "freetype") "/lib" ":"
                  (assoc-ref %build-inputs "fontconfig") "/lib" ":"
                  (assoc-ref %build-inputs "libx11") "/lib" ":"
                  (assoc-ref %build-inputs "libxrender") "/lib" ":"
                  (assoc-ref %build-inputs "libxext") "/lib" ":"
                  (assoc-ref %build-inputs "libxdamage") "/lib" ":"
                  (assoc-ref %build-inputs "libxfixes") "/lib" ":"
                  (assoc-ref %build-inputs "libxt") "/lib" ":"
                  (assoc-ref %build-inputs "libxcomposite") "/lib" ":"
                  (assoc-ref %build-inputs "alsa-lib") "/lib" ":"
                  (assoc-ref %build-inputs "dbus-glib") "/lib" ":"
                  (assoc-ref %build-inputs "glib") "/lib" ":"
                  (assoc-ref %build-inputs "gtk+") "/lib" ":"
                  (assoc-ref %build-inputs "atk") "/lib" ":"
                  (assoc-ref %build-inputs "pango") "/lib" ":"
                  (assoc-ref %build-inputs "cairo") "/lib" ":"
                  (assoc-ref %build-inputs "gdk-pixbuf") "/lib" ":"
                  (assoc-ref %build-inputs "dbus") "/lib" ":"
                  (assoc-ref %build-inputs "libstdc++") "/lib" ":"
                  (assoc-ref %build-inputs "glibc") "/lib" ":"
                  (assoc-ref %build-inputs "gcc") "/lib"))
                (ln (string-append (assoc-ref %build-inputs "coreutils") "/bin/ln")))
             (mkdir-p out)
             (mkdir-p (string-append out "/bin"))
             (with-directory-excursion out
               (setenv "PATH" PATH)
               (system* tar "xf" source)
               (system* patchelf "--set-interpreter" (string-append ld "/ld-linux-x86-64.so.2") (string-append out "/thunderbird/thunderbird"))
               (system* patchelf "--set-rpath" RPATH (string-append out "/thunderbird/thunderbird"))
               (system* patchelf "--set-rpath" RPATH (string-append out "/thunderbird/libxul.so"))
               (system* ln "-s" (string-append out "/thunderbird/thunderbird") (string-append out "/bin/thunderbird")))))))
      (synopsis "Thunderbird")
      (description "Pretty cool")
      (home-page "http://mozilla.com/thunderbird")
      (license "somewhat shaky"))))


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 473 bytes --]

Re: General Question: Execution of binaries not built for GuixSD

[Thompson, David]

On Wed, Jun 22, 2016 at 9:27 AM, Florian Paul Schmidt
<mista.tapas@gmx.net> wrote:
> See the attached file for reference. Put it into a guix checkout under
>
> nonfree/packages/
>
> Does that help as a starting point?

Do the upstream Firefox and/or Thunderbird contain nonfree components?
 If so, please do not promote such software on this list.

I really really really cannot recommend this approach to packaging,
because it goes against one of the core properties of Guix:
reproducibility.  Taking someone else's binary and hacking it until it
works isn't a real solution.  The proper way to do this is to build
binaries from source that are compatible with Guix.

Thanks,

- Dave

Re: General Question: Execution of binaries not built for GuixSD

[ng0]

On 2016-06-22(09:53:34AM-0400), Thompson, David wrote:
> On Wed, Jun 22, 2016 at 9:27 AM, Florian Paul Schmidt
> <mista.tapas@gmx.net> wrote:
> > See the attached file for reference. Put it into a guix checkout under
> >
> > nonfree/packages/
> >
> > Does that help as a starting point?
>
> Do the upstream Firefox and/or Thunderbird contain nonfree components?
>  If so, please do not promote such software on this list.
>
> I really really really cannot recommend this approach to packaging,
> because it goes against one of the core properties of Guix:
> reproducibility.  Taking someone else's binary and hacking it until it
> works isn't a real solution.  The proper way to do this is to build
> binaries from source that are compatible with Guix.
>
> Thanks,
>
> - Dave

Regarding thunderbird, I would compare to what parabola does
with icedove.

I would say that it might take more time, but it's more benificial
for Guix if for example a slightly modified version of firefox lts (45.1.x)
could be packaged. This way torbrowser is just an inherit away.

https://projects.parabola.nu/abslibre.git/tree/libre/icedove
>

--
♥Ⓐ ng0
For non-prism friendly talk find me on
psyced.org / loupsycedyglgamf.onion

Re: General Question: Execution of binaries not built for GuixSD

[Florian Paul Schmidt]

[-- Attachment #1.1: Type: text/plain, Size: 1343 bytes --]

On 06/22/2016 03:53 PM, Thompson, David wrote:
> On Wed, Jun 22, 2016 at 9:27 AM, Florian Paul Schmidt
> <mista.tapas@gmx.net> wrote:
>> See the attached file for reference. Put it into a guix checkout under
>>
>> nonfree/packages/
>>
>> Does that help as a starting point?
> 
> Do the upstream Firefox and/or Thunderbird contain nonfree components?
>  If so, please do not promote such software on this list.
> 
> I really really really cannot recommend this approach to packaging,
> because it goes against one of the core properties of Guix:
> reproducibility.  Taking someone else's binary and hacking it until it
> works isn't a real solution.  The proper way to do this is to build
> binaries from source that are compatible with Guix.

I'm answering a technical question, not promoting the use of this
software, also I'm fully aware of the caveats you mentioned. I didn't
deem it worthy to reiterate them.

Even if you do consider it promotion I think it's a far stretch from an
email of a participant in a list to the guix project itself promoting
these kinds of solutions.

Also you can consider this email as a test case for the policies the
Guix project has in place :) How you want to deal with it, etc :) Have
at it (/me puts on his flame retardant suite)!

Regards,
Flo


-- 
https://fps.io


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 473 bytes --]

Re: General Question: Execution of binaries not built for GuixSD

[Thompson, David]

On Tue, Jun 21, 2016 at 7:21 PM, christoph vogelsberg
<chrissvberg@gmx.de> wrote:
> Hello
>
> I have a fresh installation of GuixSD and I'm keen of learning more. I struggle with a concept and don't know how to circumvent best:
>
> Binaries built for ordinary Linux's (as, in my case, Firefox Developer Edition) expect libraries on certain standard pathes which doesn't exist on GuixSD. I found three possibilities that might work:

To be clear, an "ordinary" GNU/Linux system is one which uses the
Filesystem Hierarchy Standard, which GuixSD deliberately does not
conform to.

> * use fakechroot to make my user profile a pseudo root (disadvantage: I have no experience, it seams there is no package for that)
> * create symbolic links (disadvantage: new builts get a new path with a hash value and must be re-linked properly)
> * install a minimal Linux and use Guix instead of GuixSD (disadvantage: this would the point where I had capitulated)
>
> My question: is there an already elabored way, documentation and so on, for this specific problem?

The solution is to build the software from source to produce a binary
that is compatible with GuixSD.  Guix has a GNU Icecat package that
may provide a good base for this.

HTH,

- Dave

General Question: What if Binary Building breaks?

[christoph vogelsberg]

Hello again GuixSD-ers

thanks for the last fast response! Now I face another question due to:

> > My question: is there an already elabored way, documentation and so
> > on, for this specific problem?
> 
> The solution is to build the software from source to produce a binary
> that is compatible with GuixSD.  Guix has a GNU Icecat package that
> may provide a good base for this.

I actually tried installing the IceCat package but my laptop got hang. It
was heavily virtualizing so I looked up hardware requirements for
building Firefox. I learned that 8GiB RAM (and therefor 64bit system) are
required. So this makes this package unbuildable for me.

Is there any solution for that case in general? I fear persons like me who
can't efford hardware like that need to wait until a prebuild is available.

Thanks in advance and thumbs up for your project!

cvog

Re: General Question: What if Binary Building breaks?

[ng0]

Hi!

On 2016-06-24(08:19:34PM+0200), christoph vogelsberg wrote:
> Hello again GuixSD-ers
>
> thanks for the last fast response! Now I face another question due to:
>
> > > My question: is there an already elabored way, documentation and so
> > > on, for this specific problem?
> >
> > The solution is to build the software from source to produce a binary
> > that is compatible with GuixSD.  Guix has a GNU Icecat package that
> > may provide a good base for this.
>
> I actually tried installing the IceCat package but my laptop got hang. It
> was heavily virtualizing so I looked up hardware requirements for
> building Firefox. I learned that 8GiB RAM (and therefor 64bit system) are
> required. So this makes this package unbuildable for me.

Maybe false, 8GiB disk space for debug build,
4GiB disk space for normal build.
There are binary substitutes for icecat (I did
not follow all of the thread so this is just
some extra info).

> Is there any solution for that case in general? I fear persons like me who
> can't efford hardware like that need to wait until a prebuild is available.

Binary substitutes.. but our build server
currently has limited disk space, so sometimes
a binary substitute is not yet available or
was removed due to garbage collection.

> Thanks in advance and thumbs up for your project!
>
> cvog
>

--
♥Ⓐ ng0
For non-prism friendly talk find me on
psyced.org / loupsycedyglgamf.onion

Re: General Question: What if Binary Building breaks?

[Leo Famulari]

On Fri, Jun 24, 2016 at 08:19:34PM +0200, christoph vogelsberg wrote:
> Hello again GuixSD-ers
> 
> thanks for the last fast response! Now I face another question due to:
> 
> > > My question: is there an already elabored way, documentation and so
> > > on, for this specific problem?
> > 
> > The solution is to build the software from source to produce a binary
> > that is compatible with GuixSD.  Guix has a GNU Icecat package that
> > may provide a good base for this.
> 
> I actually tried installing the IceCat package but my laptop got hang. It
> was heavily virtualizing so I looked up hardware requirements for
> building Firefox. I learned that 8GiB RAM (and therefor 64bit system) are
> required. So this makes this package unbuildable for me.
> 
> Is there any solution for that case in general? I fear persons like me who
> can't efford hardware like that need to wait until a prebuild is available.

We do try to keep binary substitutes from our build farm available. They
are availabe from the default URL <https://mirror.hydra.gnu.org> if you
have authorized substitutes as described in the manual, section 2.1
Binary Installation.

Otherwise, we could look into methods to make the build process less
expensive, as suggested by ng0.