Firefox 52's end of life, packaging Icecat 60

Firefox 52's end of life, packaging Icecat 60

[Clément Lassieur]

Hello,

As this blog article[1] says, Firefox 52's end of life will happen on
August 28, 2018.  That is, in 47 days.  I imagine that by that time
Icecat 60 will be released, but it seems that we are pretty far from
being able to package it, because of the Rust packages that are needed.
There might be other technical difficulties that I'm not aware of,
though.

Is there any plan in this regard?  Maybe, as a first step, we should
list the required dependencies so that people can pick them up?  My
understanding of that article[2] is that the top level dependencies are
listed in Cargo.toml[3].

What is the current state of the Rust build system (cargo-build-system)?
Is there anything blocking that would prevent its use?  I'm asking
because I see very few packages in gnu/packages/rust.scm.

Thank you,
Clément

[1]: https://blog.mozilla.org/futurereleases/2018/01/11/announcing-esr60-policy-engine/
[2]: https://developer.mozilla.org/en-US/Firefox/Building_Firefox_with_Rust_code
[3]: https://dxr.mozilla.org/mozilla-central/source/toolkit/library/rust/shared/Cargo.toml

Re: Firefox 52's end of life, packaging Icecat 60

[Nils Gillmann]

Clément Lassieur transcribed 1.1K bytes:
> Hello,
> 
> As this blog article[1] says, Firefox 52's end of life will happen on
> August 28, 2018.  That is, in 47 days.  I imagine that by that time
> Icecat 60 will be released, but it seems that we are pretty far from
> being able to package it, because of the Rust packages that are needed.

As far as my experience with trying to finish newer firefox packages
goes, you can use cendor bundled crates. My idea so far was something
along thoe lines (not succesful so far):

;; (add-before 'configure 'unpatch-sh
;;   (lambda _
;;     (substitute* (list "third_party/rust/libc-0.2.24/ci/android-install-ndk.sh"
;;                        "third_party/rust/mio/ci/docker/arm-linux-androideabi/install-sdk.sh")
;;       (((string-append "^#!" (which "sh")))
;;        "#!/bin/sh"))
;;     #t))
(add-after 'patch-source-shebangs 'patch-cargo-checksums
  (lambda* (#:key inputs #:allow-other-keys)
;; third_party/rust/ has no Cargo.lock, but all the checksum.json files
;; Cargo.lock files occur outside of third_party/rust/ though.
;; (for-each
;;  (lambda (filename)
;;    (substitute* "Cargo.lock"
;;      (("(\"checksum .* = )\".*\"" all name)
;;       (string-append name "\"" ,%cargo-reference-hash "\""))))
;;  (find-files "third_party/rust" filename))
(for-each
(lambda (filename)
(use-modules (guix build cargo-build-system))
(delete-file filename)
(let* ((dir (dirname filename)))
(display (string-append
"patch-cargo-checksums: generate-checksums for "
dir "\n"))
(generate-checksums dir ,%cargo-reference-project-file)))
(find-files "third_party/rust" ".cargo-checksum.json"))
#t))


I'm more concerned about the unreliable building of rust itself.
A handful of us tried to build the new version and for some it
failed (in the testsuite), for some it succeeded.

> There might be other technical difficulties that I'm not aware of,
> though.
> 
> Is there any plan in this regard?  Maybe, as a first step, we should
> list the required dependencies so that people can pick them up?  My
> understanding of that article[2] is that the top level dependencies are
> listed in Cargo.toml[3].
> 
> What is the current state of the Rust build system (cargo-build-system)?

the inofficcial (no bug filed) guidelines: https://lists.gnu.org/archive/html/guix-devel/2017-04/msg00051.html

> Is there anything blocking that would prevent its use?  I'm asking

Depends on what you want to use, but it is far from good enough for "real"
applications (like 200 crates dependencies).

> because I see very few packages in gnu/packages/rust.scm.
> 
> Thank you,
> Clément
> 
> [1]: https://blog.mozilla.org/futurereleases/2018/01/11/announcing-esr60-policy-engine/
> [2]: https://developer.mozilla.org/en-US/Firefox/Building_Firefox_with_Rust_code
> [3]: https://dxr.mozilla.org/mozilla-central/source/toolkit/library/rust/shared/Cargo.toml
> 

Re: Firefox 52's end of life, packaging Icecat 60

[Clément Lassieur]

Pinging (one month left), and adding Mark.

Clément Lassieur <clement@lassieur.org> writes:

> Hello,
>
> As this blog article[1] says, Firefox 52's end of life will happen on
> August 28, 2018.  That is, in 47 days.  I imagine that by that time
> Icecat 60 will be released, but it seems that we are pretty far from
> being able to package it, because of the Rust packages that are needed.
> There might be other technical difficulties that I'm not aware of,
> though.
>
> Is there any plan in this regard?  Maybe, as a first step, we should
> list the required dependencies so that people can pick them up?  My
> understanding of that article[2] is that the top level dependencies are
> listed in Cargo.toml[3].
>
> What is the current state of the Rust build system (cargo-build-system)?
> Is there anything blocking that would prevent its use?  I'm asking
> because I see very few packages in gnu/packages/rust.scm.
>
> Thank you,
> Clément
>
> [1]: https://blog.mozilla.org/futurereleases/2018/01/11/announcing-esr60-policy-engine/
> [2]: https://developer.mozilla.org/en-US/Firefox/Building_Firefox_with_Rust_code
> [3]: https://dxr.mozilla.org/mozilla-central/source/toolkit/library/rust/shared/Cargo.toml

Re: Firefox 52's end of life, packaging Icecat 60

[Mark H Weaver]

Hi Clément,

Clément Lassieur <clement@lassieur.org> writes:

> As this blog article[1] says, Firefox 52's end of life will happen on
> August 28, 2018.  That is, in 47 days.  I imagine that by that time
> Icecat 60 will be released, but it seems that we are pretty far from
> being able to package it, because of the Rust packages that are needed.
> There might be other technical difficulties that I'm not aware of,
> though.
>
> Is there any plan in this regard?  Maybe, as a first step, we should
> list the required dependencies so that people can pick them up?  My
> understanding of that article[2] is that the top level dependencies are
> listed in Cargo.toml[3].
>
> What is the current state of the Rust build system (cargo-build-system)?
> Is there anything blocking that would prevent its use?  I'm asking
> because I see very few packages in gnu/packages/rust.scm.
>
> Thank you,
> Clément
>
> [1]: https://blog.mozilla.org/futurereleases/2018/01/11/announcing-esr60-policy-engine/
> [2]: https://developer.mozilla.org/en-US/Firefox/Building_Firefox_with_Rust_code
> [3]: https://dxr.mozilla.org/mozilla-central/source/toolkit/library/rust/shared/Cargo.toml

Thanks for looking into this, and for raising the issue.  If you, or
someone else, would like to take the lead on this, I would be grateful.

For now, I would suggest trying to package upstream Firefox ESR 60.
Although we cannot add Firefox itself to Guix, IceCat 60 will be almost
identical to Firefox ESR 60, so we should be able to simply drop it in
when it becomes available.

To simplify things initially, you could comment out some or all of the
patches, snippet code, configure flags, and phases which try to avoid
bundled libraries and to use system libraries instead.  However, I would
not assume that commenting *all* of that out will help.  It's possible
that some of the bundled libraries won't work as-is on Guix because of
our unusual filesystem layout, whereas our corresponding system
libraries have already been patched to address those issues.
Alternatively, any needed patches and/or substitutions from our system
libraries could be applied to the corresponding bundled libraries.

The 'link-libxul-with-libraries' phase could also be commented out
temporarily, and instead you could manually set LD_LIBRARY_PATH as
needed before launching Firefox, so that it can find the shared
libraries it needs.

Any of these temporary solutions would be fine for now.  If you run into
difficulties, I would be glad to take a look.

If you can get Firefox ESR 60 working with the above simplifications,
then I would be glad to work on avoiding the bundled libraries, adapting
the 'link-libxul-with-libraries' phase, and swapping in IceCat 60.

What do you think?

      Mark

Re: Firefox 52's end of life, packaging Icecat 60

[Clément Lassieur]

Mark H Weaver <mhw@netris.org> writes:

> Hi Clément,

[...]

> Thanks for looking into this, and for raising the issue.  If you, or
> someone else, would like to take the lead on this, I would be grateful.
>
> For now, I would suggest trying to package upstream Firefox ESR 60.
> Although we cannot add Firefox itself to Guix, IceCat 60 will be almost
> identical to Firefox ESR 60, so we should be able to simply drop it in
> when it becomes available.
>
> To simplify things initially, you could comment out some or all of the
> patches, snippet code, configure flags, and phases which try to avoid
> bundled libraries and to use system libraries instead.  However, I would
> not assume that commenting *all* of that out will help.  It's possible
> that some of the bundled libraries won't work as-is on Guix because of
> our unusual filesystem layout, whereas our corresponding system
> libraries have already been patched to address those issues.
> Alternatively, any needed patches and/or substitutions from our system
> libraries could be applied to the corresponding bundled libraries.
>
> The 'link-libxul-with-libraries' phase could also be commented out
> temporarily, and instead you could manually set LD_LIBRARY_PATH as
> needed before launching Firefox, so that it can find the shared
> libraries it needs.
>
> Any of these temporary solutions would be fine for now.  If you run into
> difficulties, I would be glad to take a look.
>
> If you can get Firefox ESR 60 working with the above simplifications,
> then I would be glad to work on avoiding the bundled libraries, adapting
> the 'link-libxul-with-libraries' phase, and swapping in IceCat 60.
>
> What do you think?

Thank you for your reply, it's very helpful.  I'm still busy working on
Cuirass and then I'll need to take some vacations.  I'll try to have a
look though, if I find some time.

Note that I have a powerful machine (it builds Firefox 52 in 8 minutes),
and I could temporarily give ssh access to someone working on
Firefox/Icecat if it helps.

Clément